Hi, CostlyOstrich36 , Sure!

i running this on 2.35 am, but the job not launching after 2.40 am

clearml-agent, if you looking for clearml.conf, the place is '/root/default_clearml.conf'

Hi CostlyOstrich36 ,
nope, i mean my server does not have pip/conda. so i will go for docker/container, is that possible if i install clearml-agent inside python:3.10 container?

i set like this: for init Task Scheduler

    task_scheduler = TaskScheduler(
        sync_frequency_minutes=5,
        force_create_task_name='controller_feedback',
        force_create_task_project='Automation/Controller',

    )

wow, so cool!
Thanks, really great tools!

remove this params will solve use_current_task=True,

Thanks! i just prove it can run in next day, but not for the same day. i hope can run in same day too.

Syncing scheduler
Waiting for next run, sleeping for 5.13 minutes, until next sync.
Launching job: ScheduleJob(name='fetch feedback', base_task_id='', base_function=<function test_make at 0x7f91fd123d90>, queue=None, target_project='Automation/testing', single_instance=False, task_parameters={}, task_overrides={}, clone_task=True, _executed_instances=None, execution_limit_hours=None, r...

Hi @<1523701205467926528:profile|AgitatedDove14> , Thanks for rresponse!
this my simple code to test scheduler

import datetime
from clearml.automation import TaskScheduler
  
def test_make():
    print('test running', datetime.datetime.now())

if __name__ == '__main__':
    task_scheduler = TaskScheduler(
        sync_frequency_minutes=30,
        force_create_task_name='controller_feedback',
        force_create_task_project='Automation/Controller',
    )

    print('\n[utc_timestamp]...

alright, will try thanks!

i feel pain to make it as form if so much varible want to changes.

i attach train.py here,
and to run it i do python src/train.py

Hi @<1523701070390366208:profile|CostlyOstrich36> , just want to update,
this is solve by

• remove

-f 

2. change Task.force_requirements_env_freeze(False, req_path) -> Task.add_requirements(req_path)
3. change my clearml-agent settings

Hi @<1523701070390366208:profile|CostlyOstrich36> , i think can try this to run it as standalone:

hmm i want to make custom function that need credential registed on clearml.conf, like aws s3.
is clearml-agent have clearml.conf? where the path for that? i just test it, running using clearml-agent, but not found /root/clearml.conf

i want to download model before i run the my inference code. i can actually make simple script using cleaml-sdk before that, but i just look for CLI based solution.

i see, it solved right now using default_output_uri, Thanks!
i need custom output_uri for some function because split dataset and model artifacs.

alright, will try, i just worried about if execution mode is docker mode? should i mount to /var/run/docker.sock?

oh okay, so i need to set that to path ssd, yeah?
is it this one? or there is

    docker_internal_mounts {
        sdk_cache: "/clearml_agent_cache"
        apt_cache: "path/to/ssd/apt-cache"
        ssh_folder: "/root/.ssh"
        pip_cache: "path/to/ssd/clearml-cache/pip"
        poetry_cache: "/mnt/hdd_2/clearml-cache/pypoetry"
        vcs_cache: "path/to/ssd/clearml-cache/vcs-cache"
        venv_build: "path/to/ssd/clearml-cache/venvs-builds"
        pip_download: "path/to/ssd/cle...

https://github.com/mert-kurttutan/torchview
maybe can try this one, and can send to logger clearml at the end.

Hi @<1523701070390366208:profile|CostlyOstrich36> , thanks for response, sorry for late replay,

this is my configuration in yaml, i facing difficulty when there is params in list. somehow, form to display bunch list not easy to see. do you have suggestion? Thanks!

download-data:
  dataset_train:
  - 

  - 

  - 

  dataset_test:
  - 

  - 

  - 

train:
  data:
    batch: 4
    input_size: 224
    split:
      t...

Hi @<1523701070390366208:profile|CostlyOstrich36>
i attach for complete log

here my structure:

.
├── app
│   ├── backend
│   └── frontend
├── assets
│   ├── demo-app-sample.png
│   └── workflow.png
├── config
│   ├── clearml.conf
│   ├── list_models.py
│   ├── list_optimizer.py
│   ├── __pycache__
│   └── train_config.py
├── docker
│   ├── Dockerfile
│   ├── Dockerfile.app
│   ├── requirements.prod.txt
│   ├── requirements.train.txt
│   └── requirements.txt
├── lightning_logs
├── Mak...

sorry, but can clearml pipeline do this scenario?

https://github.com/allegroai/clearml/issues/865#issue-1517283786

i am using dictionary, more convinient for me and can categorize each params.
Thanks, hope that feature will ready soon!

I’m running the agent in ‘pip’ mode. I need to fetch certain secret values, which would be best done using Python code. If I incorporate it into the script (repository), others could deduce the path to retrieve the environment or secret value. Storing the environment variables in the clearml.config isn’t very flexible either.

i see thankyou!

Hi AgitatedDove14 , is the Dataset.get will take all child too?

Hi, @<1523701070390366208:profile|CostlyOstrich36> ,yes! correct! how to achive that? it will save my storage.