Hi @<1523701087100473344:profile|SuccessfulKoala55> ,
We have successfully created a sample for the migration. Here are the changes:

• URL for MongoDB from s3:// to azure://
• Elasticsearch as you suggested
  However, our main focus is that most of our production fetches models from ClearML, which are configured with s3:// URLs.

There is an issue/bug in the UI when downloading via Azure. Here are the details: None .

Hi @<1523701205467926528:profile|AgitatedDove14> , Thanks for rresponse!
this my simple code to test scheduler

import datetime
from clearml.automation import TaskScheduler
  
def test_make():
    print('test running', datetime.datetime.now())

if __name__ == '__main__':
    task_scheduler = TaskScheduler(
        sync_frequency_minutes=30,
        force_create_task_name='controller_feedback',
        force_create_task_project='Automation/Controller',
    )

    print('\n[utc_timestamp]...

its auto to fileserver instance

my case more like there is a task/process that running but somehow its takes too long to completed. it can be because connection issue forgot to put connection timeout, a problem connection database, etc that makes status still running, but its traped in a situation like that.

so i want to force shutdown a task to failed if that happen

I’m running the agent in ‘pip’ mode. I need to fetch certain secret values, which would be best done using Python code. If I incorporate it into the script (repository), others could deduce the path to retrieve the environment or secret value. Storing the environment variables in the clearml.config isn’t very flexible either.

i set like this: for init Task Scheduler

    task_scheduler = TaskScheduler(
        sync_frequency_minutes=5,
        force_create_task_name='controller_feedback',
        force_create_task_project='Automation/Controller',

    )

Hi @<1523701070390366208:profile|CostlyOstrich36> , thanks for response, sorry for late replay,

this is my configuration in yaml, i facing difficulty when there is params in list. somehow, form to display bunch list not easy to see. do you have suggestion? Thanks!

download-data:
  dataset_train:
  - 

  - 

  - 

  dataset_test:
  - 

  - 

  - 

train:
  data:
    batch: 4
    input_size: 224
    split:
      t...

`

payload

{"id":["75d04598197a445ebef533814022c58d"],"include_stats":true,"check_own_contents":true,"active_users":["a174c4e36b0446a7b3b5dd1ff5261962"],"search_hidden":true}

Response

{"meta":{"id":"8d18a89599db4d899ab40959a39970b3","trx":"8d18a89599db4d899ab40959a39970b3","endpoint":{"name":"projects.get_all_ex","requested_version":"2.20","actual_version":"1.0"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"projects":[{"id":"75d0459819...

Correct! Thanks AppetizingMouse58 !

yeah, we cannot do anything to that [undefined]. i got this when i click that.

I see, yeah my alternative solution right now is just to show the list of options outside on ClearML UI.

Hi @<1523701070390366208:profile|CostlyOstrich36> , just want to update,
this is solve by

• remove

-f 

2. change Task.force_requirements_env_freeze(False, req_path) -> Task.add_requirements(req_path)
3. change my clearml-agent settings

sorry, but can clearml pipeline do this scenario?

Thanks for response.

from clearml import Task
from clearml.automation import TaskScheduler
from datetime import timedelta, datetime

def my_task():
    task = Task.init(...)
    # do somthinge
    print("do something")
    # sleep 10
    condition = True
    if condition:
        # i want to trigger run another task by 
        # set some config in task, but execute tomorrow/sometime
        # not directly run at the time.
        # here i use
        task_id = task.id 
        task.cl...

Hi @<1523701994743664640:profile|AppetizingMouse58> , i have update on #228. Thanks!

projects.get_all_ex API call

i see okay thanks

i see,
thanks for clarify. i just want to find other solutions to storing secret value. rightnow i just storing secret value on env in clearml.conf in my workers. but it will complicated if there is new value, i need update workers conf and redeploy workers.

Thanks, its works!

Hi @<1523701070390366208:profile|CostlyOstrich36>
i attach for complete log

here my structure:

.
├── app
│   ├── backend
│   └── frontend
├── assets
│   ├── demo-app-sample.png
│   └── workflow.png
├── config
│   ├── clearml.conf
│   ├── list_models.py
│   ├── list_optimizer.py
│   ├── __pycache__
│   └── train_config.py
├── docker
│   ├── Dockerfile
│   ├── Dockerfile.app
│   ├── requirements.prod.txt
│   ├── requirements.train.txt
│   └── requirements.txt
├── lightning_logs
├── Mak...

it seems only happen if i change the user preference to My Works , if set Team's Work it will show like this.

do you mean i can change?

files_server: 

 -> 

yup,
example just need choosing between SGD, Adam, AdamW on optimizer field

` {"meta":{"id":"17c6e609ace54bf8bfdf3113c39fd470","trx":"17c6e609ace54bf8bfdf3113c39fd470","endpoint":{"name":"projects.get_all_ex","requested_version":"2.20","actual_version":"1.0"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"projects":[{"id":"75d04598197a445ebef533814022c58d","company":{"id":"d1bd92a3b039400cbafc60a7a5b1e52b"},"user":{"id":"a174c4e36b0446a7b3b5dd1ff5261962"},"name":"ex-1","basename":"ex-1","description":"","created":"20...

Hi @<1523701070390366208:profile|CostlyOstrich36> , i think can try this to run it as standalone:

remove this params will solve use_current_task=True,

it seems i forgot using clearml-agent init.
i follow this way: https://clear.ml/docs/latest/docs/guides/ide/google_colab/

maybe accidently install my custom solution on this https://github.com/muhammadAgfian96/clearml/commit/01db9aa40537a6c2f83977220423556a48614c3a at that time. so i said the test is passed.

Hi, CostlyOstrich36 , Sure!

yeah, it seems i will try option 2 that you mentioned.
i upload my full my configuration, and when done using google colab my configuration will gone too.