We don't have a file server. The clearml conf have :
sdk.development.default_output_uri=" None "

Do I need not make changes into clearml.conf so that it doesn't ask for my credentials or is there another way around

You have 2 options:

• set credential inside cleaml.conf : i am not familiar with this and never test it.
• or setup password less ssh with public key None

(I never played with pipeline feature so I am not really sure that it works as I imagined ...)

Based on this : it feels like S3 is supported

How are you using the function update_output_model ?

Solved @<1533620191232004096:profile|NuttyLobster9> . In my case:
I need to from clearml import Task very early in the code (like first line), before importing argparse
And not calling task.connect(parser)

if you want to replace MLflow by ClearML: do it !! It's like "Should I use sandal or running shoes for my next marathon ..."
Let your user try ClearML, and I am pretty sure all of them will want to swap over !!!

you are forcing ssh with force_git_ssh_protocol: true
Have you setup ssh keys ?
If you are using ssh keys, why enable_git_ask_pass: true ?

In the web UI, in the queue/worker tab, you should see a service queue and a worker available in that queue. Otherwise the service agent is not running. Refer to John c above

you should be able to explicitly upload a file of your choice as artefact using something like this: None

most of the time, "user" would expect that clearml handle the caching by itself

Are you sure all the files needed are pushed to your git repo ?
Go to a another folder and git clone that exact branch/commit and check the files are there ?

most of people probable wont even know what that do

What should I put in there? What is the syntax for git package?

what about having 2 agents, one on each GPU, on the same machine, serving the same queue ? So that when you enqueue, which ever agent (thus GPU) available will take the new task

all good. Just wanted to know in case I missed it

I understand to from the agent, point of view, I just need to update the conf file to use new credential and new server address.

so what was the solution/hack then ?

got it

Thanks @<1523701070390366208:profile|CostlyOstrich36>

I saw that page ... but nothing about number of worker of a queue .... or did I miss it ?

or which worker is in a queue ...

we are usign mmsegmentation by the way

I understand for cleaml-agent
What I mean is that I have 2 self deployed server. I want to switch between the 2 config when running the code locally, not inside the agent

what is the command you use to run clearml-agent ?

I am not familiar with autoscaler ... are you using the paid version of Clearml ?

@<1523701087100473344:profile|SuccessfulKoala55> Yes, I am aware of that one. It build docker container ... I wanted to build without docker. Like when clearml-agent run in non-docker mode, it is already building the running env inside it caching folder structure. I was wondering if there was a way to stop that process just before it execute the task .py

So I tried:

import livsdk.livbatch

import clearml

clearml.Task.add_requirements("livsdk","

")
task = clearml.Task.init(project_name="hieu-test", task_name='base_config')

print("Done")

Which give me this list of Packages Installed:

# Python 3.10.10 (main, Mar 05 2023, 19:07:49) [GCC]

# Local modules found - skipping:
# livsdk == ../[REDACTED]/livsdk/__init__.py

Augmentor == 0.2.10
Pillow == 9.2.0
PyYAML == 6.0
albumentations == 1.2.1
azure_storage_blob == 12.1...

nice !! That is exactly what I am looking for !!

Oh, I was assuming you are passing the entire DB backups to the cloud.

Yes, that is what I want to do.
So I need to migrate both the MongoDB database and elastic search database from my local docker instance to the equivalent in the cloud ?

can you make train1.py use clearml.conf.server1 and train2.py use clearml.conf2 ?? In which case I would be intersted @<1523701087100473344:profile|SuccessfulKoala55>