if you are using a self hosted clearml server spin up with docker-compose, then you can just mount your NAS to /opt/clearml/fileserver on the host machine, prior to starting clearml server with docker-compose up

I tried mounting azure storage account on that path and it worked: all files end up in the cloud storage

not sure how that work with Docker and machine that is not set up with ssh public key ... We will go to that path sometime in the future so I am quite interested too, on how people do it without ssh public key

with ssh public key, if from a terminal, I can do git clone, then so do the clearml agent, as it run on behalf of an local user. That apply to both local and VM

I use ssh public key to access to our repo ... Never tried to provide credential to clearml itself (via clearml.conf ) so I cannot help much here ...

there is a whole discussion about it here: None

(I never played with pipeline feature so I am not really sure that it works as I imagined ...)

Hi.
How do you tell the server to use my azure storage instead of local drive, on the host machine ? Isn't it by setting azure.storage in /opt/clearml/config/clearml.conf ?

had you made sure that the agent inside GCP VM have access to your repository ? Can you ssh into that VM and try to do a git clone ?

you should be able to use as many agent as you want.
On the same or different queue

Do I need not make changes into clearml.conf so that it doesn't ask for my credentials or is there another way around

You have 2 options:

• set credential inside cleaml.conf : i am not familiar with this and never test it.
• or setup password less ssh with public key None

because when I was running both agents on my local machine everything was working perfectly fine

This is probably you (or someone) had set up ssh public key with your git repo sometime in the past

i need to do a git clone

You need to do it to test if it works. Clearml-agent will run it itself when it take in a task

yes

do you mean having the ClearML FileServer store on azure blob instead of on the local drive?

Yes, that is what I wanted.

If so, that's not possible. You can however point the fileserver data folder to some mounted folder - if you have something that can create a mount from a filesystem folder to azure blob, it will work (the file server will always treat it as a local file system)

Thanks for confirming that it's the only solution. 👍

Is this the same issue as per here ?
In which case, can you make your script run using that docker container nvidia/cuda:11.0.3-cudnn8-runtime-ubuntu20.04 , manually, without ClearML ?
What change did you do to make it worked ? Does updating python enough ?

That --docker_args seems to be for clearml-task as described here , while you are using clearml-agent which is a different thing

None

We need to focus first on Why is it taking minutes to reach Using env.
In our case, we have a container that have all packages installed straight in the system, no venv in the container. Thus we don't use CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
But then when a task is pulled, I can see all the steps like git clone, a bunch of Requirement already satisfied .... There may be some odd package that need to be installed because one of our DS is experimenting ... But all that we can see what is...

@<1523701205467926528:profile|AgitatedDove14> About why we stay on 1.12.2 : None

depend on how the agent is launched ...

Try to set CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=true in the terminal start clearml-agent
See None

your need both in certain case

are you using the agent docker mode ?

you mean this dump section ?

Can you share the agent log, in the console tab, before the error?

I use CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/my/vemv/bin/python3.12 and it work for me

I will try it. But it's a bit random when this happen so ... We will see

Do you want to use https or ssh to do git clone ? Setting up both in the same time is confusing

you are forcing ssh with force_git_ssh_protocol: true
Have you setup ssh keys ?
If you are using ssh keys, why enable_git_ask_pass: true ?