you should be able to see int the Console tab that show what is happening

To "attach" that zip to the model, do you just use the update_weight and point to that zip file?

Please refer to here None
The doc need to be a bit clearer: one require a path and not just true/false

We need to focus first on Why is it taking minutes to reach Using env.
In our case, we have a container that have all packages installed straight in the system, no venv in the container. Thus we don't use CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
But then when a task is pulled, I can see all the steps like git clone, a bunch of Requirement already satisfied .... There may be some odd package that need to be installed because one of our DS is experimenting ... But all that we can see what is...

Nice ! That is handy !!
thanks !

do you mean having the ClearML FileServer store on azure blob instead of on the local drive?

Yes, that is what I wanted.

If so, that's not possible. You can however point the fileserver data folder to some mounted folder - if you have something that can create a mount from a filesystem folder to azure blob, it will work (the file server will always treat it as a local file system)

Thanks for confirming that it's the only solution. 👍

so what was the solution/hack then ?

all good. Just wanted to know in case I missed it

@<1523701205467926528:profile|AgitatedDove14> About why we stay on 1.12.2 : None

I didn;t know that from the client side, you can specify the storage elsewhere than the clearML server. Good to know !
But I still want to know, if possible, to use a blob storage by default, configured on the ClearML server, and each client don't need to do that ...

you are forcing ssh with force_git_ssh_protocol: true
Have you setup ssh keys ?
If you are using ssh keys, why enable_git_ask_pass: true ?

I think a proper screenshot of the full log with some information redacted is the way to go. Otherwise we are just guessing in the dark

Do you want to use https or ssh to do git clone ? Setting up both in the same time is confusing

So "Using env ..." take minutes without any output ?

About the caching: how does it work ? ClearML maintain it own cache and monitor if any of you code changes? Even code that get change inside an import ?

Thanks !!

Sure:

def main():
    repo = "redacted"
    commit = "redacted"
    commit = "redacted"
    bands = ["redacted"]
    test_size = 0.2
    batch_size = 64
    num_workers = 12
    img_size = (128, 128)
    random_seed = 42
    epoch = 20
    learning_rate = 0.1

    livbatch_list = get_livbatch_list(repo, commit)
    lbs = download_batches(repo, commit, livbatch_list)

    df, label_map = get_annotation_df(lbs, bands)

    df_train, df_val = deterministic_train_val(df, test_size=test_siz...

will do

very hard to diagnose with this tiny bit of log ...

your need both in certain case

please provide the full logs and error message.

You are using CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL the wrong way

in that case yes. What happen is that in docker mode:
you run a clearml agent, that then receive a task
create a container
install another agent inside that container
then run that second agent inside the container
that second agent then pull the task and do the usuall build/install

CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=true need to be set on that second agent somehow ...

I tried mounting azure storage account on that path and it worked: all files end up in the cloud storage

Try to set CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=true in the terminal start clearml-agent
See None

I use CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/my/vemv/bin/python3.12 and it work for me

are you using the agent docker mode ?

depend on how the agent is launched ...

I mean, depend on what do you want to report ... if you want to stick to table, I suggest earlier to gather your stats in table format ...
Otherwise, matplotlib seems to be the most user friendly way

there is a tricky thing: clearml-agent should not be running from a venv itself ... don't remember where I read that doc