can you make train1.py use clearml.conf.server1 and train2.py use clearml.conf2 ?? In which case I would be intersted @<1523701087100473344:profile|SuccessfulKoala55>

Task.export_task() will contains what you are looking for.
In this case ['script']['diff']

I don;t think ClearML is designed to handle secrets other than git and storage ...

so it's not suppose to say "illegal output destination ..." ?

is this mongodb type of filtering?

Nice ! That is handy !!
thanks !

nevermind, all the database files are in data folder

I use ssh public key to access to our repo ... Never tried to provide credential to clearml itself (via clearml.conf ) so I cannot help much here ...

and just came across this: None
That sounds like what you may be looking for

I also use this: None
Which can give more control

None

never tried 😛

yup, you have the flexibility and option, that what so nice with ClearML

From SDK ?

You can either set your user permission to allow group write by default ?
Or maybe create a dedicated user with group write permission and run the agent with that user ?

if you want plot, you can simply generate plot with matplotlib and clearml can upload them in the Plot or Debug Sample section

We need to focus first on Why is it taking minutes to reach Using env.
In our case, we have a container that have all packages installed straight in the system, no venv in the container. Thus we don't use CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
But then when a task is pulled, I can see all the steps like git clone, a bunch of Requirement already satisfied .... There may be some odd package that need to be installed because one of our DS is experimenting ... But all that we can see what is...

Ok I think I found the issue. I had to point the file server to azure storage:

api {
    # Notice: 'host' is the api server (default port 8008), not the web server.
    api_server: 

    web_server: 

    files_server: "

"
    credentials {"access_key": "REDACTED", "secret_key": "REDACTED"}
}

Clear. Thanks @<1523701070390366208:profile|CostlyOstrich36> !

@<1523701070390366208:profile|CostlyOstrich36> Is there a way to tell clearml to not try to detect the Installed package ?

But then how did the agent know where is the venv that it needs to use?

Nevermind: None

By default, the File Server is not secured even if Web Login Authentication has been configured. Using an object storage solution that has built-in security is recommended.

My bad

--gpus 0,1 : I believe this basically say that your code launched by the agent has access to both GPUs and that is it. Now it is up to your code to choose which GPU to use and what not and how ...

I mean, depend on what do you want to report ... if you want to stick to table, I suggest earlier to gather your stats in table format ...
Otherwise, matplotlib seems to be the most user friendly way

with

    df = pd.DataFrame({'num_legs': [2, 4, 8, 0],
                       'num_wings': [2, 0, 0, 0],
                       'num_specimen_seen': [10, 2, 1, 8]},
                      index=['falcon', 'dog', 'spider', 'fish'])

    import clearml
    task = clearml.Task.current_task()
    task.get_logger().report_table(title='table example', series='pandas DataFrame', iteration=0, table_plot=df)
    # logger.report_table(title='table example',series='pandas DataFrame',iteration=0,tabl...

this looks like the agent running inside your docker did not have any username/password to do git clone. so the default behavior is to wait for keyboard input: which look like hanging ....

you can upload the df as artifact.
Or the statistics as a DataFrame and upload as artifact ?

Can you share the agent log, in the console tab, before the error?