I mean if I do CLEARML_DOCKER_IMAGE=my_image clearml-task something something it will not work, right?

My clearml-server server crashed for some reason, so I won't be able to verify until tomorrow.

It is weird though. The task is submitted by the original user and then run on the agent. The task however is still registered by the original user, since it is created by the original user.

Makes more sense to just inherit the user from the task than from the agent?

Oh you are right. I did not think this through... To implement this properly it gets to enterprisy for me, so I ll just leave it for now :D

At least when you use docker containers the agent will reuse the existing python environment.

I was wondering whether some solution is builtin in clearml, so I do not have to configure each server manually. However, from your answer I take that this is not the case.

Yea, something like this seems to be the best solution.

I have no idea myself, but what the serverfault thread says about man-in-the-middle makes sense. However this also prohibits an automatic solution except for a shared known_hosts file I guess.

I see. Thank you!

What do you mean?

Latest version for everything. I will message you again, if I encounter this problem again.

It is not explained there, but do you mean
CLEARML_API_ACCESS_KEY: ${CLEARML_API_ACCESS_KEY:-} CLEARML_API_SECRET_KEY: ${CLEARML_API_SECRET_KEY:-}?

If you compare the two outputs it put at the top of this thread, the one being the output if executed locally and the other one being the output if executed remotely, it seems like command is different and wrong on remote.

` ocker-compose ps
Name Command State Ports

clearml-agent-services /usr/agent/entrypoint.sh Restarting
clearml-apiserver /opt/clearml/wrapper.sh ap ... Up 0.0.0.0:8008->8008/tcp, 8080/tcp, 8081/tcp ...

AgitatedDove14 Thank you, that explains it.

Depends on how you start the task afaik. I think clearml-task uses requirements.txt by default, but otherwise clearml will parse your files dependencies or if you changed in clearml.conf it will use your conda/pip environment to generate the requirements.

Here is the minimal example: https://github.com/mctigger/test_clearml

Can You tell me which python version is running on the agent/docker and which docker image?

Okay, thank you anyways. I was just asking because I thought I had seen such a setting before. Must have been something different.

Perfect! That sounds like a good solution for me.

The ordering of the channels seems to matter!

conda_channels: ["defaults", "conda-forge", "pytorch", ]

No idea what's happening there.

Perfect and thank you for your efforts! :)

Thank you! 🙂

Ah, thank you very much

Based on https://github.com/lanpa/tensorboardX/blob/34d1616c035faaa0f3f7c9d19cb8bb4425f19939/tensorboardX/summary.py#L355 I would guess that it is already encoded before added to the tensorboard summary.

Thanks! 🙂

One more thing: The cuda_version that clearml finds automatically is wrong.

` =============
== PyTorch ==

NVIDIA Release 22.03 (build 33569136)
PyTorch Version 1.12.0a0+2c916ef ... Looking in indexes: ,
Requirement already satisfied: pip in /root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages (22.0.4)
2022-04-07 16:40:57
Looking in indexes: ,
Requirement already satisfied: Cython in /opt/conda/lib/python3.8/site-packages (0.29.28)
Looking in indexes: ,
Requirement already satisfied: numpy==1.22.3 in /opt/conda/...