Or alternatively I just saw that Task.create takes a requirements.txt as an argument. This would also be fine for me, however I am not sure whether I should use Task.create ?

However, this seems like a pretty edge-case to me - why would you do that on a regular basis?

For me this is how I use ClearML as tensorboard replacement. To start some debug runs before adding it to a clearml-agent queue. For me this seems like the most common usage case or am I missunderstanding ClearML?

Just tested it again. Here is my config:
https://gist.github.com/mctigger/086c5f8071a604605e9f7a172800b51d
In the Web UI under Configuration -> Hyper Parameters -> Environment I can see the following:
MUJOCO_GL osmesa

Maybe the problem is that I do not start my docker containers from the root user, so 1001 is a mapping inside the docker to my actual user. Could it be that on the host the owner if your .ssh files is called root ?

Sure, let me try.

It is only a single agent that is sending a single artifact. server-->agent is fast, but agent-->server is slow.

What exactly do you mean by that?

Unfortunately, not. Quick question: Is there caching happening somewhere besides .clearml ? Does the boto3 driver create cache?

Or there should be an early error for trying to run conda based tasks on pip agents

Afaik, clearml-agent will use existing installed packages if they fit the requirements.txt. E.g. pytorch >= 1.7 will only install PyTorch if the environment does not already provide some version of PyTorch greater or equal to 1.7.

Thanks! I am fascinated by what you guys offer with clearml 🙂

Not for the logger, I just tested it.

I installed as told on pytorch.org : pip3 install --pre torch torchvision torchaudio --index-url None

Latter

Thank you, gonna try that!

But you can manually add them with Task.add_requirements, no?

In my opinion an ugly solution. I would have to keep track of which requirements are missing. Then I would rather just add all requirements manually.

I will create a minimal example.

Can you maybe also tell me which docker image you used? For me this is all not working unfortunately

Thank you. Seems like someone implemented a type check Error: Dataset id=8d7355655830427f9243671c8cf0a6b0 is not of type Dataset :)

I am also wondering how I integrate my (preexisting) main task in the pipeline. I start my main task like this: python my_script.py --myarg "myargs" . How are the arguments captured? I am very confused, how one integrates this correctly...

Could you elaborate on that:
"So the agent failed to actually restore it from the git (files that are not added are not considered part of the git diff, this is usually git behavior)."

Or I am using this method wrong.

What I get for args when I print it locally is not the same as what ClearML extracts from args .

It is server version 1.0 and everything that came with it.

CharmingPuppy6 These threads may also be interesting for you: https://clearml.slack.com/archives/CTK20V944/p1614867532303700 https://clearml.slack.com/archives/CTK20V944/p1617963053397600

@<1576381444509405184:profile|ManiacalLizard2> Thank you, but afaik this only works locally and not if you run your task on a clearml-agent!

Yes

I mean, could my hard drive not become full at some point? Can clearml-agent currently detect this?

Can you explain what you meant by entropy point file? In a new git repository my code works fine.

Nvm, I took a look at conda history and there I see it