And which is for metrics?

@<1523701435869433856:profile|SmugDolphin23> Good catch. I have a good but unsatisfying message for you guys: I restarted the whole machine (server and agent) and now it works fine ...

Okay, I found something out: When I use docker image ubuntu:22.04 it does not spin up a service agent and aborts the task. When I used python:latest everything works fine!

Here is the log

Agent runs in docker mode. I ran the agent on the same machine as the server this time.

Yea exactly. tqdm sounds good!

Yes

Interesting. Will probably only matter for very small experiments or experiments, where validation is run very infrequently.

What's the reason for the shift?

Yea, tensorboardX is using moviepy.

Sure, no problem!

AgitatedDove14 https://github.com/allegroai/clearml/pull/372

pytorch.tensorboard is the same as tensorboardx https://github.com/pytorch/pytorch/blob/6d45d7a6c331ddb856ac34a76bcd3613aa05185b/torch/utils/tensorboard/summary.py#L461

I guess this is the current way to do it: https://github.com/tensorflow/tensorboard/issues/39#issuecomment-568917607 so I would say: Yes, it supports gif.

Yes, but this seems pretty reasonable to assume imo.

You mean I should have opencv/ffmpeg available on the clearml-server machine?

Based on https://github.com/lanpa/tensorboardX/blob/34d1616c035faaa0f3f7c9d19cb8bb4425f19939/tensorboardX/summary.py#L355 I would guess that it is already encoded before added to the tensorboard summary.

See also: https://github.com/tensorflow/tensorboard/issues/39#issuecomment-360906712

Exactly. I don't want people to circumvent the queue 🙂

Thank you very much, gonna try that!

Ah, sore should have been more specific. I mean on the ClearML server.

In the UI

Exactly!

Okay, thank you anyways. I was just asking because I thought I had seen such a setting before. Must have been something different.

SweetBadger76 I am using the Cleanup Service

AgitatedDove14 Thank you, that explains it.

Tried to install cudatoolkit==11.1 manually in this environemnt and got:

Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed                                                                                                                                                                         

UnsatisfiableError: The following specifications were found to be incompatible with each other:



Package xz conflicts for:
python=3....

Okay, but are you logs still stored on MinIO with only using sdk.development.default_output_uri ?

But this means the logger will use the default fileserver or not?

Now trying changing the default file server.