Okey thanks! I'll try this, if it does not work I'll just deactivate the automatic detection feature.

The fileserver is remote, but the bandwidth is not an issue.
Is the automatic artifact storage of clearml async ? (meaning even if the task is finished it could still be uploading associated artifacts ?)

Well, I assume ClearML SDK is waiting for models to be uploaded to the fileserver - is your self-hosted server remote, or on the same network? Can it be you have a limited bandwidth to it?

so if you have very large snapshots that are close to one another one might wait for the other for quite some time

yeah, it is async, but when talking a snapshot it will wait for the previous model to finish uploading I think

Oh, if it's using the automagic than it's always in the background, which means it's also async

you can try calling task.flush(wait_for_uploads=True)

Is there a way to make it synchronous ?

how are you reporting your models?

I don't really know. I just detected it automatically from the start, so I haven't looked into it yet.