thanks a lot, yes it was the daemon :man-facepalming: I already could recover one checkpoint!

It's probably the daemon agent docker, not the one that was spun for your experiment

SuccessfulKoala55 I'm currently inside the docker container to recover the ckpt files. But /root/.clearml/venvs-builds seems to be empty. Any idea where I could then find the ckpt files?

I can see the following using docker ps:
d5330ec8c47d allegroai/clearml-agent "/usr/agent/entrypoi…" 3 weeks ago  Up 3 weeks  clearml

I execute the following to access the container
docker exec -u root -t -i clearml /bin/bash

I went to /root/.clearml/venv-builds but it is empty

which docker did you exec into?

thanks for the info, thats really bad 😬 I thought that the output_uri defaults to the fileserver 🙄

Hi ClumsyElephant70 , by default, the agent will not send these to the server. In order to do that, you'll need to set the output_uri parameter in Task.init() (or set the agent's default_output_uri in the clearml.conf file) and point it to the ClearML file server's address

I assume you've executed the agent in docker mode?

I assume it was run using a ClearML agent?