I can see the following using docker ps:
d5330ec8c47d allegroai/clearml-agent "/usr/agent/entrypoi…" 3 weeks ago Up 3 weeks clearml
I execute the following to access the container
docker exec -u root -t -i clearml /bin/bash
I went to
/root/.clearml/venv-builds but it is empty
It's probably the daemon agent docker, not the one that was spun for your experiment
I assume you've executed the agent in docker mode?
SuccessfulKoala55 I'm currently inside the docker container to recover the ckpt files. But
/root/.clearml/venvs-builds seems to be empty. Any idea where I could then find the ckpt files?
thanks a lot, yes it was the daemon :man-facepalming: I already could recover one checkpoint!
which docker did you exec into?
I assume it was run using a ClearML agent?
Hi ClumsyElephant70 , by default, the agent will not send these to the server. In order to do that, you'll need to set the
output_uri parameter in
Task.init() (or set the agent's
default_output_uri in the
clearml.conf file) and point it to the ClearML file server's address
thanks for the info, thats really bad 😬 I thought that the output_uri defaults to the fileserver 🙄