Hi ClumsyElephant70 , by default, the agent will not send these to the server. In order to do that, you'll need to set the output_uri
parameter in Task.init()
(or set the agent's default_output_uri
in the clearml.conf
file) and point it to the ClearML file server's address
I assume you've executed the agent in docker mode?
It's probably the daemon agent docker, not the one that was spun for your experiment
I can see the following using docker ps:d5330ec8c47d allegroai/clearml-agent "/usr/agent/entrypoi…" 3 weeks ago Up 3 weeks clearml
I execute the following to access the containerdocker exec -u root -t -i clearml /bin/bash
I went to /root/.clearml/venv-builds
but it is empty
SuccessfulKoala55 I'm currently inside the docker container to recover the ckpt files. But /root/.clearml/venvs-builds
seems to be empty. Any idea where I could then find the ckpt files?
thanks for the info, thats really bad 😬 I thought that the output_uri defaults to the fileserver 🙄
I assume it was run using a ClearML agent?
thanks a lot, yes it was the daemon :man-facepalming: I already could recover one checkpoint!