ReassuredTiger98 this should not happen - are you sure this is not at the very start of the experiment?
I don't think so. It is related to issue with the clearml-server I posted in the other thread. Essentially the clearml-server hangs, then I restart it with docker-compose down && docker-compose up -d
and the experiments sometimes show as running, but on the clearml-agents I see that actually nothing is running or they show as aborted.
I know that usually clearml-agents do not abort on server restart and just continue.
I don't think this the intended behavior. Can you please elaborate how it happens exactly?