BoredPigeon26 , you can scroll through iterations 🙂
BoredPigeon26 , are images from previous iterations still showing?
Hi CloudySwallow27 ,
I think currently the way to do this is by disabling the framework detection and reporting the debug images manually.
You can do this by Task.init(
auto_connect_frameworks=False
)
Are you using a self hosted server? Were the files written to some bucket/storage or directly to the fileserver?
Hi @<1590514584836378624:profile|AmiableSeaturtle81> , I'm guessing it's a self deployed server. What version are you on? Did you ever see any errors/issues in mongodb/elastic?
Do you mean that ALL experiments are being deleted from all projects?
BoredPigeon26 , it looks like the file isn't accessible through your browser. Are you sure the remote machine files are accessible?
That makes sense... If you turn auto_connect_streams to false this mean that auto reporting will be disabled as per the documentation.. If you turn it to True then logging should resume.
@<1719524641879363584:profile|ThankfulClams64> , can you provide a small code snippet that reproduces this behaviour? Can you also test with the latest version of clearml
?
My bad, if you set auto_connect_streams to false, you basically disable the console logging... Please see the documentation:
auto_connect_streams (Union[bool, Mapping[str, bool]]) – Control the automatic logging of stdout and stderr.
Hi @<1719524641879363584:profile|ThankfulClams64> , how are you reporting debug samples?
What setting do you have in this section of your clearml.conf
None
@<1806135353375985664:profile|ColossalHawk50> , you basically run the agent outside of a docker container and then use the --docker
flag to run it in docker mode. Makes sense?
If you remove any reference of ClearML from the code on that machine, does it still hang?
Do you have a way to see which docker images you have locally (if they weren't removed) to see on which version you were previously?
Hi @<1526371965655322624:profile|NuttyCamel41> , can you add the full log?
Hi @<1581454875005292544:profile|SuccessfulOtter28> , you can archive an experiment, go into the archive and then you can delete from there
Also, how are you trying to download them?
Do you have a log of the triton server?
What happens if you remove git_host="
http://bitbucket.org "
Hi @<1544128920209592320:profile|BewilderedLeopard78> , I don't think there is such an option currently. Maybe open a GitHub feature request to track this 🙂
Hi @<1544853695869489152:profile|NonchalantOx99> , as the error states:
raise ValueError('Could not find queue named "{}"'.format(queue_name))
ValueError: Could not find queue named "services"
It couldn't find the queue services. This is for the pipeline controller to run on. Pipelines consist of steps and the controller and they all can run on different machines. By default, the controller will try to run on the services queue. Makes sense? 🙂
Hi @<1570583237065969664:profile|AdorableCrocodile14> , is it possible you have some models inside?
Hi @<1790915053747179520:profile|KindParrot86> , currently Slack alerts are available as an example for the OS - None
You can write an adapter for it to send emails instead of Slack alerts
task that reads a message from a queue
Can you give a specific example?
You can set the docker image you want to run with using Task.set_base_docker
None
Also, is it an AWS S3 or is it some similar storage solution like Minio?
Hi JumpyDragonfly13 , can you try going to http://localhost:8080/login ? What happens when you open developer tools (F12) when browsing currently
Have you played with the env variables?
It should look something like this