How about this by the way?
https://clear.ml/docs/latest/docs/references/sdk/model_outputmodel#outputmodelset_default_upload_uri
The highlighted line is exactly that. Instead of client.tasks.get_all()
I think it would be along the lines of client.debug.ping()
DepressedChimpanzee34 , Damn that's a shame. Then it means that to use the endpoint you'll need to implement some network communication in python (something like curl through python)
I found another one that might help:client.session.get_clients()
This will return the clearm and server versions. This should be validation enough if server is up or not.
However I'd suggest implementing some sort of ability to send POST api calls via your script.
Also, how are you saving your models?
That sounds like a good idea! Can you please open a GitHub issue to track this?
Hi @<1523701491863392256:profile|VastShells9> , I assume you're using the autoscaler?
RotundHedgehog76 , Hi π
In none-docker mode, the agent simply runs on the target machine and creates a python virtual environment and runs the experiment in it. This means that you need to handle CUDA and other system installations for the agent to run your experiments.
When agent runs in docker mode, it will set up a docker image (that was pre-defined) and will create the virtual environment in that docker.
Basically the docker option gives you the most surgical and exactly pre-defined...
I would also suggest using pipelines if you want to do several actions with a task controlling the progress.
What version of python do you need to run on?
Why does the figure change so drastically? And how can I solve it?
What are you referring yo specifically? The data plots seem to be identical.
Sidenote: there seems to be a bug in the plot viewer, as the axis are a bit chaotic..
Do you mean the x/y intersection?
Hi CharmingStarfish14 , I think it comes from the way that the clearml-agent
works if I understand correctly your issue. When running in remote it uses the values on the backend. So for example if you take a task and clone it, assuming the task uses parameters from the repo and they change, the agent will take the parameters that are logged in the ClearML backend. So for new parameters to take affect you need to clone the task, change the parameters in the cloned task (Either by UI or ...
You certainly can do it with the python APIClient OR through the requests library
RipeAnt6 , you have to manage your storage on the NAS yourself. We delete data only on the fileserver.
However, you could try mounting the NAS to the fileserver docker as a volume and then deletion should also handle files on the NAS π
Hi @<1635813046947418112:profile|FriendlyHedgehong10> , you can achieve that by using env variables - None
In the installed packages, try removing the version for imageio (Is this a private package?). This looks like the environment (OS/Python version) doesn't support the specific package OR the package is inside a private artifactory
Hi @<1546303293918023680:profile|MiniatureRobin9> , can you add the full console log?
I see, thank you for the input π
If the process is dead it will be removed from the UI after some time
Hi @<1579280543999070208:profile|SourFly7> , this index holds scalars of some experiments. You can reduce it by deleting some experiments. Do you have any other large scalar indices?
Here is an example for auto cleaning. Did you delete ALL experiments?
Hi @<1585078763312386048:profile|ArrogantButterfly10> , does the controller stay indefinitely in the running state?
Hi Danil,
You can use the following env variable to set it πCLEARML_AGENT_SKIP_PIP_VENV_INSTALL
Aw you deleted your response fast
Yeah I misread the part where it's not in ps aux
^^
Hi ReassuredTiger98 , I'm not sure I understand.
You're asking why you need to put in credentials to use minio as a files_server
or why you got the error directly on port 9000 but not on s://minio_instance:9000/ <MY_BUCKET_NAME>
?
Hi @<1618780810947596288:profile|ExuberantLion50> , what happens if you set the python binary path itself? Also, any specific reason you're not running in docker mode?
Is there any specific reason you're not running in docker mode? Running in docker would simplify things
RotundSquirrel78 , do you have an estimate how much RAM the machine running ClearML server? Is it dedicated to ClearML only or are there other processes running?
In the web UI you can click the settings icon at the top right -> settings. At that screen the version should be shown at the bottom right