For example I have a DATA_DIR
environment variable which points to the directory where disk-data is stored
Depending on where the agent is, the value of DATA_DIR
might change
Oh I get it, that also makes sense with the docs directing this at inference jobs and avoiding GPU - because of the 1-N thing
cluster.routing.allocation.disk.watermark.low:
I manually deleted the allegroai/trains:latest
image, that didn't help either
a machine that had previous installation, but I deleted the /opt/trains
directory beforehand
Sorry I meant this link
https://azuremarketplace.microsoft.com/en-us/marketplace/apps/apps-4-rent.clearml-on-centos8
I think a good idea is to add to the error message when the clearml agent fails due to import error, a suggestion ot try out with pip freeze
sudo curl
https://raw.githubusercontent.com/allegroai/trains-server/master/docker-compose.yml -o /opt/trains/docker-compose.yml
Can you lend a few a words about how the not-pip freeze mechanism of detecting packages work?
but shouldn't the :lastest
make it redownload the right image?
this is the selection from the column setting menu
But does it disable the agent? or will the tasks still wait for the agent to dequeue?
cool, didn't know about the PAT
One sec I'll paste the relevant pieces of code
I re-executed the experiemnt, nothing changes
Hi guys, just updated the issue - seems like the new release did fix the color scale, but I notice some data points are missing (the plot is missing data!)
see my comment on the issue
https://github.com/allegroai/clearml/issues/373#issuecomment-894756446
checking and will let you know