I am very confused now, I tried switch to my local machine and change the clearml.conf.
It only partly worked :Dataset.list_datasets() returns the correct list (from the remote server).
But Dataset.get(dataset_id="ce2abe847e004ac282cc435bfa9c4bd5")
gives me :2021-12-20 13:46:39,404 - clearml.storage - ERROR - Could not download ` , err: Failed getting object localhost:8081/annotation_dataset/annotation.ce2abe847e004ac282cc435bfa9c4bd5/artifacts/state/state.json (404): <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title> <h1>Not Found</h1> <p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p> `
What is the proper way to change a clearml.conf ?
of course, I am checking using the env command
I am very confused now, I tried switch to my local machine and change the clearml.conf.
It only partly worked :
Notice that the Dataset.get (...) is downloading an artifact that was uploaded before, basically it gets the full URL and downloads the data. it seems the original dataset uploaded to "localhost:8081", could that be the case?
Check the environment variables, maybe test with export maybe there's some env var hiding there 🙂
here is the command I am using :sudo docker run -it -v /home/ubuntu/app/:/app/ -v /home/ubuntu/folder/clearml.conf:/root/clearml.conf --network "clearml_backend" my_image bash
Can you try going into the docker and verifying you have the same clearml.conf inside?
How is the docker being raised and which volumes are being shared?
Ah! That's it, thank you very much ! I did not know this was an issue. I though the dataset was only linked to the fileserver and not to the specific url used to upload it.
I was looking at the code of the Dataset class, but I could not find where the files_server is retrieved.
I though the dataset was only linked to the fileserver and not to the specific url used to upload it. (
ShinyRabbit94 yep exactly! the idea is that you can actually do the storage on any solution (S3/GS etc.) the file server is just the default one 🙂
What is the proper way to change a clearml.conf ?
inside a container you can mount an external clearml.conf, or override everything with OS environment
https://clear.ml/docs/latest/docs/configs/env_vars#server-connection
I did, I copy pasted the config from within the docker