https://demoapp.demo.clear.ml/projects/55d709bfcbc4429a8ebe4751d9fecd30/experiments/22c5e2ee1d574cb2a453ec0050ddb29f/output/debugImages

yes

i appreciate your help today. it can’t be very fun working on a sunday. i hope you get some relax time away from the computer today, and look forward to hearing more when you are working.

i did want to point out, though, that when manually reporting, it looks like the plots don’t get “cleared” properly: https://demoapp.demo.clear.ml/projects/52eb5c9d938244daaa6fa460edce5e22/experiments/78fa65250e0544d7b50425a82dde75f5/info-output/metrics/plots?columns=selected&columns=type&columns=name&colu...

` $ clearml-agent config
Current configuration (clearml_agent v1.0.0, location: /home/username/clearml.conf):

agent.worker_id =
agent.worker_name = computer
agent.force_git_ssh_protocol = false
agent.python_binary =
agent.package_manager.type = pip
agent.package_manager.pip_version = <20.2
agent.package_manager.system_site_packages = false
agent.package_manager.force_upgrade = false
agent.package_manager.conda_channels.0 = pytorch
agent.package_manager.conda_channels.1 ...

here’s a the file with the keys and IP redacted: https://clearml.slack.com/files/U01PN0S6Y67/F0231N0GZ19/clearml.conf

hmmm, looks like maybe i should set it directly: https://clearml.slack.com/archives/CTK20V944/p1603369102359500?thread_ts=1603362214.350500&cid=CTK20V944

but “download plot as png” generates a blank image

yep, that was it. thanks for all your help and sorry to bother 🙂

we do use all those packages, and the version numbers are correct

okay, so if i set set_default_upload_destination as URI that’s local to the computer running the task (and the server):

• the server is “unable to load the image”—not surprising because the filesystem URI was not mounted into the container
• the files are present at the expected location on the local filesystem, but they are…blank! all white.that tells me that report_media might have been successful, but there’s some issue …encoding the data to a jpeg?

i’ll try changing the IP and look for a different error.

well, as generated by clearml-agent init —i pasted the text directly from the web app into the CLI interface, and it generated clearml.conf

but, the call used to start the script was python -m module.name --args

hi SubstantialElk6 , not sure if you were successful on this but i struggled with it as well, and it looks like the information is not in the linked document anymore.

in the end i realized that i needed to download apiserver.conf from the clearml-server repo ( https://github.com/allegroai/clearml-server/blob/master/apiserver/config/default/apiserver.conf ) and then add a user/pass for myself (starting at line 82).

https://demoapp.demo.clear.ml/projects/55d709bfcbc4429a8ebe4751d9fecd30/experiments/15f54924faa245f2baadc446e1315e39/output/metrics/plots

actually its missing imports from the second level too

but hmm, report_media generates a file that is 0 bytes, whereas report_image generates a 33KB file

$ conda list | grep matplotlib matplotlib 3.4.3 py39hf3d152e_1 conda-forge matplotlib-base 3.4.3 py39h2fa2bec_1 conda-forge

but now i’m confused about why set_default_upload_destination is different from output_uri . i kind of get it? but wouldn’t that be a safe default?

installed all of those using pip

also, i’m noticing the “last used” field does not update when I try to start an agent, but does change when I issue the curl command you gave earlier

no change in behavior and enable_git_ask_pass does not show up in the agent init output

thanks much for your help. should have thought to check there earlier, but kind of forgot that was a thing.

okay, so my problem is actually that using a “local” package is not supported—ie i need to pip install the code i’m running and that must correctly specify its dependencies

but there was a pip_version: “<20.2” line in myclearml.conf` , which would possibly have been a default in the config file like, 2 years ago or something

i tried lots of things, but values in the conf file (specifically the pip and cuda versions) overriding things in my code/env confused me for a long time