hmm, it was confusing to me, but it’s kind of an edge case where I was taking over a computer after a colleague left, seems like that might not be a common scenario

Hmmm. Just tried cloning a brand new task and the agent is still using the expired github access token.

installed all of those using pip

if i have code that’s just in a git repo but is not installed in any way, it runs fine if i invoke the entrypoint in a shell. but clearml will not find dependencies in secondary imports (as described above) if the agent someone just clones the repo but does not install the python package in some way.

okay, so looks like the docs for report_media need an update.

:face_palm:

will try the git ask pass thing.

also, i’m noticing the “last used” field does not update when I try to start an agent, but does change when I issue the curl command you gave earlier

if you’re able to check the data store, folders for all 120 plots will be on disk.

using clearML 0.17.5

wondering if there has been an update on this?

https://demoapp.demo.clear.ml/projects/55d709bfcbc4429a8ebe4751d9fecd30/experiments/15f54924faa245f2baadc446e1315e39/output/metrics/plots

great news! thank you! when there’s a new release, i need to docker-compose build && docker-compose up to get the latest?

okay, that’s a fresh install, and the backend is agg:
` Python 3.8.8 (default, Feb 24 2021, 21:46:12)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import matplotlib
matplotlib.get_backend()
'agg' `
the machine is headless, and there’s no window server running.

hey Martin.B, wondering if you were able to find anything out about this?

it looks like that worked: https://demoapp.demo.clear.ml/projects/52eb5c9d938244daaa6fa460edce5e22/experiments/40ad94b6544144e38da78d051c6cee59/output/log

thanks for that tip. i cleared out the vcs cache and was already using the latest version of the agent, same problem persists.

there’s a python version mismatch, i will make a different env for the agent to run in that has a matching python version

great, thank you very much for this info!

will give it a try. thanks 🙂

good questions 🙂
they are plots. they have unique titles. i’m using the auto-logging mechanism—so set up the task, then plt.show() no more than 114 plots are shown in the plots tab.

i noticed that the agent was downgrading to pip=20.1.1 at every attempt, so i added

Task.add_requirements("pip", "23.1.2")

and even then, it downgrades to 20.1.1?

` $ clearml-agent -d daemon --gpus 1 --foreground
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): DIFFERENT_IP_ADDRESS:8008
DEBUG:urllib3.util.retry:Incremented Retry for (url='/auth.login'): Retry(total=239, connect=3, read=240, redirect=240, status=240)
WARNING:urllib3.connectionpool:Retrying (Retry(total=239, connect=3, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff49318dd10>: Failed to ...

thanks for doing that and thanks for your work on the project 🙂

thank you, will do.

we do use all those packages, and the version numbers are correct

i appreciate your help today. it can’t be very fun working on a sunday. i hope you get some relax time away from the computer today, and look forward to hearing more when you are working.

i did want to point out, though, that when manually reporting, it looks like the plots don’t get “cleared” properly: https://demoapp.demo.clear.ml/projects/52eb5c9d938244daaa6fa460edce5e22/experiments/78fa65250e0544d7b50425a82dde75f5/info-output/metrics/plots?columns=selected&columns=type&columns=name&colu...

looking in the web app, under the “App Credentials” section, it lists those credentials as “used” when I attempted the curl command.

but there was a pip_version: “<20.2” line in myclearml.conf` , which would possibly have been a default in the config file like, 2 years ago or something

🤔

Media is uploaded to a preconfigured bucket (see setup_upload()) with a key (filename) describing the task ID, title, series and iteration.

` $ clearml-agent config
Current configuration (clearml_agent v1.0.0, location: /home/username/clearml.conf):

agent.worker_id =
agent.worker_name = computer
agent.force_git_ssh_protocol = false
agent.python_binary =
agent.package_manager.type = pip
agent.package_manager.pip_version = <20.2
agent.package_manager.system_site_packages = false
agent.package_manager.force_upgrade = false
agent.package_manager.conda_channels.0 = pytorch
agent.package_manager.conda_channels.1 ...