Reputation
Badges 1
5 × Eureka!Have you tried using an existing virtual environment?
https://clear.ml/docs/latest/docs/clearml_agent#virtual-environment-reuse
Could it be multiple metrics that were combined into a single metric later on? Before the optimizer?
Could you upload the log so I can have a look?
That's pretty weird. I don't see any clear indications something is wrong, it simply doesn't execute the rest it would seem. Did it successfully run the first time before cloning it?
PIP can install from git repositories!
So you can point to your own repository or even a specific commit hash.
You can fix this by using a requirements.txt or the --packages parameter
https://clear.ml/docs/latest/docs/apps/clearml_task/#package-dependencies
If you run it, what does it say in experiment list -> experiments -> execution -> installed packages?
And pandas is in your requirements.txt?
That doesn't seem normal, let me ask around and get back to you
Did you use --git-credentials ?
https://clear.ml/docs/latest/docs/apps/clearml_session#accessing-a-git-repository
Well seems like you have a solution for now?
If you still want to run it as a notebook, the following should make pip install the required packages:
import sys !{sys.executable} -m pip install -r requirements.txt
I'll check if this something we need to update in our documentation or if it's a bug.
So for notebooks requirements are indeed not checked elsewhere.
You can however include them with using this line before Task.init
Task.force_requirements_env_freeze(requirements_file=requirements.txt)
You can find more info here: https://clear.ml/docs/latest/docs/references/sdk/task#taskforce_requirements_env_freeze
Can you give me a bit more info what exactly you're trying to log and what framework you're using?
Could you try to see if it does work when you log those manually?
https://clear.ml/docs/latest/docs/clearml_sdk/model_sdk#manually-logging-models
I don't see SB3 here so PyTorch would be best: https://clear.ml/docs/latest/docs/integrations/libraries
Did you first init the Task?
https://clear.ml/docs/latest/docs/references/sdk/task/
You can get all tasks: https://clear.ml/docs/latest/docs/references/sdk/task#taskget_all
You can search tasks: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#querying--searching-tasks
And you can get the status:
https://clear.ml/docs/latest/docs/references/sdk/task#get_status
Do you mean in the WebUI or via the API?
ZanyPig66 maybe this example can help?
https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/tensorboard_toy_pytorch.py
You could also try to upload an image or directory:
https://clear.ml/docs/latest/docs/guides/reporting/artifacts/#image-files
You can add them in env files:
https://clear.ml/docs/latest/docs/clearml_serving/clearml_serving_setup/#advanced-setup---s3gsazure-access-optional
If you're just looking to reuse virtual environments, have a look here: https://clear.ml/docs/latest/docs/clearml_agent/#environment-caching
Do you mean what's visible in the UI, projects -> Execution: Installed Packages?
Well you could let ClearML create the config file with: https://clear.ml/docs/latest/docs/references/sdk/task#taskset_credentials
store_conf_file=True
And then go edit the file.
But it's probably easier in your case to use https://clear.ml/docs/latest/docs/references/sdk/task#connect_configuration
and pass it your full configuration?
Could you test the following:
Without reusing the virtual environment you made manually:
Can you run a task twice and see if the second run is at least reusing the virtual environment of the first run?
ThoughtfulBadger56 Have you uncommented the existing venvs_cache section in the config file?
https://clear.ml/docs/latest/docs/clearml_agent#virtual-environment-reuse
I'm afraid what you're trying to do isn't a supported implementation.
You'll have to choose between using docker mode to have one virtual environment for everything or using the pip mode where you can used the cached virtual environments but you can't reuse the one you currently have.