Just checking, are you just trying to use a different docker image in a task? Because then you might want to use this: https://clear.ml/docs/latest/docs/apps/clearml_task/#docker
https://clear.ml/docs/latest/docs/clearml_agent#docker-mode

AnxiousSeal95 Do you know this one?

Both server and agent can be configured with different ports. Which is it you`re looking for?

Could you try to see if it does work when you log those manually?
https://clear.ml/docs/latest/docs/clearml_sdk/model_sdk#manually-logging-models

I know zip and tar.gz are supported for auto extraction. But you're looking for a setting to have artifacts compressed with tar instead of zip?

That doesn't seem normal, let me ask around and get back to you

You can get all tasks: https://clear.ml/docs/latest/docs/references/sdk/task#taskget_all

You can search tasks: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#querying--searching-tasks

And you can get the status:
https://clear.ml/docs/latest/docs/references/sdk/task#get_status

Did you first init the Task?
https://clear.ml/docs/latest/docs/references/sdk/task/

Can you try: '${pipeline.docker_image}' ?

As far as I know, you can start a docker container with the same version and the same volumes and you should be able to just continue.

So for notebooks requirements are indeed not checked elsewhere.
You can however include them with using this line before Task.init

Task.force_requirements_env_freeze(requirements_file=requirements.txt)

If you run it, what does it say in experiment list -> experiments -> execution -> installed packages?

I don't see SB3 here so PyTorch would be best: https://clear.ml/docs/latest/docs/integrations/libraries

You can fix this by using a requirements.txt or the --packages parameter
https://clear.ml/docs/latest/docs/apps/clearml_task/#package-dependencies

You can use the same project name and the same Taskname if that helps?

I don't think it's possible to set a custom ID if it doesn't exist yet. But I'll double check!

Could it be multiple metrics that were combined into a single metric later on? Before the optimizer?

no this should work with this one. I'll double check if I'm remembering it correctly but I thought you should be able to start a task after loading your own configuration object, where can set the agent.package_manager.system_site_packages = true .

Are you running it with docker?

You can add them in env files:
https://clear.ml/docs/latest/docs/clearml_serving/clearml_serving_setup/#advanced-setup---s3gsazure-access-optional

ThoughtfulBadger56 Have you uncommented the existing venvs_cache section in the config file?
https://clear.ml/docs/latest/docs/clearml_agent#virtual-environment-reuse

you can pass use the compression parameter in dataset.upload . The supported values are:
ZipFile.ZIP_STORED (no compression) ZipFile.ZIP_DEFLATED (requires zlib) ZipFile.ZIP_BZIP2 (requires bz2) ZipFile.ZIP_LZMA (requires lzma)Note that you need to import ZipFile beforehand: from zipfile import ZipFile
You're probably looking for ZIP_BZIP2 , but I'm not sure about that.

Could you test again with
https://clear.ml/docs/latest/docs/references/sdk/task/#mark_completed ?

For pipelines there's currently no way to use different compressions. You can still use it when explicitly uploading https://clear.ml/docs/latest/docs/references/sdk/dataset/#upload

I'm not sure if you can delete it when using pipelines but I would say try it on a new project?

Could you test the following:
Without reusing the virtual environment you made manually:
Can you run a task twice and see if the second run is at least reusing the virtual environment of the first run?

To use a specific binary you can set in in the config: https://clear.ml/docs/latest/docs/configs/clearml_conf/#:~:text=python%20version%20(default)-,agent.python_binary,-(string)

But if you're trying to cache virtual environments you might be more interested in: https://clear.ml/docs/latest/docs/clearml_agent#environment-caching

So could you just setup your virtual environment with a task?

Can you try again with "-m 16g" ?

Also have a look at --memory-swap
It seems you might not anticipated this usage:

If --memory-swap is unset, and --memory is set, the container can use as much swap as the --memory setting, if the host container has swap memory configured. For instance, if --memory="300m" and --memory-swap is not set, the container can use 600m in total of memory and swap.

Happy to help :)