no this should work with this one. I'll double check if I'm remembering it correctly but I thought you should be able to start a task after loading your own configuration object, where can set the agent.package_manager.system_site_packages = true .

It should, or you might need to nest the objects.
Edit: I asked, it won't there's a difference in configs I mixed up.

You can find more info here: https://clear.ml/docs/latest/docs/references/sdk/task#taskforce_requirements_env_freeze

If you run it, what does it say in experiment list -> experiments -> execution -> installed packages?

You can set where to store it via this config file: https://clear.ml/docs/latest/docs/fundamentals/artifacts/#setting-upload-destination

you can pass use the compression parameter in dataset.upload . The supported values are:
ZipFile.ZIP_STORED (no compression) ZipFile.ZIP_DEFLATED (requires zlib) ZipFile.ZIP_BZIP2 (requires bz2) ZipFile.ZIP_LZMA (requires lzma)Note that you need to import ZipFile beforehand: from zipfile import ZipFile
You're probably looking for ZIP_BZIP2 , but I'm not sure about that.

For pipelines there's currently no way to use different compressions. You can still use it when explicitly uploading https://clear.ml/docs/latest/docs/references/sdk/dataset/#upload

You can definitely turn off logging partially or completely: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#automatic-logging

I'm not aware of changing this during the execution of the task (you'll have to set this in task.init) but you can manually log what you need after you (partially) turned off logging.

I also learned :p

Can you elaborate on the hidden project? Is this part of a task you created and something is not showing up?

Could you try to see if it does work when you log those manually?
https://clear.ml/docs/latest/docs/clearml_sdk/model_sdk#manually-logging-models

And pandas is in your requirements.txt?

That doesn't seem normal, let me ask around and get back to you

Can you try: '${pipeline.docker_image}' ?

Typo will be fixed 🙂 https://github.com/allegroai/clearml-docs/pull/362

You can explicitly log them: https://clear.ml/docs/latest/docs/guides/reporting/scalar_reporting/

You can add them in env files:
https://clear.ml/docs/latest/docs/clearml_serving/clearml_serving_setup/#advanced-setup---s3gsazure-access-optional

Well seems like you have a solution for now?

If you still want to run it as a notebook, the following should make pip install the required packages:

import sys !{sys.executable} -m pip install -r requirements.txt
I'll check if this something we need to update in our documentation or if it's a bug.

ExasperatedCrab78 do you know how this could be?

You can fix this by using a requirements.txt or the --packages parameter
https://clear.ml/docs/latest/docs/apps/clearml_task/#package-dependencies

ReassuredTiger98 you can set different parameters per task:
https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#configuration

I think if you use explicit logging it only logs things you've selected but I'm not entirely sure
https://clear.ml/docs/latest/docs/guides/reporting/clearml_logging_example/

We're sorry about that, this seem like a bug indeed. Could you open a github issue?

You can get all tasks: https://clear.ml/docs/latest/docs/references/sdk/task#taskget_all

You can search tasks: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#querying--searching-tasks

And you can get the status:
https://clear.ml/docs/latest/docs/references/sdk/task#get_status

Does this help? https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server#what-is-clearml-server

PIP can install from git repositories!
So you can point to your own repository or even a specific commit hash.

https://pip.pypa.io/en/stable/topics/vcs-support/

You can disable it if you want to. Sometimes it's useful to retrace your steps but it works without :)

You can use the same project name and the same Taskname if that helps?

I don't think it's possible to set a custom ID if it doesn't exist yet. But I'll double check!

There seem to be a similar error here: https://github.com/aws/aws-cli/issues/4758

Could it be minio isn't on the same API version as S3?

Can you give me a bit more info what exactly you're trying to log and what framework you're using?

I don't see SB3 here so PyTorch would be best: https://clear.ml/docs/latest/docs/integrations/libraries