Or you can give it a configuration object: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#configuration-objects

Wait, I noticed you need another set of quotes:
Sample in the docs is: --memory="300m"

https://docs.docker.com/config/containers/resource_constraints/

Do you get any error when uploading?
It looks like it can upload but can't download afterwards.

I'm not exactly sure but it seems this is an Airflow error when a library isn't working.
Can you try
os.environ["no_proxy"]="*"I've found this both here: https://github.com/apache/airflow/discussions/24463#discussioncomment-4211269
and here: https://stackoverflow.com/a/73983599

I also learned :p

ZanyPig66 maybe this example can help?
https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/tensorboard_toy_pytorch.py

You could also try to upload an image or directory:
https://clear.ml/docs/latest/docs/guides/reporting/artifacts/#image-files

Could you upload the log so I can have a look?

I'd you've got a self hosted instance you can have a look yourself https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config
But other then that I'm not sure. AnxiousSeal95 any thoughts?

And pandas is in your requirements.txt?

I'm not sure if you can delete it when using pipelines but I would say try it on a new project?

ReassuredTiger98 anything in the configuration file can be overruled 🙂
https://clear.ml/docs/latest/docs/configs/configuring_clearml

Happy to help!

Could you try to see if it does work when you log those manually?
https://clear.ml/docs/latest/docs/clearml_sdk/model_sdk#manually-logging-models

Also have a look at --memory-swap
It seems you might not anticipated this usage:

If --memory-swap is unset, and --memory is set, the container can use as much swap as the --memory setting, if the host container has swap memory configured. For instance, if --memory="300m" and --memory-swap is not set, the container can use 600m in total of memory and swap.

There seem to be a similar error here: https://github.com/aws/aws-cli/issues/4758

Could it be minio isn't on the same API version as S3?

I don't see SB3 here so PyTorch would be best: https://clear.ml/docs/latest/docs/integrations/libraries

Well you could let ClearML create the config file with: https://clear.ml/docs/latest/docs/references/sdk/task#taskset_credentials
store_conf_file=True

And then go edit the file.

But it's probably easier in your case to use https://clear.ml/docs/latest/docs/references/sdk/task#connect_configuration
and pass it your full configuration?

Have you tried
logger = Logger.current_logger()in your code?
Logger is a singleton so you should get the same object from your previously created task
https://clear.ml/docs/latest/docs/references/sdk/logger/#loggercurrent_logger

As far as I know, you can start a docker container with the same version and the same volumes and you should be able to just continue.

You've got it all working now?

I'm afraid what you're trying to do isn't a supported implementation.
You'll have to choose between using docker mode to have one virtual environment for everything or using the pip mode where you can used the cached virtual environments but you can't reuse the one you currently have.

Could you test the following:
Without reusing the virtual environment you made manually:
Can you run a task twice and see if the second run is at least reusing the virtual environment of the first run?

I think if you use explicit logging it only logs things you've selected but I'm not entirely sure
https://clear.ml/docs/latest/docs/guides/reporting/clearml_logging_example/

Do you have the same python version locally as remotely?
Some ways you could continue now:
you can reuse an existing python virtual environment: https://clear.ml/docs/latest/docs/clearml_agent/#virtual-environment-reuse

You can also run the agent in docker mode: https://clear.ml/docs/latest/docs/clearml_agent/#docker-mode

I'll have a look at the differences concerning the dev disappearing.

I don't think there is a direct way but if you use a docker container with a different python version inside, that works.

You can use https://clear.ml/docs/latest/docs/references/sdk/task/#taskget_project_id to get the id of the last updated project with that name

Are you running it with docker?

In this particular case using a naming convention is probably the best answer.
If you already have a naming convention for projects, you could just reuse that to prevent confusion :)

You can configure what to log and what not in the task init: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk/#automatic-logging

You can turn it all off by setting auto_connect_frameworks to false but you can do a finer grained control of logged frameworks with framework-boolean pairs

Typo will be fixed 🙂 https://github.com/allegroai/clearml-docs/pull/362

You can explicitly log them: https://clear.ml/docs/latest/docs/guides/reporting/scalar_reporting/