Did you first init the Task?
https://clear.ml/docs/latest/docs/references/sdk/task/

I'd you've got a self hosted instance you can have a look yourself https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config
But other then that I'm not sure. AnxiousSeal95 any thoughts?

There seems to be a discrepancy in the docs I'm trying to figure out and solve.
Most of the statuses are more explained here: https://clear.ml/docs/latest/docs/fundamentals/task/#task-states

Closed isn't yet.
Close is normally for manually closing a task: https://clear.ml/docs/latest/docs/references/sdk/task#close
You'll find more info here: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk/ and here: https://clear.ml/docs/latest/docs/guides/advanced/multiple_tasks_single_process

This ^
If you're not getting any errors, it should work just fine 🙂

In https://github.com/thepycoder/urbansounds8k/blob/main/preprocessing.py i'm seeing dataset_task.get_logger().report_image , dataset_task.get_logger().report_table , dataset_task.get_logger().report_histogram and dataset_task.get_logger().report_media which are all manual loggings. Hence, why the author probably didn't use any automatic logging.

Have you tried
logger = Logger.current_logger()in your code?
Logger is a singleton so you should get the same object from your previously created task
https://clear.ml/docs/latest/docs/references/sdk/logger/#loggercurrent_logger

Could you upload the log so I can have a look?

I'm afraid what you're trying to do isn't a supported implementation.
You'll have to choose between using docker mode to have one virtual environment for everything or using the pip mode where you can used the cached virtual environments but you can't reuse the one you currently have.

I'm not sure if you can delete it when using pipelines but I would say try it on a new project?

It looks like there is this option under Settings->Configuration->Show Hidden Projects

If I'm 'ot mistaken closed status should only appear when you've used Close ()

Do you have the same python version locally as remotely?
Some ways you could continue now:
you can reuse an existing python virtual environment: https://clear.ml/docs/latest/docs/clearml_agent/#virtual-environment-reuse

You can also run the agent in docker mode: https://clear.ml/docs/latest/docs/clearml_agent/#docker-mode

I'll have a look at the differences concerning the dev disappearing.

You can configure what to log and what not in the task init: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk/#automatic-logging

You can turn it all off by setting auto_connect_frameworks to false but you can do a finer grained control of logged frameworks with framework-boolean pairs

And this is also remotely?

Can you elaborate on question #2?
Do you want to reuse a task or something else?

Also have a look at --memory-swap
It seems you might not anticipated this usage:

If --memory-swap is unset, and --memory is set, the container can use as much swap as the --memory setting, if the host container has swap memory configured. For instance, if --memory="300m" and --memory-swap is not set, the container can use 600m in total of memory and swap.

Wait, I noticed you need another set of quotes:
Sample in the docs is: --memory="300m"

https://docs.docker.com/config/containers/resource_constraints/

ReassuredTiger98 anything in the configuration file can be overruled 🙂
https://clear.ml/docs/latest/docs/configs/configuring_clearml

Could you test again with
https://clear.ml/docs/latest/docs/references/sdk/task/#mark_completed ?

ThoughtfulBadger56 Have you uncommented the existing venvs_cache section in the config file?
https://clear.ml/docs/latest/docs/clearml_agent#virtual-environment-reuse

And in the logs?

Do you get any error when uploading?
It looks like it can upload but can't download afterwards.

That's pretty weird. I don't see any clear indications something is wrong, it simply doesn't execute the rest it would seem. Did it successfully run the first time before cloning it?

Could you test the following:
Without reusing the virtual environment you made manually:
Can you run a task twice and see if the second run is at least reusing the virtual environment of the first run?

To use a specific binary you can set in in the config: https://clear.ml/docs/latest/docs/configs/clearml_conf/#:~:text=python%20version%20(default)-,agent.python_binary,-(string)

But if you're trying to cache virtual environments you might be more interested in: https://clear.ml/docs/latest/docs/clearml_agent#environment-caching

Do you mean in the WebUI or via the API?

As far as I know, you can start a docker container with the same version and the same volumes and you should be able to just continue.

Wait... Will it limit memory usage but not show it in overview or just plain stop the container if it happens to use more memory?

So could you just setup your virtual environment with a task?

Could you elaborate on S3 checkpoint name?
I'm assuming it's a filename?
Possibly post those few lines of code?

We checked in the UI and if the model description is edited with double spaces, they remain, so the problem is likely somewhere in the SDK.

Did you use --git-credentials ?
https://clear.ml/docs/latest/docs/apps/clearml_session#accessing-a-git-repository