Could you test again with
https://clear.ml/docs/latest/docs/references/sdk/task/#mark_completed ?

There seems to be a discrepancy in the docs I'm trying to figure out and solve.
Most of the statuses are more explained here: https://clear.ml/docs/latest/docs/fundamentals/task/#task-states

Closed isn't yet.
Close is normally for manually closing a task: https://clear.ml/docs/latest/docs/references/sdk/task#close
You'll find more info here: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk/ and here: https://clear.ml/docs/latest/docs/guides/advanced/multiple_tasks_single_process

Could you elaborate on S3 checkpoint name?
I'm assuming it's a filename?
Possibly post those few lines of code?

We checked in the UI and if the model description is edited with double spaces, they remain, so the problem is likely somewhere in the SDK.

Are you running it with docker?

I'm not exactly sure but it seems this is an Airflow error when a library isn't working.
Can you try
os.environ["no_proxy"]="*"I've found this both here: https://github.com/apache/airflow/discussions/24463#discussioncomment-4211269
and here: https://stackoverflow.com/a/73983599

You can configure what to log and what not in the task init: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk/#automatic-logging

You can turn it all off by setting auto_connect_frameworks to false but you can do a finer grained control of logged frameworks with framework-boolean pairs

You can get all tasks: https://clear.ml/docs/latest/docs/references/sdk/task#taskget_all

You can search tasks: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#querying--searching-tasks

And you can get the status:
https://clear.ml/docs/latest/docs/references/sdk/task#get_status

Did you first init the Task?
https://clear.ml/docs/latest/docs/references/sdk/task/

Happy to help!

I'd you've got a self hosted instance you can have a look yourself https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config
But other then that I'm not sure. AnxiousSeal95 any thoughts?

Does this help? https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server#what-is-clearml-server

Do you get any error when uploading?
It looks like it can upload but can't download afterwards.

no this should work with this one. I'll double check if I'm remembering it correctly but I thought you should be able to start a task after loading your own configuration object, where can set the agent.package_manager.system_site_packages = true .

ReassuredTiger98 anything in the configuration file can be overruled 🙂
https://clear.ml/docs/latest/docs/configs/configuring_clearml

Or you can just load a config file or object: https://clear.ml/docs/latest/docs/references/sdk/task/#connect_configuration

It should, or you might need to nest the objects.
Edit: I asked, it won't there's a difference in configs I mixed up.

PIP can install from git repositories!
So you can point to your own repository or even a specific commit hash.

https://pip.pypa.io/en/stable/topics/vcs-support/

You can also use https://clear.ml/docs/latest/docs/references/sdk/task/#taskget_task since task.clone also accepts a task object

You can use https://clear.ml/docs/latest/docs/references/sdk/task/#taskget_project_id to get the id of the last updated project with that name

I think if you use explicit logging it only logs things you've selected but I'm not entirely sure
https://clear.ml/docs/latest/docs/guides/reporting/clearml_logging_example/

Do you have the same python version locally as remotely?
Some ways you could continue now:
you can reuse an existing python virtual environment: https://clear.ml/docs/latest/docs/clearml_agent/#virtual-environment-reuse

You can also run the agent in docker mode: https://clear.ml/docs/latest/docs/clearml_agent/#docker-mode

I'll have a look at the differences concerning the dev disappearing.

If you're looking for what docker volumes were used, that's in the docker compose file:
https://github.com/allegroai/clearml-server/blob/master/docker/docker-compose.yml

As far as I know, you can start a docker container with the same version and the same volumes and you should be able to just continue.

Is this after you've started the clearML server that you can't find the experiments?

Have you tried
logger = Logger.current_logger()in your code?
Logger is a singleton so you should get the same object from your previously created task
https://clear.ml/docs/latest/docs/references/sdk/logger/#loggercurrent_logger

ExasperatedCrab78 do you know how this could be?

To use a specific binary you can set in in the config: https://clear.ml/docs/latest/docs/configs/clearml_conf/#:~:text=python%20version%20(default)-,agent.python_binary,-(string)

But if you're trying to cache virtual environments you might be more interested in: https://clear.ml/docs/latest/docs/clearml_agent#environment-caching

If you're just looking to reuse virtual environments, have a look here: https://clear.ml/docs/latest/docs/clearml_agent/#environment-caching

https://clear.ml/docs/latest/docs/webapp/webapp_exp_track_visual/#installed-packages

Wait, I noticed you need another set of quotes:
Sample in the docs is: --memory="300m"

https://docs.docker.com/config/containers/resource_constraints/