That's pretty weird. I don't see any clear indications something is wrong, it simply doesn't execute the rest it would seem. Did it successfully run the first time before cloning it?

Do you mean in the WebUI or via the API?

PIP can install from git repositories!
So you can point to your own repository or even a specific commit hash.

https://pip.pypa.io/en/stable/topics/vcs-support/

Well you could let ClearML create the config file with: https://clear.ml/docs/latest/docs/references/sdk/task#taskset_credentials
store_conf_file=True

And then go edit the file.

But it's probably easier in your case to use https://clear.ml/docs/latest/docs/references/sdk/task#connect_configuration
and pass it your full configuration?

You can use the same project name and the same Taskname if that helps?

I don't think it's possible to set a custom ID if it doesn't exist yet. But I'll double check!

I'm not exactly sure but it seems this is an Airflow error when a library isn't working.
Can you try
os.environ["no_proxy"]="*"I've found this both here: https://github.com/apache/airflow/discussions/24463#discussioncomment-4211269
and here: https://stackoverflow.com/a/73983599

Happy to help!

Happy to help :)

It should, or you might need to nest the objects.
Edit: I asked, it won't there's a difference in configs I mixed up.

I'm not sure about the preview part but after uploading I think you might find the images with list and --filter
https://clear.ml/docs/latest/docs/clearml_data/clearml_data_cli#list
I'm not sure if that helps?

You've got it all working now?

Could it be multiple metrics that were combined into a single metric later on? Before the optimizer?

Do you have the same python version locally as remotely?
Some ways you could continue now:
you can reuse an existing python virtual environment: https://clear.ml/docs/latest/docs/clearml_agent/#virtual-environment-reuse

You can also run the agent in docker mode: https://clear.ml/docs/latest/docs/clearml_agent/#docker-mode

I'll have a look at the differences concerning the dev disappearing.

There seems to be a discrepancy in the docs I'm trying to figure out and solve.
Most of the statuses are more explained here: https://clear.ml/docs/latest/docs/fundamentals/task/#task-states

Closed isn't yet.
Close is normally for manually closing a task: https://clear.ml/docs/latest/docs/references/sdk/task#close
You'll find more info here: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk/ and here: https://clear.ml/docs/latest/docs/guides/advanced/multiple_tasks_single_process

PricklyRaven28 I think then you're looking for: ZipFile.ZIP_STORED

You can use https://clear.ml/docs/latest/docs/references/sdk/task/#taskget_project_id to get the id of the last updated project with that name

you can pass use the compression parameter in dataset.upload . The supported values are:
ZipFile.ZIP_STORED (no compression) ZipFile.ZIP_DEFLATED (requires zlib) ZipFile.ZIP_BZIP2 (requires bz2) ZipFile.ZIP_LZMA (requires lzma)Note that you need to import ZipFile beforehand: from zipfile import ZipFile
You're probably looking for ZIP_BZIP2 , but I'm not sure about that.

Could you try to see if it does work when you log those manually?
https://clear.ml/docs/latest/docs/clearml_sdk/model_sdk#manually-logging-models

Can you try: '${pipeline.docker_image}' ?

This ^
If you're not getting any errors, it should work just fine 🙂

In https://github.com/thepycoder/urbansounds8k/blob/main/preprocessing.py i'm seeing dataset_task.get_logger().report_image , dataset_task.get_logger().report_table , dataset_task.get_logger().report_histogram and dataset_task.get_logger().report_media which are all manual loggings. Hence, why the author probably didn't use any automatic logging.

So for notebooks requirements are indeed not checked elsewhere.
You can however include them with using this line before Task.init

Task.force_requirements_env_freeze(requirements_file=requirements.txt)

Have you tried using an existing virtual environment?
https://clear.ml/docs/latest/docs/clearml_agent#virtual-environment-reuse

We're sorry about that, this seem like a bug indeed. Could you open a github issue?

You can get all tasks: https://clear.ml/docs/latest/docs/references/sdk/task#taskget_all

You can search tasks: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#querying--searching-tasks

And you can get the status:
https://clear.ml/docs/latest/docs/references/sdk/task#get_status

Did you first init the Task?
https://clear.ml/docs/latest/docs/references/sdk/task/

ThoughtfulBadger56 Have you uncommented the existing venvs_cache section in the config file?
https://clear.ml/docs/latest/docs/clearml_agent#virtual-environment-reuse

Could you test the following:
Without reusing the virtual environment you made manually:
Can you run a task twice and see if the second run is at least reusing the virtual environment of the first run?

So could you just setup your virtual environment with a task?

And in the logs?

I'm afraid what you're trying to do isn't a supported implementation.
You'll have to choose between using docker mode to have one virtual environment for everything or using the pip mode where you can used the cached virtual environments but you can't reuse the one you currently have.