This sounds like you don't have clearml installed in the ubuntu container. Either this, or your clearml.conf in the container is not pointing to the server, as a result all information is missing.

I'd rather suggest you change the approach, and run a clearml-agent setup with docker and when you want to run YOLOv5 training you actually execute it remotely on the queue that the agent is listening to

To link a dataset to a task you need to pass the alias= parameter to the Dataset.get . See here: https://clear.ml/docs/latest/docs/clearml_data/clearml_data_sdk#accessing-datasets

Then change from git+ssh to git+https

Hey @<1523701066867150848:profile|JitteryCoyote63> , could you please open a GH issue on our repo too, so that we can more effectively track this issue. We are working on it now btw

Hello @<1533257278776414208:profile|SuperiorCockroach75> , thanks for asking. It’s actually unsupervised, because modern LLMs are all trained to predict next/missing words, which is an unsupervised method

Can you update the clearml version to latest (1.11.1) and see whether the issue is fixed?

Hello @<1523710243865890816:profile|QuaintPelican38> , could you try Dataset.get ing an existent dataset and tell whether there are any errors or not?

It won't, for that you need full support from Ultralytics

Is this a jupyter notebook or something ? Can you download it properly as either a .ipynb or .py file?

Ok, then launch an agent using clearml-agent daemon --queue default that way your steps will be sent to the agent for execution. Note that in this case, you shouldn't change your code snippet in any way.

The line before the last in your code snippet above. pipe.start_locally .

This is doing fine-tuning. Training a multi-billion parameter model from scratch would be economically unfeasible for most of existing enterprises

I see you want to use the services queue for both the pipeline controller and pipeline steps, but you have only one worker/agent listening to this queue. In this case you need at least 2 agents listening to the services queue. Try spawning an additional agent that listens to this queue and let me know how it goes .

Hey @<1523701083040387072:profile|UnevenDolphin73> , sorry for late reply, I’m investigating now the issue that you mentioned that running a remote task with create_function_task fails. I can’t quite reproduce it, can you please provide a complete runnable code snippet that fails like you just described

This sounds like a use case for the enterprise version of ClearML. In it you can set read/write permissions. Publishing is considered a "write", so you can limit who can do it. Another thing that might be useful in your scenario is to try using "Reports", and connect the "approved" experiments info to a report and then publish it. Here's a short video introducing reports .

By the way, please note that if the experiment/report/whatever is publis...

Can you paste here the code of the pipeline that you're trying to run?

Hey @<1545216070686609408:profile|EnthusiasticCow4> , for requirements pointing to packages in git repositories you need to make sure that the environment the agent is running in has the valid credentials to access the repo. In your case ( git+ssh ) it means you need to have a pair of ssh keys, and the public key should be registered with the repo.

The issue may be related to the fact that right now we have some edge cases when working with lightning >= 2.0, we should have better support in the upcoming release

Hey @<1574207113163444224:profile|ShallowCoyote86> , what exactly do you mean by "depends on private_repo_b "? Another question - after you push the changes, do you re-run script_a.py ?

Do you mean that you want your published experiments to be either “approved” or “not approved“ based on the presence of the attachments you mentioned ?

Hey @<1547390438648844288:profile|ScaryJellyfish75> , can you provide the whole code for the pipeline, and also mention what clearml version are you using?

Hey @<1564422650187485184:profile|ScaryDeer25> , we just released clearml==1.11.1rc2 which should solve the compatibility issues for lightning >= 2.0. Can you install it and check whether it solves your problem?

This is the method you're looking for None . But make sure you have a model saved on disk before using it. And if you don't want the model to be deleted from disk after it, make sure to set auto_delete_file=False

Hey @<1523704757024198656:profile|MysteriousWalrus11> , given your use case, did you consider passing the path to the dataset? Like an address to an S3 bucket

Glad I could be of help

Hey @<1681836303299121152:profile|RoundElk14> , it seems you are using a self-hosted ClearML server. This error you're getting happens because your email is not configured in the server. Ask your admin to perform the following steps:

• [The admin] Go to Settings > Users & Groups > Users and click on "+ Add User" where they will be prompted to specify the user's email
• [The user] Once the admin confirms that they did step 1, the user should first Sign In with their email to the server
• [The...

Wait, my config looks a bit different, what clearml package version are you using?

Yes, you can do that. But it may make it harder to identify the task later on

Hey @<1533257278776414208:profile|SuperiorCockroach75> , we do both, and much more 🙂

Could you please run the misbehaving example, try to add a breakpoint in clearml/backend_interface/task/task.py in Task.update_output_model on the line with url = output_model.update_weights( , and tell me what the value of model_path is? In case you're using virtual environments, clearml library should be installed somewhere in <virtual env directory>/lib/python3.10/site-packages/clearml/