Great, thank you very much TimelyPenguin76

Hi AgitatedDove14 , so isn't it ClearML best practice to create a draft pipeline to have the task on the server so that it can be cloned, modified and executed at any time?

Yes, before removing the 'default' queue I was able to shut down agents without specifying further options after the --stop command. I just had to run clearml-agent daemon --stop as many times as there were agents. Of course, I will open the issue as soon as possible :D

BTW, let's say I accidentally removed the 'default' queue from the queue list. As a result, when I try to stop an agent using clearml-agent daemon --stop , I get the following error:
clearml_agent: ERROR: APIError: code 400/707: No queue is tagged as the default queue for this company
I have already created another queue also called 'default' but it had no effect :/

Thanks! I will try it

Exactly, when 'extra' has a default value (in this case, 43), the argument preserves its original type. However, when 'extra' is a positional argument then it is transformed to 'str'

Well, instead of plain functions or files I use components because I need some of those steps to run on one machine and some on another. And it works perfectly fine (ignoring some minor bugs like this one). So I'm actually inserting component-decorated functions into 'helper_functions' parameter

That's right. There is no such package, it's just a custom module.
But that module uses tensorflow , and ClearML does not add it to the list of packages to install. The only solution available so far is to include it manually via Task.add_requirements ?

Sure, here is a trivial example:
from clearml import Dataset dataset = Dataset.create(dataset_name="Dataset_v1.1.3", dataset_project="Mocks") dataset.finalize() loaded_dataset = Dataset.get(dataset_id=dataset.id)

For any reason I can't get the values in their original types. Only the dictionary keys are returned as the raw nested dictionary, but the values remain casted.

Perfect, that's exactly what I was looking for 🙂 Thanks!

If I try to connect a dictionary of type dict[str, list] with task.connect , when retrieving this dictionary with task.get_parameter I get another dictionary dict[str, str] . Therefore, I see the same behavior using task.connect :/

Sure, just by changing a few things from the previous example:
` from clearml import Task

task = Task.init()
task.connect({"metrics": ["nmae", "bias", "r2"]})

metrics_names = task.get_parameter("General/metrics")

print(metrics_names)
print(type(metrics_names)) `

Hi AgitatedDove14 Yes, I think so. When I have more time next week I will take a closer look at it and elaborate an example.

Hi AgitatedDove14 ,
Any updates on the new ClearML release that fixes the bugs we mentioned in this thread? :)

Anyway, is there any way to retrieve the information stored in the RESULTS tab of ClearML Web UI?

Can you think of any other way to launch multiple pipelines concurrently? Since we have already seen it is only possible to run a single Pipelinecontroller in a single Python process

I have also tried with type hints and it still broadcasts to string. Very weird...

BTW I would really appreciate it if you let me know when you get it fixed 🙏

Oh, I see. I guess somehow I can retrieve that information via Task.logger , since it is stored in JSON format? Thanks!

I think it could be a convenient approach. The new parameter abort_on_failed_steps could be a list containing the name of the steps for which the pipeline will stop its execution if any of them fail (so that we can ignore other steps that are not crucial to continue the pipeline execution)

But maybe another solution would be to pass the configuration files paths as function arguments, then read and parse them inside the pipeline

AgitatedDove14 I ended up with two pipelines being executed until they completed the workflow but duplicating each of their steps. You can check it here:
https://clearml.slack.com/files/U02A5DGPMPU/F02SR3G9RDK/image.png

Hey AgitatedDove14 ! Any news on this? 🙂

AgitatedDove14 I have the strong feeling it must be an agent issue, because when I place PipelineDecorator.run_locally() before calling the pipeline, everything works perfectly. See:

Well, this is just a mock example 🙂 . In the real application I'm working on there will be more than one configuration file (in principle one for the data and one for the DL model). Regarding the fix, I am not in a hurry at the moment. I'll happily wait for tomorrow (or the day after) when the commit is pushed!

Great! Thank you 👍

Hi CostlyOstrich36 AgitatedDove14
Oh no, I am not trying to say that I am using each agent to run a single task. I have several agents listening to a number of queues so that they are busy most of the time. As we talked about, it is not possible to run multiple pipelines ( PipelineDecorator.pipeline ) simultaneously in a single process. That's why I had been testing locally launching pipelines in different subprocesses and this way I have managed to run several pipelines concurrently....

AgitatedDove14 After checking, I discovered that apparently it doesn't matter if each pipeline is executed by a different worker, the error persists. Honestly this has me puzzled. I'm really looking forward to getting this functionality right because it's an aspect that would make ClearML shine even more.

AgitatedDove14 So did you get the same results without step duplication?