Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
WobblyFrog79
Moderator
7 Questions, 29 Answers
  Active since 08 February 2025
  Last activity one month ago

Reputation

0

Badges 1

19 × Eureka!
0 Votes
6 Answers
617 Views
0 Votes 6 Answers 617 Views
How to configure ClearML agent to keep pods around after they finish/fail? I want to debug a pod that crashes, but it gets deleted quickly
4 months ago
0 Votes
9 Answers
645 Views
0 Votes 9 Answers 645 Views
Hello, a question about pipelines. I have a repository with one pipeline using decorators, defined in pipeline.py . It uses multiple components that import c...
6 months ago
0 Votes
0 Answers
444 Views
0 Votes 0 Answers 444 Views
Is there a way to change the name of MongoDB databases used by ClearML? We want to have two self-hosted instances of ClearML that are going to use the same M...
4 months ago
0 Votes
1 Answers
441 Views
0 Votes 1 Answers 441 Views
Hello, I'm having issues with cloning a private repository that uses submodules with private repositories. I'm using CLEARML_AGENT_GIT_PASS and CLEARML_AGENT...
5 months ago
0 Votes
4 Answers
513 Views
0 Votes 4 Answers 513 Views
6 months ago
0 Votes
6 Answers
350 Views
0 Votes 6 Answers 350 Views
A question regarding using clearml-agent with k8s clusters. We use ClearML pipelines to train our models. The pods sometimes fail due to intermittent failure...
one month ago
0 Votes
3 Answers
516 Views
0 Votes 3 Answers 516 Views
How can I access task IDs of tasks running within a PipelineDecorator.pipeline ? I know PipelineController has get_running_nodes method, but how to achieve t...
6 months ago
0 Hello, A Question About Pipelines. I Have A Repository With One Pipeline Using Decorators, Defined In

when I add repo="." to definition of all my component decorators it works (but not the pipeline decorator), but it doesn’t work without that part… the problem i’m having now is that my components hang when executed in the cluster… i have 2 agents deployed (default and services queues)

6 months ago
0 Hello, A Question About Pipelines. I Have A Repository With One Pipeline Using Decorators, Defined In

I think so, but haven’t investigated what is the problem exactly, I’ll report it though.

6 months ago
0 Hello, A Question About Pipelines. I Have A Repository With One Pipeline Using Decorators, Defined In

the components start hanging indefinitely right after printing Starting Task Execution

6 months ago
6 months ago
0 Hello, A Question About Pipelines. I Have A Repository With One Pipeline Using Decorators, Defined In

Huh, I see. Thanks for your answers. How difficult would it be to implement some way to automatically inferring repository information for components, or having a flag repo_inherit (or similar) when defining a component (which would inhering repository information from the controller)? My workflow is based around executing code that lives in the same repository, so it’s cumbersome having to specify repository information all over the place, and changing commit hash as I add new code.

6 months ago
0 Hello, A Question About Pipelines. I Have A Repository With One Pipeline Using Decorators, Defined In

@<1523701205467926528:profile|AgitatedDove14> I managed to fix the issue FYI. I replaced from clearml import PipelineDecorator with from clearml.automation.controller import PipelineDecorator and it suddenly works. What a weird issue.

6 months ago
6 months ago
0 How To Configure Clearml Agent To Keep Pods Around After They Finish/Fail? I Want To Debug A Pod That Crashes, But It Gets Deleted Quickly

Awesome @<1729671499981262848:profile|CooperativeKitten94> , will definitely add that. It would also be very helpful if there was a way to delay deleting "completed/failed" pods. This is useful when something fails unexpectedly and ClearML logs are not enough to debug the issue. Does that make sense to you? I could contribute to your codebase if you're interested.

4 months ago
0 Hi Everyone, I'M Experiencing An Issue With Clearml Running On K8S. After Upgrading The Clearml Server Helm Chart From Version 7.11.5, I'M Seeing The Following Errors: In The Agent:

@<1523701205467926528:profile|AgitatedDove14> for me it hasn’t worked when I specified agentk8sglue.queue: "queue1,queue2" in the Helm chart options which should be possible according to documentation. What also hasn’t worked is that flag for creating a queue if it doesn’t exists ( agentk8sglue.createQueueIfNotExists ). Both failed parsing at runtime, so those are 2 bugs I’d say.

6 months ago
0 Hi Everyone, I'M Experiencing An Issue With Clearml Running On K8S. After Upgrading The Clearml Server Helm Chart From Version 7.11.5, I'M Seeing The Following Errors: In The Agent:

This hasn’t worked for me either, I use multiple queues instead. Another reason I also use multiple queues is because I need to specify different resource requirements for pods launched by each queue (CPU-only vs GPU).

6 months ago
0 A Question Regarding Using

@<1576381444509405184:profile|ManiacalLizard2> but the task controller has access to that information. Before deleting the pod, it could retrieve the exit code and status message that all pods provide, and log it under "Info" section in ClearML.

one month ago
0 Hello! I Am Setting Up A Clearml-Server With Self-Hosted Minio. Do I Would Like To Keep The Clearml.Conf As Default As Possible (Such That Users Do Not Need To Configure Much And Do Not Need Access To Mino Keys). I Am Trying To Use The Server-Config File

The way I understand it:

  • if you’re executing tasks locally (e.g. on your laptop) then you need this setting because the clearml package needs to know where to upload artifacts (artifacts aren’t proxied through the clearml-server they are rather uploaded directly to the storage of your choice)
  • if you’re executing code using ClearML agent, then you can configure agent the way I wrote earlier, and it will use your MinIO instance for uploading artifacts for all of the tasks it executes
6 months ago
0 A Question Regarding Using

@<1523701070390366208:profile|CostlyOstrich36> they don't as the pod is killed as soon as the process inside oversteps the memory limit

one month ago
0 A Question Regarding Using

Logging the pod exit code and status message would be very useful, before deleting the pod. The data scientists would see that an OOM happened and they wouldn't bother other teams to see what happened.

one month ago
0 A Question Regarding Using

I'm not talking about node failure, rather pod failure, which is out-of-memory in 99% of the cases.

one month ago
0 How To Configure Clearml Agent To Keep Pods Around After They Finish/Fail? I Want To Debug A Pod That Crashes, But It Gets Deleted Quickly

Hey @<1523701070390366208:profile|CostlyOstrich36> , could you provide any suggestions here, please?

4 months ago
0 How Can I Access Task Ids Of Tasks Running Within A

Thanks @<1806497735218565120:profile|BrightJellyfish46>

6 months ago
0 How Can I Access Task Ids Of Tasks Running Within A

Yes, that seems like an option as well. I found this as well (in case someone looks for it in the future):

p = PipelineDecorator.get_current_pipeline()
p.get_running_nodes()
6 months ago