Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi All, I'M Trying To Create Pipeline With Decorator. When I Run The Code To Push The Pipeline To Clearml Server, It Seems The Pipeline Is Pending In "Services" Queue. But My Worker Is Listening To "Default" Queue, So The Pipeline Is Not Executed. I Read

Hi all,
I'm trying to create pipeline with decorator. When I run the code to push the pipeline to clearml server, it seems the pipeline is pending in "services" queue. But my worker is listening to "default" queue, so the pipeline is not executed. I read in the doc https://clear.ml/docs/latest/docs/pipelines/pipelines_sdk_function_decorators and there are 2 arguments:
default_queue - The default https://clear.ml/docs/latest/docs/fundamentals/agents_and_queues#what-is-a-queue in which to enqueue all pipeline steps
pipeline_execution_queue - The queue in which to enqueue the pipeline controller task
so these 2 queues supposed to be different, right? How should I execute my pipeline? Should I set the pipeline_execution_queue to "default" or should I let my worker to listen on "services" queue?

Posted one year ago
Votes Newest

Answers 2

thank CostlyOstrich36 so should I point the clearml-agent to listen to "services" queue instead of "default" queue? Would it be able to process the pipeline steps?
If I understand correctly, the agent will spin additional pod to run the steps in the pipeline, right?

Also just another question: How do I set the git credential for the agent? In the values file https://github.com/allegroai/clearml-helm-charts/blob/main/charts/clearml-agent/values.yaml
look like I can set git user and pass, but can I set ssh key instead?

Posted one year ago

Hi PunyWoodpecker71 ,

It's best to run the pipeline controller in the services queue because the assumption is that the controller doesn't require much compute power as opposed to steps that can be resource exhausting (depends on pipeline of course)

Posted one year ago
2 Answers
one year ago
one year ago
Similar posts