ClearML FAQ | Hi Everyone

Answered

Hi Everyone

Hi everyone 👋
I'm working on a pipeline with PipelineController . I was wondering if you can make an "optional" step with a condition :
Imagine I create a pipeline with some steps, but sometimes I don't want to execute a step. This particular step is written as the pipeline should work when this step is executed and when not. Can you put a condition in the pipeline like if arg = None : skip step ? 🤔
Thanks 😁

  				
Posted 
	10 months ago

					More
				  		
  Report
		
					MiniatureRobin9
				
					0
					 × 1

Votes Newest

Answers 2

Hi @<1546303293918023680:profile|MiniatureRobin9> ! When it comes to pipeline from functions/other tasks, this is not really supported. You could however cut the execution short when your step is being ran by evaluating the return values from other steps.

Note that you should however be able to skip steps if you are using pipeline from decorators

  				
Posted 
	10 months ago

					More
				  		
  Report
		
					SmugDolphin23
				
					0

Hi @<1523701435869433856:profile|SmugDolphin23> ! Thank you for your answer, I will try both your suggestions 😉

  				
Posted 
	10 months ago

					More
				  		
  Report
		
					MiniatureRobin9
				
					0
					 × 1

Write your answer

500 Views

2 Answers

10 months ago

10 months ago

Tags

Similar posts

Hi :slightly_smiling_face: we have a clearml pipeline that has a step that runs a multi gpu training (with hugginface), we need to invoke it with `accelerate launch` so we use `subprocess.run` inside the step but it hangs when finished. Is this the righ

Hi All, How Can I Have A Global Variable Used In A Pipeline Step? I Have To Define Them In Each Pipeline Step, Otherwise They Are Not Included In The Pipeline Step

Hi There. In A Clearml Pipeline Step With Docker, I Specify The Git Repo And Branch I Want To Use. How Can I Also Specify A Repos Optional Dependecies? It Uses Poetry For Deendency Management

Hi Community, I Am Trying To Run A Pipeline That Has Only A Single Step Defined As A Task, But I Get A Bizarre Error When I Run

Hey Guys! I Would Love To Know How To Integrate Hpo Inside Clearml Pipelines. I Have Made A Continuous Learning Pipeline With Data Etl And Model Training And As A Next Step, It Would Be Cool To Add Hpo. Most Of The Examples On The Website Create A New Ta

Hi All, I Have A Pipeline Step That Does A Shutil.Copytree And I Would Like The Progress To Be Shown On The Task. Is There A Way, Since Shutil Copytree Has No Tqdm?

A Question On Pipeline Artifacts And Their Use In Steps. I Want To Add An Artifact To The Pipeline (See Below), And Use It As Input To A Function Step. But The Following Does Not Work Because

Hello ! I Am Having Some Issues While Running A Pipeline Remotely (Same Device But Queue + Agent) . Hitting The Following Issue On My First Step:

Hey Everyone, I'M Using Clearml Pipeline, I Have A Pre-Processing Step Which I Would Like To Use There Small Instance, And Processing Step Which Executed On A Gpu Queue. The Initialisation Time Of The Gpu Instance Is Very Long. Is There A Way To Configure

Hi All, I Have A Pipeline Construct From Multiple Base Tasks. Is There A Way To Get Output Or Notification Somehow From One Of The Steps In The Pipeline On The Fly? I Want To Know If I Can Trigger The A Child Step Based On Some "Go" Command That I Will Pu