ClearML FAQ | Hi Everyone

Answered

Hi Everyone

Hi everyone 👋
I'm working on a pipeline with PipelineController . I was wondering if you can make an "optional" step with a condition :
Imagine I create a pipeline with some steps, but sometimes I don't want to execute a step. This particular step is written as the pipeline should work when this step is executed and when not. Can you put a condition in the pipeline like if arg = None : skip step ? 🤔
Thanks 😁

  				
Posted 
	one year ago

					More
				  		
  Report
		
					MiniatureRobin9
				
					0
					 × 1

Votes Newest

Answers 2

Hi @<1523701435869433856:profile|SmugDolphin23> ! Thank you for your answer, I will try both your suggestions 😉

  				
Posted 
	one year ago

					More
				  		
  Report
		
					MiniatureRobin9
				
					0
					 × 1

Hi @<1546303293918023680:profile|MiniatureRobin9> ! When it comes to pipeline from functions/other tasks, this is not really supported. You could however cut the execution short when your step is being ran by evaluating the return values from other steps.

Note that you should however be able to skip steps if you are using pipeline from decorators

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SmugDolphin23
				
					0

Write your answer

866 Views

2 Answers

one year ago

one year ago

Tags

Similar posts

Hi Everyone! I Have Couple Of Questions Regarding The Agent And Spawned By It Worker Containers. I Have A 2 Step Pipeline:

Hello Everyone, I'M Working On A Flexible Machine Learning Pipeline Using Clearml, Covering Data Retrieval, Preprocessing, Training, And Evaluation. I Want To Add Hyperparameter Optimization (Hpo) As A Pipeline Step That Could Optimize The Preprocessing A

Hi, Guys! Encountered Strange Behavior. My Task Has Been Completed( De Facto ( I Can Clearly See That Code Inside The Task Has Been Executed)), But The Next Step In The Pipeline Didn'T Start+ The Task Is Still In The Running Process(De Jure) . I Have A Su

Hi Everyone ! I'M Working With Pipelinecontroller. Is It Possible To Create A Pipeline With Optional Steps ? To Clarify I Add An Example Of A Pipeline In The Picture. For Example, The User Would Be Able To Modify The Value Of A Parameter To Execute The St

Hello! Is There A Way To Override The Configuration Vault Parameters Of A Pipeline Step With The Add_Function_Step Method? I See In The Docs That Add_Step Method Has The Option To Override The Vault With The Configuration_Overrides Argument, But Not Add_F

Hi :slightly_smiling_face: we have a clearml pipeline that has a step that runs a multi gpu training (with hugginface), we need to invoke it with `accelerate launch` so we use `subprocess.run` inside the step but it hangs when finished. Is this the righ

Hi All, How Can I Have A Global Variable Used In A Pipeline Step? I Have To Define Them In Each Pipeline Step, Otherwise They Are Not Included In The Pipeline Step

Hi There. In A Clearml Pipeline Step With Docker, I Specify The Git Repo And Branch I Want To Use. How Can I Also Specify A Repos Optional Dependecies? It Uses Poetry For Deendency Management

Hi Community, I Am Trying To Run A Pipeline That Has Only A Single Step Defined As A Task, But I Get A Bizarre Error When I Run

Hey Guys! I Would Love To Know How To Integrate Hpo Inside Clearml Pipelines. I Have Made A Continuous Learning Pipeline With Data Etl And Model Training And As A Next Step, It Would Be Cool To Add Hpo. Most Of The Examples On The Website Create A New Ta