Unanswered
			
			
 
			
	
		
			
		
		
		
		
	
			
		
		Hi There, Currently I Have A Clearml Pipeline That Takes In A Bunch Of Parameters For Various Tasks And Passes These Parameters Via Parameter_Override For Every Pipe.Add_Step(). However, I Have A Lot Of Parameters, And So My Pipeline Code Is A Little Unwi
 would that mean that multiple pre_callback()s would have to be defined for every add_step, since every step would have different configs? Sorry if there's something I'm missing, I'm still not quite good at working with ClearML yet.
Yes, you could have multiple callbacks, or you could check the name of each step via   node.name   and map the name of the node to its config.
One idea would be to have only 1 pipeline config file, that would look like:
step_1:
  # step_1 config
step_2:
  # step_2 config
then load this config with something like  config = hocon.load("config_file_path")
and in the callback you would do:node.job.task.set_configuration_object(config[node.name])
289 Views
				0
Answers
				
					 
	one year ago
				
					
						 
	one year ago