ClearML FAQ | Hi All! I Have A Question - I Have A Pipeline Build Out Of Multiple Tasks. One Of The Tasks Creates A File As An Output. I Don'T Need To Save This File Anywhere (Not In My Local Nor As An Artifact), But I Do Want To Use It An An Input For The Next Step In

Answered

Hi All! I Have A Question - I Have A Pipeline Build Out Of Multiple Tasks. One Of The Tasks Creates A File As An Output. I Don'T Need To Save This File Anywhere (Not In My Local Nor As An Artifact), But I Do Want To Use It An An Input For The Next Step In

Hi all!
I have a question - I have a pipeline build out of multiple tasks. One of the tasks creates a file as an output. I don't need to save this file anywhere (not in my local nor as an artifact), but I do want to use it an an input for the next step in the pipeline.
Is there a way to use this output without uploading this file as artifact?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					DangerousMole43
				
					0
					 × 1

Votes Newest

Answers 4

Thanks @<1523701070390366208:profile|CostlyOstrich36> !
so, both steps are running on the same machine. I would love to get your suggestion in this case.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					DangerousMole43
				
					0
					 × 1

yes!
I'll try that out. Thank you @<1523701070390366208:profile|CostlyOstrich36>

  				
Posted 
	one year ago

					More
				  		
  Report
		
					DangerousMole43
				
					0
					 × 1

Hi @<1539417873305309184:profile|DangerousMole43> , in that case I think you can simply save the file path as a configuration in the first step and then in the next step you can simply access this file path from the previous step. Makes sense?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Hi @<1539417873305309184:profile|DangerousMole43> , unless both steps run on the same machine you will have to upload it as an artifact somehow

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Write your answer

1K Views

4 Answers

one year ago