Is There A Way I Can Create A Dataset As Part Of A Pipeline And Be Able To See That This Dataset Came From This Pipeline / Task ?

Answered

Is there a way I can create a dataset as part of a pipeline and be able to see that this dataset came from this pipeline / task ?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					TrickySheep9
				
					0
					 × 1

Votes Newest

Answers 3

Ah…that’s great!

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					TrickySheep9
				
					0
					 × 1

Sure:
Dataset.create(..., use_current_task=True)This will basically attach/make the main Task the Dataset itself (Dataset is a type of a Task, with logic built on top of it)
wdyt ?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Will try it out. Pretty impressed 🙂

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					TrickySheep9
				
					0
					 × 1

Write your answer

911 Views

3 Answers

3 years ago

one year ago

Hi, I Have A Question About The Pipeline, Especially About The Parallelism Part. We Are Considering Implementing A Use Case And Are Interested In Knowing Whether It Can Be Efficiently Managed Using Clearml Pipeline. Our Use Case Involves A Dataset That

Hi Everyone, I'M Encountering This Sqlalchemy Error While Running The Pipeline Using Asteroid Dataset "Sqlalchemy.Exc.Objectnotexecutableerror: Not An Executable Object: 'Select * From Asteroids Where Strftime("%Y-%M-%D", `Date`) <= Strftime("%Y-%M-%D",

Hello, I Created Dataset A, Then Created A New Version Of This Dataset (Dataset B) And Specified Dataset A As The Parent. After I Deleted Dataset A, I Can'T Download Dataset B Because Part Of Dataset A Is Missing. Is There A Way To Prevent The Dataset Fro

Hi, While Trying To Upload To Gcp Bucket From A Task Part Of Pipeline We Are Getting This Error:

I Have A Situation Where I’D Like To “Promote” The Pipeline (And Dataset) By Creating It In A Completely Separate Instance Of Clearml Server Which Is Used For Production Retraining (Vs. The Dev. Clearml Server That Is Used For Experiments) A) Is This Some

If I Have A Task And A Dataset Is Being Created In A Task, How Can I Get A “Link” That This Dataset Is Created In This Task, Similar To How Model Has The Task Where It Came From

Hello Community! Is There An Option To Only Download A Part Of A Dataset With .Get_Local_Copy()? I Imagine Something Like This, But I Can'T Find The Right Way To Do It.

Hi, Recently Came Across Trains And Very Impressed By The Work So Far. But A Problem Has Been Bugging Me, This Is Part Of The Trains Log Files I Thought Might Be Useful From Cloning And Enqueuing The Same Task On 2 Remote Machines. The First Machine Defau