ClearML FAQ | Hi! I Have A Dataset Like This: V1.0.0

Answered

Hi! I Have A Dataset Like This: V1.0.0

Hi! I have a dataset like this:
v1.0.0

file_1.csv
file_2.csvv1.0.1
file_2.csv
file_3.csvv1.0.2
file_3.csv
file_4.csvHow can I get a new dataset with:
file_1.csv
file_2.csv
file_3.csv
file_4.csvwhere I always keep the latest version of each file?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AbruptCow41
				
					0
					 × 1

Votes Newest

Answers 3

if I squash, this will rewrite the datasets, right? I want a new dataset, but keeping those there

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AbruptCow41
				
					0
					 × 1

You can create a new dataset and specify the parent datasets as all the previous ones. Is that something that would work for you ?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					EnthusiasticShrimp49
				
					0

that depends…would that only keep the latest version of each file?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AbruptCow41
				
					0
					 × 1

Write your answer

1K Views

3 Answers

one year ago

one year ago

Tags

Similar posts

Hi All, My Organisation Wants To Be Able To Trace The Changes Of All Dataset Versions From The Latest Version To The Root (Aka We Would Like To Be Able To Query And Display The Entire Version Tree). However, Clearml From My Knowledge Does Not Allow For Th

Hi!! I Have A Question, I Have A Dataset Saved In S3 Using Clearml, Is There A Way Of Getting The Full Path Of This Dataset In S3?? Something Like

Hey Guys, If I Set File_Upload_Retries Arbitrarily High Like 1000, Should I Be Good To Upload A Large Dataset >1Gb On Spotty Connections?

Hello, Im Having Huge Performance Issues On Large Clearml Datasets How Can I Link To Parent Dataset Without Parent Dataset Files. I Want To Create A Smaller Subset Of Parent Dataset, Like 5% Of It. To Achieve This, I Have To Call Remove_Files() To 60K It

I Tried To Get Data From Dataset, But Agent Always Look On Localhost:8081. I Change Host In Clearml.Conf But Have Same Error. How Can I Change Host Of Clearml Fileserver?

Hi! Is There Any Way To Add Git-Like Ignore File For Versioning Clearml Data? I Saw In Docs A Wildcard Argument When Files Are Added To A Dataset. How Can I Specify Ignoring Of Some File Types? For Example, I Want To Ignore Ipynb Checkpoints. How Can I Do

Hey All, I Have A Question That Might Have Been Asked Before But I Can'T Find An Answer On This Slack Channel Or The Github So I Hope Somebody Can Help Me. I Have Created A Dataset Via The Python Sdk With A Local Path As The Output Uri Like So:

Hello, Clearml.Task Has Great Ability Save Environment State Like Packages Uncommited Changes Etc. How Can I Get The Same Effect When Creating Clearml.Dataset, It Doesnt Save The Package List. When Looking At Dataset Page, I See The Dataset Upload Task Bu

Hello. I Am Currently Using A Clearml Server On-Premises. Thank You For The Great Tool! I Would Like To Ask About The Status Of The Clearml-Server And Clearml-Web Github Repositories. I Think That Clearml-Server 1.13 Is Currently Released As The Latest Ve

Hi All, I Was Trying To Use Clearml-Task To Run A Custom Docker(With Poetry To Install All The Python Dependencies And Activated The Environment) Using Clearml Gpu, But It Seems Like Clearml Always Create A Virtual Environment And Run The Python Script Fr