Hi, Currently Running

Answered

Hi, currently running Dataset.list_datasets() on a giant list of datasets takes a very long time to return the full datasets list. Similarly running Dataset.get() also takes a fairly long time to run and retrieve a single dataset object. Does anyone know if there is a way to retrieve a whole list of datasets and getting a single dataset faster?

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					CumbersomeDuck6
				
					0
					 × 1

Votes Newest

Answers 4

Hm I see. Because I am wondering how come when I view clearml datasets in the clearml GUI, it can just show all my data almost instantly, but when using ClearML SDK, it is so slow?

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					CumbersomeDuck6
				
					0
					 × 1

So you're using the community server? Response time really depends on the resources of the machine that is running the server and amount of data to filter

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

currently not, but the aim is to host it locally. Would that make retrieval much faster?

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					CumbersomeDuck6
				
					0
					 × 1

Hi @<1747428509627715584:profile|CumbersomeDuck6> , are you using a self hosted server?

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Write your answer

596 Views

4 Answers

11 months ago