Hello, Im Having Huge Performance Issues On Large Clearml Datasets How Can I Link To Parent Dataset Without Parent Dataset Files. I Want To Create A Smaller Subset Of Parent Dataset, Like 5% Of It. To Achieve This, I Have To Call Remove_Files() To 60K It

Unanswered

AmiableSeaturtle81
Yes, I can also feel that slowness.

But what about if we have millions of pdfs and we need to generate a image dataset along with their bounding boxes. We cannot generate it on the fly while training the model as it will very very very slow.
So I’m doing all preprocessing the upload preprocessed data back to storage.
And while training again pulling preprocessed dataset from storage. 😄

  				
Posted 
	10 months ago

					More  		
  Report
		
					HomelyBluewhale47
				
					0
					 × 1

154 Views

0 Answers

10 months ago