Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone. I Am Working For The First Time With Clearml. And I Am Trying To Process Some Data Using Clearml Data Sdk And The Public Webui. I Am Experiencing A Lot Of Delay Getting The Dataset From Previous Tasks (Hours To Get 15Gb Of Data). Is This Norm

Hi everyone.
I am working for the first time with ClearML. And I am trying to process some data using ClearML Data sdk and the public WebUI. I am experiencing a lot of delay getting the dataset from previous tasks (hours to get 15gb of data). Is this normal? what could be the issue if not? thanks a lot for any reply!

  
  
Posted one year ago
Votes Newest

Answers 6


CostlyElephant1 I'm sure it will be faster if the server is closer to you, but if you're going to store lots of GBs I think using some object storage (like S3, or locally hosted minio, for examples), makes more sense (and will also take some load off the clearml server who's main job is serving your experiments and dashboards)

  
  
Posted one year ago

We are thinking to host our own server later on, so this wont be a problem in that case? but I was asking just know what was going on. thanks again

  
  
Posted one year ago

yes, the network seems fine. And it finally ends by getting the dataset. It just seems to hang more than it should between zips files.

  
  
Posted one year ago

And other downloads work OK?

  
  
Posted one year ago

Hi John, on the fileserver

  
  
Posted one year ago

Hi CostlyElephant1 , where is the data stored? on the fileserver or some s3 bucket or other solution?

  
  
Posted one year ago
1K Views
6 Answers
one year ago
one year ago
Tags