Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone. I Am Working For The First Time With Clearml. And I Am Trying To Process Some Data Using Clearml Data Sdk And The Public Webui. I Am Experiencing A Lot Of Delay Getting The Dataset From Previous Tasks (Hours To Get 15Gb Of Data). Is This Norm

Hi everyone.
I am working for the first time with ClearML. And I am trying to process some data using ClearML Data sdk and the public WebUI. I am experiencing a lot of delay getting the dataset from previous tasks (hours to get 15gb of data). Is this normal? what could be the issue if not? thanks a lot for any reply!

  
  
Posted 2 years ago
Votes Newest

Answers 6


CostlyElephant1 I'm sure it will be faster if the server is closer to you, but if you're going to store lots of GBs I think using some object storage (like S3, or locally hosted minio, for examples), makes more sense (and will also take some load off the clearml server who's main job is serving your experiments and dashboards)

  
  
Posted 2 years ago

We are thinking to host our own server later on, so this wont be a problem in that case? but I was asking just know what was going on. thanks again

  
  
Posted 2 years ago

Hi John, on the fileserver

  
  
Posted 2 years ago

Hi CostlyElephant1 , where is the data stored? on the fileserver or some s3 bucket or other solution?

  
  
Posted 2 years ago

yes, the network seems fine. And it finally ends by getting the dataset. It just seems to hang more than it should between zips files.

  
  
Posted 2 years ago

And other downloads work OK?

  
  
Posted 2 years ago
1K Views
6 Answers
2 years ago
2 years ago
Tags