Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello, I’M Using

Hello, Iโ€™m using Dataset.get_local_copy to get the data in the machine.
Everything works nicely except when there are two different users running the get_local_copy function. I understand this is a problem if the data has to be downloaded, but the not when the cache is not corrupted and all the data has already been downloaded. Does that mean that each user must have its own cache (i.e. the cache isnโ€™t shared)?

  
  
Posted 2 years ago
Votes Newest

Answers 21


yes

  
  
Posted 2 years ago

which the cache directory is set to be the same of everybody else

Just making sure, there are several cache folder/options in there so all are pointing to the same cache folders accordingly, yes?

  
  
Posted 2 years ago

Yep, makes sense ^^

  
  
Posted 2 years ago

CostlyOstrich36 any updates on this?

  
  
Posted 2 years ago

Yep

  
  
Posted 2 years ago

Hi RattyLouse61 ๐Ÿ™‚

Are these two different users using two sets of different credentials?

  
  
Posted 2 years ago

storage { cache { # Defaults to system temp folder / cache default_base_dir: "/scratch/clearml-cache" }this is it. Iโ€™ve only changed this bit

  
  
Posted 2 years ago

Is there a vital reason why you want to keep the two accounts separate when they run on the same machine?
Also, what if you try aligning all the cache folders for both configuration files to use the same folders?

  
  
Posted 2 years ago

This is interesting, I'll take a look into it ๐Ÿ™‚

  
  
Posted 2 years ago

Is there a vital reason why you want to keep the two accounts separate when they run on the same machine?

Iโ€™ve already implicitly answered this, but to be more precise, having multiple users allows to know who ran the experiment ๐Ÿ™‚

  
  
Posted 2 years ago

yes, sure ๐Ÿ™‚ thank you

  
  
Posted 2 years ago

cc. SuccessfulKoala55

  
  
Posted 2 years ago

RattyLouse61 , great!

  
  
Posted 2 years ago

Great, thanks!

  
  
Posted 2 years ago

the setup we have is that each ML person has an account on the ClearML server (you probably recall https://clearml.slack.com/archives/CTK20V944/p1650551097924099 ) and its own set of credentials. On the machine each ML person has a clearml.conf file in which the cache directory is set to be the same of everybody else

  
  
Posted 2 years ago

RattyLouse61 , looks like a bug, but I didn't have a chance to play with it myself yet. Maybe open a github issue to follow up on this?

  
  
Posted 2 years ago

so, yes each user has its own credentials

  
  
Posted 2 years ago

just to clarify, you mean clearml api credentials (not the aws ones)?

  
  
Posted 2 years ago

Iโ€™ll put together some more info and raise an issue

  
  
Posted 2 years ago

FYI: Iโ€™ve added a possible fix to the issue https://github.com/allegroai/clearml/issues/671#issuecomment-1146640498

  
  
Posted 2 years ago
722 Views
21 Answers
2 years ago
one year ago
Tags
Similar posts