Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello, I’M Using

Hello, Iโ€™m using Dataset.get_local_copy to get the data in the machine.
Everything works nicely except when there are two different users running the get_local_copy function. I understand this is a problem if the data has to be downloaded, but the not when the cache is not corrupted and all the data has already been downloaded. Does that mean that each user must have its own cache (i.e. the cache isnโ€™t shared)?

  
  
Posted one year ago
Votes Newest

Answers 21


storage { cache { # Defaults to system temp folder / cache default_base_dir: "/scratch/clearml-cache" }this is it. Iโ€™ve only changed this bit

  
  
Posted one year ago

Hi RattyLouse61 ๐Ÿ™‚

Are these two different users using two sets of different credentials?

  
  
Posted one year ago

yes

  
  
Posted one year ago

so, yes each user has its own credentials

  
  
Posted one year ago

Yep

  
  
Posted one year ago

the setup we have is that each ML person has an account on the ClearML server (you probably recall https://clearml.slack.com/archives/CTK20V944/p1650551097924099 ) and its own set of credentials. On the machine each ML person has a clearml.conf file in which the cache directory is set to be the same of everybody else

  
  
Posted one year ago

Is there a vital reason why you want to keep the two accounts separate when they run on the same machine?

Iโ€™ve already implicitly answered this, but to be more precise, having multiple users allows to know who ran the experiment ๐Ÿ™‚

  
  
Posted one year ago

Is there a vital reason why you want to keep the two accounts separate when they run on the same machine?
Also, what if you try aligning all the cache folders for both configuration files to use the same folders?

  
  
Posted one year ago

This is interesting, I'll take a look into it ๐Ÿ™‚

  
  
Posted one year ago

Yep, makes sense ^^

  
  
Posted one year ago

which the cache directory is set to be the same of everybody else

Just making sure, there are several cache folder/options in there so all are pointing to the same cache folders accordingly, yes?

  
  
Posted one year ago

CostlyOstrich36 any updates on this?

  
  
Posted one year ago

yes, sure ๐Ÿ™‚ thank you

  
  
Posted one year ago

Iโ€™ll put together some more info and raise an issue

  
  
Posted one year ago

FYI: Iโ€™ve added a possible fix to the issue https://github.com/allegroai/clearml/issues/671#issuecomment-1146640498

  
  
Posted one year ago

RattyLouse61 , great!

  
  
Posted one year ago

cc. SuccessfulKoala55

  
  
Posted one year ago

RattyLouse61 , looks like a bug, but I didn't have a chance to play with it myself yet. Maybe open a github issue to follow up on this?

  
  
Posted one year ago

Great, thanks!

  
  
Posted one year ago

just to clarify, you mean clearml api credentials (not the aws ones)?

  
  
Posted one year ago
565 Views
21 Answers
one year ago
one year ago
Tags
Similar posts