Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All

Hi All 🙂
Every time I run a new task my whole git repo is cloned, to pull a single .py file. Is there a way to define (I suppose in the clearml.conf file) the non root repo ? e.g.: instead of cloning projectrepo , clone projectrepo/my_branch/src/training

  
  
Posted 2 years ago
Votes Newest

Answers 6


I don't know if it has anything to do with it but I now see that the repo which is cloned and save in the cache is actually a different branch than the one which is pulled by the agent.

From the log file:
repository = https://gitlab.com/data_science_team/PROJECT_NAME branch = MYBRANCH
SOMENUMBER IP### DEBUG
warning: redirecting to https://gitlab.com/data_science_team/PROJECT_NAME.git/

  
  
Posted 2 years ago

Hi AbruptWorm50

I am currently using the repo cache,

What do you mean by "using the repo cache" ? This is transparent, the agent does that, users should not access that folder?

I also looked at the log you send, why do you think it is re-downloading the repo?

  
  
Posted 2 years ago

CostlyOstrich36 Another clarification:
The master branch cache is stored at ". clearml/vcs-cache " - the code file doesn't exist there + the problem described above is occuring in this folder (multiple cache files of same repo).
While my branch is stored at " .clearml/venvs-builds/3.7/task_repository/"

  
  
Posted 2 years ago

I really like the first idea, but I already see a problem if I make changes to the file, I will need to re-upload it every time?
I am currently using the repo cache, but unfortunately every time I run a new task with an existing cached repo, a new cache file is created.. very weird behaviour which I have already described in previous threads ( https://clearml.slack.com/archives/CTK20V944/p1651393535530439?thread_ts=1651063782.712379&cid=CTK20V944 )

  
  
Posted 2 years ago

Also, the repo itself is cached, so each time it will only fetch the delta.

  
  
Posted 2 years ago

Hi AbruptWorm50 ,

You can use a stand alone file, this way the file will be saved to the backend and used every time without needing to clone the repo. What do you think?

  
  
Posted 2 years ago
925 Views
6 Answers
2 years ago
one year ago
Tags