I really like the first idea, but I already see a problem if I make changes to the file, I will need to re-upload it every time?
I am currently using the repo cache, but unfortunately every time I run a new task with an existing cached repo, a new cache file is created.. very weird behaviour which I have already described in previous threads ( https://clearml.slack.com/archives/CTK20V944/p1651393535530439?thread_ts=1651063782.712379&cid=CTK20V944 )
I don't know if it has anything to do with it but I now see that the repo which is cloned and save in the cache is actually a different branch than the one which is pulled by the agent.
From the log file:repository = https://gitlab.com/data_science_team/PROJECT_NAME branch = MYBRANCHSOMENUMBER IP### DEBUG
warning: redirecting to https://gitlab.com/data_science_team/PROJECT_NAME.git/
Hi AbruptWorm50
I am currently using the repo cache,
What do you mean by "using the repo cache" ? This is transparent, the agent does that, users should not access that folder?
I also looked at the log you send, why do you think it is re-downloading the repo?
Hi AbruptWorm50 ,
You can use a stand alone file, this way the file will be saved to the backend and used every time without needing to clone the repo. What do you think?
CostlyOstrich36 Another clarification:
The master branch cache is stored at ". clearml/vcs-cache " - the code file doesn't exist there + the problem described above is occuring in this folder (multiple cache files of same repo).
While my branch is stored at " .clearml/venvs-builds/3.7/task_repository/"
Also, the repo itself is cached, so each time it will only fetch the delta.