is would definetly seem that way - Although when I look at the error logs, the failure is actually in creating the venv-build folder - (under the task_repository subfolder) -
Repository cloning failed: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\clearml-admin\\.clearml\\venvs-builds\\3.1\\task_repository\\<repo>.git'
Hi @<1523701087100473344:profile|SuccessfulKoala55> - Each worker uses its own venv-builds folder
Hi @<1710827340621156352:profile|HungryFrog27> , are all of your agents using the same venv-builds folder?
From your description it seems what fails is setting the vcs cache? i.e. copying the cloned repository?
The system denies my deletion requiest since it deems the venv-builds dir as in use
Sorry, yes you have to take down the agent when you delete the cache 🙂
is it a shared network mount ? could you just delete the entire ~/.clearml on the host machine ?
but again - The system denies my deletion requiest since it deems the venv-builds dir as in use
@<1710827340621156352:profile|HungryFrog27> the venv-build folder is supposed to be deleted after each task is done. How did you end up with leftovers? Could it be windows was failing to delete it for some reason? That actually connects with you initial issue no?
Does this provide any more context? @<1523701087100473344:profile|SuccessfulKoala55>
huh. It is weird. is there any way to force deletion of it? it seems its still being held be some task and the server has been restarted several times since