Looks like its a hurray then 😄 🎉 🍾

Thanks a lot AgitatedDove14 !

JitteryCoyote63

So there will be no concurrent cached files access in the cache dir?

No concurrent creation of the same entry 🙂 It is optimized...

Great, you can test directly from the master 🙂
pip3 install -U git+

since we removed "." from the requirements?

Hmm I assume it is not running from the code directory...
(I'm still amazed it worked the first time)
Are you actually using "." ?

JitteryCoyote63 sure, this is how it was designed to work 🙂

I'm hoping we are ready to release

AgitatedDove14 The first time it installs and create the cache for the env, the second time it fails with:
Applying uncommitted changes ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found. clearml_agent: ERROR: Command '['/home/user/.clearml/venvs-builds.1/3.6/bin/python', '-m', 'pip', '--disable-pip-version-check', 'install', '-r', '/tmp/cached-reqsmncaxx45.txt']' returned non-zero exit status 1.

I carry this code from older versions of trains to be honest, I don't remember precisely why I did that

JitteryCoyote63 could you test the latest RC 😉
pip install clearml-agent==0.17.2rc4

Task.add_requirements('.')Should work

ok, now I actually remember why I used _update_requirements instead of add_requirements: The first overwrites all the other, the later only add to the already detected packages. Since my deps are listed in the dependencies of my setup.py, I don't want clearml to list the dependencies of the current environment

yes, in the code, i do:
task._wait_for_repo_detection() REQS_TASK = ["torch==1.3.1", "pytorch-ignite @ git+ ", "."] task._update_requirements(REQS_TASK) task.execute_remotely(queue_name=args.queue, clone=False, exit_process=True)

Since my deps are listed in the dependencies of my setup.py, I don't want clearml to list the dependencies of the current environment

Make sense 🙂
Okay let me check regrading the "." in the venv cache.

This is new right? it detects the local package, uninstalls it and reinstalls it?

Hurray ?

JitteryCoyote63 instead of _update_requirements, call the following before Task.init:
Task.add_requirements('torch', '1.3.1') Task.add_requirements('git+ ')

JitteryCoyote63 I found it 🙂
Are you working in docker mode or venv mode ?

So there will be no concurrent cached files access in the cache dir?

AgitatedDove14 I have a machine with two gpus and one agent per GPU. I provide the same trains.conf to both agents, so they use the same directory for caching venvs. Can it be problematic?

Sorry I missed the additional "." in the _update_requirements
Let me check ....

AgitatedDove14 Should I create an issue for this to keep track of it?

I had this problem before

Yep 🙂

ok, but will it install as expected the engine and its dependencies?

awesome thanks!

venv mode 🙂

Yey!

Installing collected packages: my-engine Attempting uninstall: my-engine Found existing installation: my-engine 1.0.0 Uninstalling my-engine-1.0.0: Successfully uninstalled my-engine-1.0.0 Successfully installed my-engine-1.0.0