AgitatedDove14 I have a machine with two gpus and one agent per GPU. I provide the same trains.conf to both agents, so they use the same directory for caching venvs. Can it be problematic?

yes, in the code, i do:
task._wait_for_repo_detection() REQS_TASK = ["torch==1.3.1", "pytorch-ignite @ git+ ", "."] task._update_requirements(REQS_TASK) task.execute_remotely(queue_name=args.queue, clone=False, exit_process=True)

JitteryCoyote63 I found it 🙂
Are you working in docker mode or venv mode ?

Looks like its a hurray then 😄 🎉 🍾

This is new right? it detects the local package, uninstalls it and reinstalls it?

Yey!

Task.add_requirements('.')Should work

Thanks a lot AgitatedDove14 !

JitteryCoyote63 sure, this is how it was designed to work 🙂

I carry this code from older versions of trains to be honest, I don't remember precisely why I did that

I had this problem before

AgitatedDove14 Should I create an issue for this to keep track of it?

Since my deps are listed in the dependencies of my setup.py, I don't want clearml to list the dependencies of the current environment

Make sense 🙂
Okay let me check regrading the "." in the venv cache.

Yep 🙂

Sorry I missed the additional "." in the _update_requirements
Let me check ....

Hurray ?

JitteryCoyote63 instead of _update_requirements, call the following before Task.init:
Task.add_requirements('torch', '1.3.1') Task.add_requirements('git+ ')

Great, you can test directly from the master 🙂
pip3 install -U git+

since we removed "." from the requirements?

AgitatedDove14 The first time it installs and create the cache for the env, the second time it fails with:
Applying uncommitted changes ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found. clearml_agent: ERROR: Command '['/home/user/.clearml/venvs-builds.1/3.6/bin/python', '-m', 'pip', '--disable-pip-version-check', 'install', '-r', '/tmp/cached-reqsmncaxx45.txt']' returned non-zero exit status 1.

ok, now I actually remember why I used _update_requirements instead of add_requirements: The first overwrites all the other, the later only add to the already detected packages. Since my deps are listed in the dependencies of my setup.py, I don't want clearml to list the dependencies of the current environment

JitteryCoyote63

So there will be no concurrent cached files access in the cache dir?

No concurrent creation of the same entry 🙂 It is optimized...

Hmm I assume it is not running from the code directory...
(I'm still amazed it worked the first time)
Are you actually using "." ?

So there will be no concurrent cached files access in the cache dir?

Installing collected packages: my-engine Attempting uninstall: my-engine Found existing installation: my-engine 1.0.0 Uninstalling my-engine-1.0.0: Successfully uninstalled my-engine-1.0.0 Successfully installed my-engine-1.0.0

venv mode 🙂

JitteryCoyote63 could you test the latest RC 😉
pip install clearml-agent==0.17.2rc4

ok, but will it install as expected the engine and its dependencies?

I'm hoping we are ready to release

awesome thanks!