Hi  JitteryCoyote63 ,
When using the  --install-globally  switch, the  agent.package_manager.system_site_packages   flag will be automatically be set to  true .
While the  system_site_packages  flag simply controls how the agent configures  pip   (thus allowing you to use the agent in systems where you've already preinstalled some python packages), the  --install-globally  switch controls how the agent sets up virtual environments, and makes it install the required packages before creating the virtual environment, thus speeding up multiple venv creation (as the packages are only installed once) - doing that required using the  system_site_packages  flag, otherwise packages installed outside of the virtual environment won't be accessible...
Since pytorch is a special example (the agent will pick the correct pytorch based on the installed CUDA) , the agent will first make sure the file is downloaded, and then pass the resolving for pip to decide if it necessary to install. (bottom line, we downloaded the torch for no reason but it is cached so no real harm done) It might be the second package needs a specific numpy version... this resolving is don't by pip, not the agent specifically. Anyhow --system-site-packages is applicable on a venv level, so no need to worry about that.Make sense ?
Will this still be considered asÂ
global site-packages
This is a pip settings, I "think" it inherits from the local user's installation, but I would actually install with "sudo pip" that will definitely be "inherited"
I hitted enter too fast ^^
Installing them globally via$ pip install numpy opencv torch   will install locally with warning:Defaulting to user installation because normal site-packages is not writeable  , therefore the installation will take place in  ~/.local/lib/python3.6/site-packages  , instead of the default one. Will this still be considered as  global site-packages   and still be included in experiments envs? From what I tested it does
AgitatedDove14  I now tested with a real experiment, it works, but I saw two issues:
It first doesnt detect torch, downloads it but then says that it is already installed so it doesn't install it. One of the dependency of my repository is another repository (repo-2 in the logs). Both my repositories require  numpy  . When installing the first repository, it says  Requirement already satisfied: numpy in /home/workeruser/.local/lib/python3.6/site-packages  . Correct. But then it says  Installing collected packages: numpy, opencv-python, ...  . So I guess when installing the package from the second repo it looses somehow the  --system-site-packages   param
It should work 🙂 as long as the versions match, if they don't the venv will install the version you need (which is great, only penalty is the install, download wise it will be cached)
Thanks for the clarification  SuccessfulKoala55 ! A follow-up question:
I would like to install several packages (opencv, numpy, torch) in the  system-site-packages  so that they are available in each experiment (to reduce setup time of the experiments). Installing them globally via
I tested by installing flask in the default env -> which was installed in the  ~/.local/lib/python3.6/site-packages  folder. Then I created a venv with flag  --system-site-packages  . I activated the venv and flask was indeed available