I am thinking about just installing this manually on the worker ...
If you install them system wide (i.e. with sudo) and add agent.package_manager.system_site_packages then they will always be available for you 🙂
And then also use
priority_optional_packages: ["carla"]
This actually means that it will always try to install the package clara first, but if it fails, it will no raise an error.
BTW: this would be a good use case for dockers, just saying 😉
But this seems like something that is not related to clearml 🙂 Anyways, thanks again for the explanations!
agent.package_manager.system_site_packages
can be used to inherit packages
Correct, it is basically venv with --system-site-packages
I do not think virtualenv nesting is support, if it was then in theory you could have executed the clearml-agent from virtual environment with system_site_packages turned on and then it would inherit from it. But again I'm not sure virtualenv supports it.
BTW: the latest clearml-agent RC already have venv caching (both pip/conda) 🙂
And then also use priority_optional_packages: ["carla"]
Yea, I am still trying to get docker to work with clearml. I do not have much experience with docker besides creating Dockerfiles and it seems like the ~/.ssh/config ownership seems broken when mounted into the container on my workstations.
Thank you. I am not trying to use this option to speed up the setup. I have some package (carla simulator PythonAPI) that has no pip support (only easy_install). So I am thinking about just installing this manually on the worker, so that tasks can assume, that carla is provided by the system
Sure 🙂
BTW: clearml-agent will mount your host .ssh into the docker to /root/.ssh by default.
So no need to do that manually