Hi everyone,
I am working on a script that gets enqueued, and executes remotely on clearml-agent workers.
I got a "Hello World" version of the script running, the python env gets set up, and installs some basic packages I need from PyPI.
However, I need to include some private packages that are hosted on our GitLab Package Registry.
These can be installed by pip on the cli or from a requirements.txt
file by specifying the URL which looks like:
package_name==0.1.0 --index-url https://<username>:<token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple
But so far I have not had success with the automatic package detection.
I've also tried the manual methods like: task.set_packages
, Task.force_requirements_env_freeze(False, requirements_file="requirements.txt")
But I get errors during the environment setup once its running remotely:
ERROR: Invalid requirement: 'package==0.1.0 --index-url https://<username>:<token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple'
Hint: It looks like a path. File 'package==0.1.0 --index-url https://<username>:<token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple' does not exist.
RequirementsManager handler <clearml_agent.helper.package.external_req.ExternalRequirements object at 0x7f13e6d4ab90> raised exception: Failed installing GIT/HTTPs package 'package==0.1.0 --index-url https://<username>:<token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple'
clearml_agent: ERROR: Could not install task requirements!
Failed installing GIT/HTTPs package 'package==0.1.0 --index-url https://<username>:<token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple'