If i were to push the private package to, say artifactory, is it possible to use that do the install?
Yes that's the recommended way 🙂
You add the private repo here, for the agent to use:
https://github.com/allegroai/clearml-agent/blob/e93384b99bdfd72a54cf2b68b3991b145b504b79/docs/clearml.conf#L65
AgitatedDove14 - apologies for late reply. So to give context this in a Sagemaker notebook which has conda envs.
I use a lifecycle like this to pip install a package (a .tar.gz downloaded from s3) in a conda env- https://github.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/blob/master/scripts/install-pip-package-single-environment/on-start.sh
In the notebook I can do things like create experiments and so on. Now the problem is in running the cloned experiment and also for pipelines running remotely etc.
# Python 3.6.13 | packaged by conda-forge | (default, Feb 19 2021, 05:36:01) [GCC 9.3.0] argparse == 1.4.0 boto3 == 1.17.70 minerva == 0.1.0 torch == 1.7.1 torchvision == 0.8.2
and I install the tar
I think the only way to do that is add it into the docker bash setup script (this is a bash script executed before Task)
TrickySheep9 is this a conda package or a wheel you are installing manually ?
When you install using pip <filename> you should end up with something like:minerva @ file://... or minerva @ https://...
If i were to push the private package to, say artifactory, is it possible to use that do the install?
The minerva one is my custom package AgitatedDove14
Could you send the "installed packages" section of the Task that was created in the notebook ?
Very bad at python packaging terms - but I use pyproject.toml and python -m build that generates a wheel and a tar and I install the tar