Hi CharmingStarfish14 , regarding #4 - this is exactly what ClearML Agent does 🙂
Hello, ClearML!
I have a question. My repository (with training code) includes link to another private repository in the requirements. It raises all kinds of issues with cloning a private repository within ClearML agent.
Example:
my_private_repo_1/requirements.txtnumpy pytorch==1.13.0
ClearML agent fails during pip install -r requirements.txt
, because it fails to clone this repository (i don't understand why, because it could copy my my_private_repo_1
in the first place.
I see a couple of solutions, but not sure which one of them would work:
Init ssh-agent within clearml worker Problem : if I will use --docker my_image
then it will probably fail there, because ssh credentials probably won't be forwarded into the dockerStore my .ssh credentials inside my docker image Problem : that doesn't sound secureUse my private Python Package IndexProblem : it will probably get me back to the problem, how to pass credentials to pipUse github submodules to link the repositoryNot sure that it will work, because it means that worker would need to call git fetch --all --recurse-submodules
and I'm not sure if ClearML is doing that.