Hi @<1644147961996775424:profile|HurtStarfish47> , to handle this, you'll need to fun your code from inside your cloned git repository folder, and the ClearML SDK will auto detect it and log it on the task. When the ClearML Agent will run your code inside a container, it will clone the repository and install any required dependencies.
AWS credentials can be configured in the agent's clearml.conf
file (and it will pass them on to the task).
Any data obtained from outside the code can either be obtained dynamically by your code (i.e. downloaded from somewhere), or you can simply make sure it already exists inside the docker container - of course, you can simply use ClearML Datasets to store the dataset, and than get a local copy of it using your code.
As for pypi packages from private git repos, you can either add a direct reference to the repo in a requirements.txt file in your repo (a link complete with credentials), or have them preinstalled on the main system python inside the docker container.