BoredHedgehog47 this is basically a wizard explaining the steps, see the 3 tabs 🙂
BTW, you can launch an experiment directly from CLI with
seems like I'm passing in my own docker image which is then used at run time?
You are passing the Default docker image, if the Task does not list a specific docker image it will use the one you passed.
Yes this is "docker mode" (in venv mode no dockers are used, it just creates a new venv per experiment and installs everything inside the venv)
and the agent default runtime mode is docker correct?
Actually the default is venv mode, to run in docker mode add
--docker to the command line
So I could install all my system dependencies in my own docker image?
Correct, inside the docker it will inherit all the preinstalled packages, But it will also install any missing ones (based on the Task requirements. i.e. "installed packages" section)
Also what is the purpose of the
block in the clearml.conf? Where are those values used?
Access to S3 buckets / credentials (passed to the internal StorageManager artifacts uploads etc)
Does the clearml module parse the python packages?
Yes it analyzes the installed packages based on the actual mports you have in the code.
If I'm using a private pypi artifact server, would I set the PIP_INDEX_URL on the workers so they could retrieve those packages when that experiment is cloned and re-ran?
Correct 🙂 the agent basically calls
pip install on those packages, so if you configure it, with PIP_INDEX_URL it should just work like any other pip install