dug deeper. if i'm to make a guess.../root/clearml.conf -> used on startup of agent-services as a template of sorts to create .clearml_agent.<id>.cfg on demand -> this task-specific file is used to mount to /tmp/clearml_default.conf in a new container (docker in docker bc of the socket mounted to the agent-services) -> used to execute the task
I tried mounting a config file (in the structure of the one on github but with just the relevant s3 section) into the agent-services container at /root/clearml.conf and after restarting the container, it seems to have had an impact. thank you!
When I inspect the console of the task I'm trying to run, I see there's a call to cp /tmp/clearml.conf ~/default_clearml.conf in the docker command and that the volume /tmp/clearml.conf is picked up from the host at some custom-named file under /opt/clearml/agent/
what exactly is the agent-services container doing re: the config file? this is very confusing that /root/clearml.conf is having an impact when there's also this conf file auto-generated. is there docker-in-docker going on here?
thank you!
I'll add a volume mount to the services-agent container, and from what I understand that will become the template it uses?
is this the structure of the file?
None
or is it the "dot" syntax (like what shows up in the console when the task executes / your snippet)?
Hi @<1593051292383580160:profile|SoreSparrow36> , to do that, you can create an external config file on the host (it should not be in the /opt/clearml/config folder since that is used by the server), and add a volume mount in the services agent section to mount this file as clearml.conf into the services agent container. If you mount it into ~/clearml.conf the agent should pick it up. Note this file should only contain the required setting, something like:
sdk.aws.s3.use_credential_chain = True
And that's all 🙂