Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
How Can I Control The

How can I control the ~/clearml.conf file being used by agent-services in the docker-compose stack for clearml-server ?

namely, if I enqueue a task, I notice /opt/clearml/agent/ gets a conf file with a hash created/overwritten, which is mounted into the docker container that will pick up the task.

How can I impact this config file? Namely I need to set sdk.aws.s3.use_credential_chain = True in there for now (more I'm sure later).

I've tried creating /opt/clearml/config/services.conf with storage_credentials as a key as per this documentation in which I modify the setting under aws/s3 sub-keys, but that seemed to have no impact.

Posted 12 months ago
Votes Newest

Answers 5

dug deeper. if i'm to make a guess...
/root/clearml.conf -> used on startup of agent-services as a template of sorts to create .clearml_agent.<id>.cfg on demand -> this task-specific file is used to mount to /tmp/clearml_default.conf in a new container (docker in docker bc of the socket mounted to the agent-services) -> used to execute the task

Posted 12 months ago

I tried mounting a config file (in the structure of the one on github but with just the relevant s3 section) into the agent-services container at /root/clearml.conf and after restarting the container, it seems to have had an impact. thank you!

When I inspect the console of the task I'm trying to run, I see there's a call to cp /tmp/clearml.conf ~/default_clearml.conf in the docker command and that the volume /tmp/clearml.conf is picked up from the host at some custom-named file under /opt/clearml/agent/

what exactly is the agent-services container doing re: the config file? this is very confusing that /root/clearml.conf is having an impact when there's also this conf file auto-generated. is there docker-in-docker going on here?

Posted 12 months ago

Hi @<1593051292383580160:profile|SoreSparrow36> , to do that, you can create an external config file on the host (it should not be in the /opt/clearml/config folder since that is used by the server), and add a volume mount in the services agent section to mount this file as clearml.conf into the services agent container. If you mount it into ~/clearml.conf the agent should pick it up. Note this file should only contain the required setting, something like:

 sdk.aws.s3.use_credential_chain = True

And that's all 🙂

Posted 12 months ago

thank you!
I'll add a volume mount to the services-agent container, and from what I understand that will become the template it uses?

is this the structure of the file?

or is it the "dot" syntax (like what shows up in the console when the task executes / your snippet)?

Posted 12 months ago

Correct 🙂

Posted 11 months ago