Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hey, Currently On A Pro Saas Deployment Using Pipelines With The Gcp Autoscaler And Setting Some Credentials Into The Init Scripts Of My Provisionned Vms (Inside The Autoscaler Config) As Environment Variables, I Want To Pass Them To Specific Components O

Hey, currently on a PRO SaaS deployment using pipelines with the GCP autoscaler and setting some credentials into the init scripts of my provisionned VMs (inside the autoscaler config) as environment variables, I want to pass them to specific components on runtime to avoid having to put credentials into my docker images and I tried to do it like this:
@PipelineDecorator.component( return_values=['dataset_id'], cache=True, task_type=TaskTypes.data_processing, execution_queue='XXXX', docker_args="-e AWS_KEY_ID=$AWS_KEY_ID -e AWS_KEY_SECRET=$AWS_KEY_SECRET" )But the values are not interpolated by the shell and the resulting values of those environment variables simply being $AWS_KEY_ID and $AWS_KEY_SECRET .

Is there a way to pass env variables on the agent VM to the component container at the time execution ?

Posted one year ago
Votes Newest

Answers 4

Yup I already setup my aws configs for clearML that way but I needed to have generally accessible credentials too so I used the init script option in this config menu ^^

Posted one year ago

Well solved, it's not as beautiful but I guess i can put them in a env file with an arbitrary name in the init script and just pass that file as exec argument...

Posted one year ago

You can use the web app to launch your auto-scalers and use the field Additional ClearML Configuration to add your env variables (or whatever other stuffs you need 🙂 )

Posted one year ago

Hi FierceHamster54
If you are working with docker containers, you can extend the arguments using extra_docker_arguments of your clearml.conf and add the necessary env variables.


Posted one year ago