Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey, Currently On A Pro Saas Deployment Using Pipelines With The Gcp Autoscaler And Setting Some Credentials Into The Init Scripts Of My Provisionned Vms (Inside The Autoscaler Config) As Environment Variables, I Want To Pass Them To Specific Components O

Hey, currently on a PRO SaaS deployment using pipelines with the GCP autoscaler and setting some credentials into the init scripts of my provisionned VMs (inside the autoscaler config) as environment variables, I want to pass them to specific components on runtime to avoid having to put credentials into my docker images and I tried to do it like this:
@PipelineDecorator.component( return_values=['dataset_id'], cache=True, task_type=TaskTypes.data_processing, execution_queue='XXXX', docker_args="-e AWS_KEY_ID=$AWS_KEY_ID -e AWS_KEY_SECRET=$AWS_KEY_SECRET" )But the values are not interpolated by the shell and the resulting values of those environment variables simply being $AWS_KEY_ID and $AWS_KEY_SECRET .

Is there a way to pass env variables on the agent VM to the component container at the time execution ?

  
  
Posted 2 years ago
Votes Newest

Answers 4


You can use the web app to launch your auto-scalers and use the field Additional ClearML Configuration to add your env variables (or whatever other stuffs you need 🙂 )

  
  
Posted 2 years ago

Yup I already setup my aws configs for clearML that way but I needed to have generally accessible credentials too so I used the init script option in this config menu ^^

  
  
Posted 2 years ago

Hi FierceHamster54
If you are working with docker containers, you can extend the arguments using extra_docker_arguments of your clearml.conf and add the necessary env variables.

https://github.com/allegroai/clearml-agent/blob/5353e9c44d6bd373083b3191ffab5b6cb7c6652c/docs/clearml.conf#L155

  
  
Posted 2 years ago

Well solved, it's not as beautiful but I guess i can put them in a env file with an arbitrary name in the init script and just pass that file as exec argument...

  
  
Posted 2 years ago