Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, I Try To Pass Secrets From My Local Machine To A Remote Agent. I'M Using The

Hi all,
I try to pass secrets from my local machine to a remote agent. I'm using the clearml-task cli to kickoff my task to a specific queue ( None ) for the executing agent i use the autoscaler which I setup through the WebUI.

My issue is that I don't quite understand how I can pass those secrets to my script that runs with the autoscaler. I tried playing with the ~/clearml.conf file but it seems that the file does not influence anything within agent that runs through the autoscaler.

The only thing that kind of worked is using --docker_args "--env VAR1=value1 --env VAR2=value2" with clearml-task to set some enviroment variables which I guess I could use to set secrets, which I can query through the enviroment in my script. Interestingly --docker_args gets ignored/ has no effect if --docker isn't set. Also it seems --docker_bash_setup_script doesn't do anything - Is that the expected behavior?

My question is: Are those docker commands the best way to get some secrets into my script that runs in an autoscaler agent? E,g, using : --docker nvidia/cuda:11.4.3-runtime-ubuntu20.04 --docker_args "--env VAR1=value1 --env VAR2=value2" or is there a better way?

  
  
Posted one year ago
Votes Newest

Answers 5


No it doesn't, the agent has its own clearml.conf file.

I'm not too familiar with clearml on docker, but I do remember there are config options to pass some environment variables to docker.
You can then set your environment variables in any way you'd like before the container starts

  
  
Posted one year ago

It seems that it's possible to provide some configuration parameters when creating the instance through the webui. Which seems to be a reasonable solution for secrets that you want to keep for a longer time.

  
  
Posted one year ago

I know the ClearML enterprise offers a vault.
If these are static-ish, you can set them directly in the agent's config file.
If not, what we did was that before executing remotely, we uploaded environment variables of interest as parameters, and then loaded them in the remote task.
These can then be overwritten with *** after loading them.

  
  
Posted one year ago

For the former (static-ish environment variables), just add:

environment {
  VAR1: value1
  VAR2: value2
}

to the agent’s clearml.conf

  
  
Posted one year ago

Thank you @<1523701083040387072:profile|UnevenDolphin73> , regarding the clearml.conf - I don’t think I have access to that config for the agent in the autoscaler as I use a standard docker image.

I tried to make changes in my local clearml.conf on my laptop but that doesn’t seem to effect the “runner” in the autoscaler ?

  
  
Posted one year ago
1K Views
5 Answers
one year ago
one year ago
Tags
Similar posts