how to display and what to pass are two different things

yes

will it pass variables to the training containers?

we're using os.getenv in the script to get a value for these secrets

agent.extra_docker_arguments: ["-e", "MY_ENV_VAR=foo"]

So maybe that's something I missed? 😞 - Can you send a more complete log so I'll try to find out where this is coming from?

in the agent configuration, use:
agent.hide_docker_command_env_vars.extra_keys: ["DB_PASSWORD"]

add any other keys you want to hide to that list

agent.hide_docker_command_env_vars.extra_keys: ["DB_PASSWORD=password"]

like this? or ["DB_PASSWORD", "password"]

agent.hide_docker_command_env_vars.extra_keys: ["DB_PASSWORD"]

ah, I see, I still keep it in agent.extra_docker_arguments

If you're using docker, you can just add environment variables in the extra_docker_args section

the agent knows which args to look for and to split them on =

right now we can pass github secrets to the clearml agent training containers ( CLEARML_AGENT_GIT_PASS) to install private repos

we need a way to pass secrets to access our database with annotations

you can actually just provide a very large list of all env var names you might use for secrets in agent.hide_docker_command_env_vars.extra_keys , it doesn't matter if you actually usem the or not, the agent will just look for them and hide them when displaying

I guess I could edit docker-compose.yaml

obviously you should replace the passwords with something there 🙂

it works, but it's not very helpful since everybody can see a secret in logs:
Executing: ['docker', 'run', '-t', '--gpus', '"device=0"', '-e', 'DB_PASSWORD=password']

works like a charm!

1.2.3

thank you

Oh, wait, my bad 😆 - you need to tell the agent this env var should be hidden 🙂

this is probably what I need, thanks. I'll check if it works

nice!

DilapidatedDucks58 can you provide an example?

no no, just the env var name

by default the agent will hide the clearml secrets, AWS secrets etc.

DilapidatedDucks58 which ClearML Agent version are you using? I remember adding specific sanitation to the logs for these cases in the latest versions...

it only hides a pre-specified set of args (since most of the time you do want to see the value for non-secrets)

yes, exactly