For some reason, I thought it was when the agent was executing the task, not when it prints out the configuration...
I didn't realize we were talking about the config printout, since you've added these env vars to the default docker...
Hi SubstantialElk6 , what's your agent / ask version?
I also see this on my logs, noting that the config is read in but its still printing the supposedly hidden keys on the logs and UI.agent.hide_docker_command_env_vars.enabled = true agent.hide_docker_command_env_vars.extra_keys.0='TRAINS_AGENT_GIT_USER' ..... docker_cmd=harbor.ai/public/detectron2:v3 --env TRAINS_AGENT_GIT_USER=gituser
Can this issue be solved with vault? It doesn't make sense to expose secrets like that.
Hi, by agent logs i suppose you meant the logs from the ClearML server console panel?
Hi SubstantialElk6 , no worries, the reason it's not working for you is probably due to the fact that it's still in RC, and the agent version installed in the K8s POS is probably of the previous official version (or some older version). Can you share the more of the agents log (from the start)?
Its 1.0.0. As printed on the top of the logs in ClearML Server UI.
ok, i'll wait till i get my hands on vault then. thanks.
and out of curiosity, what did you think we were talking about? cos i didn't see anywhere else that might print the secrets.
Ah ok. So it will be fixed on the ClearML server web UI as well? (See screenshots).
Hi SuccessfulKoala55 ,i managed to install clearml-agent==1.0.1rc5. However, the same issues occur.
As for the UI - that's a good question - how can we hide it there if it's also editable? Does it make sense to let you edit it but not see it?
SubstantialElk6 this feature was only introduced in the 1.0.1 RC releases (i.e. it is not part of v1.0.0) - you can either use one of the RC releases, or wait for the official v1.0.1 (will be released very soon).
I also see this on my logs, noting that the config is read in but its still printing the supposedly hidden keys on the logs and UI.
agent.hide_docker_command_env_vars.enabled = true
agent.hide_docker_command_env_vars.extra_keys.0='TRAINS_AGENT_GIT_USER'
The agent prints all of the configuration it loads, regardless of whether any part of the code uses it or not...
Hi just wondering if I did something wrong here. Would k8s-glue be the reason is not working? I'm purchasing the enterprise version and if vault has the same problem it'll be a big issue.
The vault (as part of the pair version) can certainly solve it, without the need for env vars that can be visible in certain scenarios (like you showed)
Also, in the UI we can't really know your agent setting (also, you probably have multiple agents), so we can't know which env var should be masked...
Well, you're indeed running 1.0.1rc5 in there
That's an easy fix, you'll see it in the next RC 🙂