SoggyBeetle95 is this secret a per Task secret, or is it for the agent itself (I.e. for all Tasks the agent will spin)?
hi SoggyBeetle95
i reproduced the issue, could you confirm me that it is the issue ?
here is what i did :
i declared some secret env var in the agent section of clearml.conf i used extra_keys to have hidden on the console, it is indeed hidden but in the execution -> container section it is clear
Hmm, that indeed looks weird 😄 Let me reproduce. Just making sure, what version of the server \ SDK are you using?
Okay, I think I get what you're saying. How can I set up a clearml-agent with access-all creds with custom entries?
We would prefer for it to be per Task secret so team members can use their creds and passwords
Yes, I just define the env variables when using clearml-task instead of defining them in conf file
hey SoggyBeetle95
You're right that's an error on our part 🙂
Could you please open an issue in https://github.com/allegroai/clearml-server/issues so we can track it?
We'll update there once a fix for that issue will be released! 😄
SoggyBeetle95 you can configure the credentials in the clearml.conf running on the agent machines:
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L320
(I'm assuming these are storage credentials)
If you need general purpose env variables, you can ad them here:
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L149
with ["-e", "MY_VAR=MY_VALUE"]
Will these general env variables show in the ClearML app?
Ohh, if this is the case then it kind of makes sense to store on the Task itself. Which means the Task object will have to store it, and then the UI will display it :(
I think the actual solution is a vault , per user, which would allow users to keep their credentials on the sever, the agent to pass those to the Task when it spins it, based on the user. Unfortunately the vault feature is only available on the paid/enterprise version ( with RBAC etc.).
Does that make sense?
This is the server version:WebApp: 1.5.0-186 • Server: 1.5.0-186 • API: 2.18This is version of clearml-task:ClearML launch - launch any codebase on remote machine running clearml-agent Version 1.1.6
SoggyBeetle95 maybe it makes sense to configure the agent with an access-all credentials? Wdyt
Hi David, I will open an issue. Do you have an estimate when this could be fixed?
I reported the issue: https://github.com/allegroai/clearml-server/issues/146
AgitatedDove14 Maybe I'm missing something, but clearml-task has --docker_args argument which enables passing any env variable to the docker container that will be running the code.
My idea is that each team member adds their creds to --docker_args which would pass it to the docker container and the creds would be available for the task.
It seems to me that the only problem in passing credentials via --docker_args is that are currently shown in the plain form in the app.
Am I missing something obvious?
SoggyBeetle95 the question is, where does clearml stores these arguments, and the answer is on the Task object (from there the agent will take them and apply to the docker execution). Now since all users see all the tasks, they also see these arguments. Wdyt?