This is the server version:WebApp: 1.5.0-186 • Server: 1.5.0-186 • API: 2.18This is version of clearml-task:ClearML launch - launch any codebase on remote machine running clearml-agent Version 1.1.6
Yes, I just define the env variables when using clearml-task instead of defining them in conf file
hi SoggyBeetle95
i reproduced the issue, could you confirm me that it is the issue ?
here is what i did :
i declared some secret env var in the agent section of clearml.conf i used extra_keys to have hidden on the console, it is indeed hidden but in the execution -> container section it is clear
SoggyBeetle95 maybe it makes sense to configure the agent with an access-all credentials? Wdyt
SoggyBeetle95 you can configure the credentials in the clearml.conf running on the agent machines:
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L320
(I'm assuming these are storage credentials)
If you need general purpose env variables, you can ad them here:
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L149
with ["-e", "MY_VAR=MY_VALUE"]
hey SoggyBeetle95
You're right that's an error on our part 🙂
Could you please open an issue in https://github.com/allegroai/clearml-server/issues so we can track it?
We'll update there once a fix for that issue will be released! 😄
We would prefer for it to be per Task secret so team members can use their creds and passwords
Hi David, I will open an issue. Do you have an estimate when this could be fixed?
Ohh, if this is the case then it kind of makes sense to store on the Task itself. Which means the Task object will have to store it, and then the UI will display it :(
I think the actual solution is a vault , per user, which would allow users to keep their credentials on the sever, the agent to pass those to the Task when it spins it, based on the user. Unfortunately the vault feature is only available on the paid/enterprise version ( with RBAC etc.).
Does that make sense?
AgitatedDove14 Maybe I'm missing something, but clearml-task has --docker_args argument which enables passing any env variable to the docker container that will be running the code.
My idea is that each team member adds their creds to --docker_args which would pass it to the docker container and the creds would be available for the task.
It seems to me that the only problem in passing credentials via --docker_args is that are currently shown in the plain form in the app.
Am I missing something obvious?
Hmm, that indeed looks weird 😄 Let me reproduce. Just making sure, what version of the server \ SDK are you using?
Will these general env variables show in the ClearML app?
SoggyBeetle95 the question is, where does clearml stores these arguments, and the answer is on the Task object (from there the agent will take them and apply to the docker execution). Now since all users see all the tasks, they also see these arguments. Wdyt?
I reported the issue: https://github.com/allegroai/clearml-server/issues/146
SoggyBeetle95 is this secret a per Task secret, or is it for the agent itself (I.e. for all Tasks the agent will spin)?
Okay, I think I get what you're saying. How can I set up a clearml-agent with access-all creds with custom entries?