It seems that it's possible to provide some configuration parameters when creating the instance through the webui. Which seems to be a reasonable solution for secrets that you want to keep for a longer time.

No it doesn't, the agent has its own clearml.conf file.

I'm not too familiar with clearml on docker, but I do remember there are config options to pass some environment variables to docker.
You can then set your environment variables in any way you'd like before the container starts

Thank you @<1523701083040387072:profile|UnevenDolphin73> , regarding the clearml.conf - I don’t think I have access to that config for the agent in the autoscaler as I use a standard docker image.

I tried to make changes in my local clearml.conf on my laptop but that doesn’t seem to effect the “runner” in the autoscaler ?

For the former (static-ish environment variables), just add:

environment {
  VAR1: value1
  VAR2: value2
}

to the agent’s clearml.conf

I know the ClearML enterprise offers a vault.
If these are static-ish, you can set them directly in the agent's config file.
If not, what we did was that before executing remotely, we uploaded environment variables of interest as parameters, and then loaded them in the remote task.
These can then be overwritten with *** after loading them.