Thanks. We set this configuration and the client ran and submitted the job for remote execution (agent running k8s glue). However when the job runs, and tries to save into model repo, this error came up.
ClearML.storage - ERROR - Failed creating storage object S3://ecs.ai Reason; Missing key and secret for S3 storage access ( S3://ECS.ai ).

I remember being told that the ClearML.conf on the client will not be used in a remote execution like the above so I think this was the problem. I also tried to set the same credentials in 'web app cloud access' under profile. That didn't help as well.

The implication is that
All models can only be saved to a single model repo defined in the ClearML agent end. Users have no choice of defining their own repo destination of choice.
Is there any way out of this? I find it strange that users can't choose a model repo destination of their choice and had to rely on just one public bucket.