Hi SuccessfulKoala55 , thanks, tested the patch and its working as expected now.

Does the enterprise version support natively?

Ok let me try that.

Thanks SuccessfulKoala55 . Just pm'ed him.

Hi i missed this. Is there a recording?

Can i dig into the mongodb or ES to pull these data?

Yes I am.

Hi, any idea if i can acheive this? I just need a list of usernames.

Any idea where i can find the relevant API calls for this?

ok thanks! will try it out.

Likely network. Can you run a curl on ClearML server api server from jenkin stage and see if that gets through?

No issues. I know its hard to track open threads with Slack. I wish there's a plugin for this too. 🙂

Do you have more info on vault?
Actually it only make sense if the entire department or organisation are saving their models in a common repo. In our case this is not possible due to client security (e.g. training data from clients can potentially be 'reverse engineered' from trained models in future). So each department and even projects will need their own repo.

Hi Jake, thanks for the suggestion, let me try it out.

I thought of another potential way but not sure if the SDK supports it.
We will perform manual save and upload of model using vanilla boto3 and credentials passed in as env var. Use ClearML SDK to update the Model Repo on the location of the model, without ClearML uploading it explicitly.Would the above work?

Going back to the open source, I think that adding the credentials as part of the source code might allow to have "credentials" auto populate as part of the remote execution, wdyt?

Not sure how this will work when i can't supply the credentials to ClearML programatically.

It would make sense on a very large resource cluster. Unfortunately we only have less than 50 GPUs to share across. A multi-tenant SAAS would cut the resources into even more smaller clusters and not help with efficiency. Or would you have a suggestion?

yes its on purpose, each user would have their own AWS credentials for default_output_uri.

Ok thanks. that explains alot. We have been doing this wrongly the whole time, thinking that the clearml.conf on the client side would be acknowledged by the remote agent execution. In reality, only the API section is utilised.

Thanks. We set this configuration and the client ran and submitted the job for remote execution (agent running k8s glue). However when the job runs, and tries to save into model repo, this error came up.
ClearML.storage - ERROR - Failed creating storage object S3://ecs.ai Reason; Missing key and secret for S3 storage access ( S3://ECS.ai ).

I remember being told that the ClearML.conf on the client will not be used in a remote execution like the above so I think this was the problem. I also...

i see. Can i take it that when the client uses
task.execute_remotely(queue_name="1gpu", exit_process=True)then none of the content in its clearml.conf will be used, except for the API part. And Clearml simply uses whatever is on the Agent side.
api { # Notice: 'host' is the api server (default port 8008), not the web server. api_server: web_server: files_server: # Credentials are generated using the webapp, `
# Override with os environment: ...

My assumption is that the agent will have pulled that off the client's clearml.conf.

Hi SuccessfulKoala55 , just to add, my clearml.conf (client) and clearml.agent.conf (agent) can have differing values. I'm not sure which one takes precedence and if this could be the cause.

Setting the credentials on agent machine means the users cannot use their own credentials since an k8s glue agent serves multiple users.

Referencing your suggestion, we can configure output_uri on task.set_base_docker() but how should we do this for the credentials?

Hi. Anything that can point to activity by user.

Ok sure. Thanks.

I think the default action of clearml-agent k8s glue when running a task is to create a virtual env and installing the dependancies. So i'm just checking how to change that behaviour to look at global instead.

Hi, it's a preference from my developers. They preferred that the they install the python libraries into the images, load them up into the registry. In other words, they prefer to have libraries installed at image time.

Any comments on using the global python libraries without the need to 'pip install' anything?

I think in general, the 'published' action can be considered an 'approval'. The question is, how do we control who has the authority to 'publish'? The Web UI today does not support any uploads outside of the coding environment, would be nice it would be supported. But for now, the only workaround is to include parameters that stores document urls in the user properties.