👍

Will push my wip to the PR. Let me know what I am missing

Let me chech

Sounds good?

Yes, it's a bit confusing, the gist of it is that we wanted to have the ability to have diff configurations for diff buckets

Anything I am missing

Hope you don’t mind linking to that repo

LOL 😄

AgitatedDove14 - added it in bucket_config.py and sdk.conf but somehow value is not being picked up

Pushed - https://github.com/allegroai/clearml/pull/342/

This is awesome, thank you!

sdk.conf will add it to the default loaded values (as I think you deduced).
can copy paste the sdk.conf here? (maybe something is missing there?)

Yeah getting a better sense

Yeah sure

But functionality is working

Awesome , I will wait with the merge until tested internally .
There is a resale coming out after the weekend, once it is out I expect we will merge it.

Notice that the actual configuration that is used is the https://github.com/allegroai/clearml/blob/b21e93272682af99fffc861224f38d65b42c2354/clearml/backend_config/bucket_config.py#L23
But it is created here:
https://github.com/allegroai/clearml/blob/b21e93272682af99fffc861224f38d65b42c2354/clearml/backend_config/bucket_config.py#L199

Hi SubstantialElk6 I believe you just need to use clearml 1.0.5 , and make sure you rae passing the correct OS environment to the agent

well done man!

(or woman or in between, we are supportive as long as code is working 🙂 )

Ref of dvc doing about the same - https://github.com/iterative/dvc/blob/master/dvc/fs/s3.py#L127-L134

Hope you don’t mind linking to that repo

Pushed the changes, not sure if it’s fully right. Do let me know. But functionality is working

Hi, i was reading this thread and wondered which version of clearml-server and clearml-agent has this taken effect with?

I assume this is missing:
https://github.com/allegroai/clearml/blob/b21e93272682af99fffc861224f38d65b42c2354/clearml/backend_config/bucket_config.py#L102

Is updating that needed in this case?

YEY!

Got it working

You added it to the deprecated function 🙂
https://github.com/allegroai/clearml/blob/8496b99888deda45955bdd95f22dc929a88ec67b/clearml/backend_config/config.py#L400
See this line
https://github.com/allegroai/clearml/blob/8496b99888deda45955bdd95f22dc929a88ec67b/clearml/backend_config/config.py#L338

Sounds good, I assumed that was the case but I was not sure.
Let's make sure that in the clearml.conf we write it in the comment above the use_credentials_chain option, so that when users look for IAM roles configuration they can quick search for it 🙂

No we don’t need temp keys to work this way

He/Him 🙂