As we can’t create keys in our AWS due to infosec requirements
Hi @<1687643893996195840:profile|RoundCat60>
I think the best way will be to configure a default output_uri
to be used by all tasks: None , under default_output_uri
just write your bucket path ( None ).
When using S3/google storage/azure, you will also need to add your credentials in this section - None (s3 in this link),
and install clearml with storage requirements:pip install clearml[s3]
for S3pip install clearml[azure]
for azurepip install clearml[gs]
for google storage
Same credentials configuration for the ClearML-Agent.
Notice that when a task is created, in the UI, under EXECUTION
tab, you can find (and change if you like) the output destination.
@<1687643893996195840:profile|RoundCat60> I'm assuming we are still talking about the S3 credentials, sadly no 😞
Are you familiar with boto and IAM roles ?
@<1523701205467926528:profile|AgitatedDove14> - any thoughts on this. Would like to use profile / iam roles as well.
Is it possible to use an IAM role rather than user credentials in the clearml.conf file?
or have I got this wrong, and it's the clearml-agent that needs to read/write to S3?
None
So this is the only place we need to change to support it, do you feel like messing around with it and adding IAM roles ?
Was asking about using iam roles without keys
yep still referring to the S3 credentials, somewhat familiar with boto and IAM roles
As we can’t create keys in our AWS due to infosec requirements
Hmmm
is there any documentation for connecting to an S3 bucket?
@<1687643893996195840:profile|RoundCat60> Looks like the docs have not caught up yet with recent structure change in the repo which renamed the 'server' folder to 'apiserver'.
So... the correct link would be None