or have I got this wrong, and it's the clearml-agent that needs to read/write to S3?
Is it possible to use an IAM role rather than user credentials in the clearml.conf file?
As we can’t create keys in our AWS due to infosec requirements
Hmmm
None
So this is the only place we need to change to support it, do you feel like messing around with it and adding IAM roles ?
@<1687643893996195840:profile|RoundCat60> Looks like the docs have not caught up yet with recent structure change in the repo which renamed the 'server' folder to 'apiserver'.
So... the correct link would be None
@<1687643893996195840:profile|RoundCat60> I'm assuming we are still talking about the S3 credentials, sadly no 😞
Are you familiar with boto and IAM roles ?
yep still referring to the S3 credentials, somewhat familiar with boto and IAM roles
is there any documentation for connecting to an S3 bucket?
Same credentials configuration for the ClearML-Agent.
Notice that when a task is created, in the UI, under EXECUTION
tab, you can find (and change if you like) the output destination.
Hi @<1687643893996195840:profile|RoundCat60>
I think the best way will be to configure a default output_uri
to be used by all tasks: None , under default_output_uri
just write your bucket path ( None ).
When using S3/google storage/azure, you will also need to add your credentials in this section - None (s3 in this link),
and install clearml with storage requirements:pip install clearml[s3]
for S3pip install clearml[azure]
for azurepip install clearml[gs]
for google storage
As we can’t create keys in our AWS due to infosec requirements
@<1523701205467926528:profile|AgitatedDove14> - any thoughts on this. Would like to use profile / iam roles as well.
Was asking about using iam roles without keys