ClearML uses the access and secret for creating the storage object, you can have those as env params too
You can look around there, and you can also open a GitHub issue for us 🙂
do you have all your AWS credentials in your ~/clearml.conf file?
If you can look around and maybe help with a PR that would be awesome 🙂
SuccessfulKoala55 Could you please point me to where I could quickly patch that in the code?
JitteryCoyote63 so you don’t need to use creds anymore?
you should have access and secret too
JitteryCoyote63 this is currently not supported in the ClearML Storage driver (only key/secret are)
Yea I really need that feature, I need to move away from key/secrets to iam roles
There is no need to add creds on the machine, since the EC2 instance has an attached IAM profile that grants access to s3. Boto3 is able retrieve the files from the s3 bucket
Why is it required in the case where boto3 can figure them out itself within the ec2 instance?
Well, I'm not sure it's that simple. It's basically in the _Boto3Driver class, but the issue is with what the code required to verify the URL
This seems to be the same issue like in https://clearml.slack.com/archives/CTK20V944/p1633599511350600
Whats the pyjwt version you are using?
I am confused now because I see in the master branch, the clearml.conf file has the following section:# Or enable credentials chain to let Boto3 pick the right credentials. # This includes picking credentials from environment variables, # credential file and IAM role using metadata service. # Refer to the latest Boto3 docs use_credentials_chain: falseSo it states that IAM role using metadata service should be supported, right?
SuccessfulKoala55 I was able to make it work with use_credentials_chain: true in the clearml.conf and the following patch: https://github.com/allegroai/clearml/pull/478
I will go for lunch actually 😄 back in ~1h
TimelyPenguin76 , no, I’ve only set the sdk.aws.s3.region = eu-central-1 param