I think I can set
AWS_PROFILE . I have theoretically placed the
clearml.conf in the correct place in the container:
$HOME/clearml.conf . But I get an error that’s actually not very helpful, I still think the issue is getting boto3 credentials but I’m not sure I can prove it at this point. I posted the error in the community channel https://clearml.slack.com/archives/CTK20V944/p1653507863528969
so a little more clarity. I get nearly this same error when downloading the same object from boto3 if I try to get at it from an s3 resource like so:
` bad_s3_resource = boto3.resource("s3")
obj = bad_s3_resource.Object(bucket, key)
botocore.exceptions.NoCredentialsError: Unable to locate credentials
Only when I set the boto3 session with the appropriate profile does it work:sess = boto3.session.Session(profile_name="foo-profile")
s3_resource = sess.resource("s3")
obj = s3_resource.Object(bucket, key)
<botocore.response.StreamingBody at 0x1463120d0> `
So this makes me wonder if there is a similar mechanism that’s giving me this error.
yes I did finally resolve this. If my memory serves, I think I needed to do some double checking that the
clearml.conf file was in the proper location. It wasn’t where I thought I put it and I had to set all the proper environment variables in the container.