yes I did finally resolve this. If my memory serves, I think I needed to do some double checking that the clearml.conf file was in the proper location. It wasn’t where I thought I put it and I had to set all the proper environment variables in the container.
I still think the issue is getting boto3 credentials
It might be the case
Are you using clearml-agent or are you running it manually ?
I think the issue is we do not pass a profile parameter to boto
BeefyHippopotamus73 indeed it seems like this is boto3 announcing it can't locate credentials on your machine
BeefyHippopotamus73 this error seems like it is coming from boto3, are you sure the credentials are properly configured and that you have read permission ?
I think I can set AWS_PROFILE . I have theoretically placed the clearml.conf in the correct place in the container: $HOME/clearml.conf . But I get an error that’s actually not very helpful, I still think the issue is getting boto3 credentials but I’m not sure I can prove it at this point. I posted the error in the community channel https://clearml.slack.com/archives/CTK20V944/p1653507863528969
SuccessfulKoala55 and AgitatedDove14 So setting my environment variable worked for my local runs, but it doesn’t work when running it from a container (in AWS ECS, for example)
well, I can get this exact object from my machine, from the same script through boto3, so I’m not sure where the disconnect is AgitatedDove14 SuccessfulKoala55
Can you perhaps use the AWS_PROFILE environment variable to specify the profile?
so a little more clarity. I get nearly this same error when downloading the same object from boto3 if I try to get at it from an s3 resource like so:
` bad_s3_resource = boto3.resource("s3")
obj = bad_s3_resource.Object(bucket, key)
obj.get()["Body"]
botocore.exceptions.NoCredentialsError: Unable to locate credentials
Only when I set the boto3 session with the appropriate profile does it work:sess = boto3.session.Session(profile_name="foo-profile")
s3_resource = sess.resource("s3")
obj = s3_resource.Object(bucket, key)
obj.get()["Body"]
<botocore.response.StreamingBody at 0x1463120d0> `
So this makes me wonder if there is a similar mechanism that’s giving me this error.
BeefyHippopotamus73 are you saying that on a remote machine you cannot set AWS_PROFILE ? or is it the clearml.conf is missing ? (not sure I follow how / who spins the remote machine)
Don’t know why I didn’t think of it earlier, but I set my env variable AWS_PROFILE=<foo-profile> and it copied it successfully.
Hi BeefyHippopotamus73
did you managed to get rid of your issue ?
BeefyHippopotamus73 are you running this using a ClearML Agent?