Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I’M Trying To Get A Copy Of A Model Through Clearml Which Is Stored In S3:

I’m trying to get a copy of a model through ClearML which is stored in S3:
` from clearml import Model

models = Model.query_models(project_name="example_project", tags=["stage"], only_published=True)
model = models[0]
file = model.get_local_copy(raise_on_error=True) When the final line executes I get and error something like: 2022-05-20 14:12:59,818 - clearml.storage - ERROR - Could not download < > , err: Unable to locate credentials I’ve verified that that object exists in S3. My clearml.conf is set to use use_credentials_chain: True ` which seems to work to upload this model to that location just fine. Not sure why I can’t download it through the clearml sdk though …

  
  
Posted 2 years ago
Votes Newest

Answers 15


I think I can set AWS_PROFILE . I have theoretically placed the clearml.conf in the correct place in the container: $HOME/clearml.conf . But I get an error that’s actually not very helpful, I still think the issue is getting boto3 credentials but I’m not sure I can prove it at this point. I posted the error in the community channel https://clearml.slack.com/archives/CTK20V944/p1653507863528969

  
  
Posted 2 years ago

BeefyHippopotamus73 this error seems like it is coming from boto3, are you sure the credentials are properly configured and that you have read permission ?

  
  
Posted 2 years ago

no, I’m not

  
  
Posted 2 years ago

Don’t know why I didn’t think of it earlier, but I set my env variable AWS_PROFILE=<foo-profile> and it copied it successfully.

  
  
Posted 2 years ago

SuccessfulKoala55 and AgitatedDove14 So setting my environment variable worked for my local runs, but it doesn’t work when running it from a container (in AWS ECS, for example)

  
  
Posted 2 years ago

Hi BeefyHippopotamus73
did you managed to get rid of your issue ?

  
  
Posted 2 years ago

Can you perhaps use the AWS_PROFILE environment variable to specify the profile?

  
  
Posted 2 years ago

so a little more clarity. I get nearly this same error when downloading the same object from boto3 if I try to get at it from an s3 resource like so:
` bad_s3_resource = boto3.resource("s3")
obj = bad_s3_resource.Object(bucket, key)
obj.get()["Body"]

botocore.exceptions.NoCredentialsError: Unable to locate credentials Only when I set the boto3 session with the appropriate profile does it work: sess = boto3.session.Session(profile_name="foo-profile")
s3_resource = sess.resource("s3")
obj = s3_resource.Object(bucket, key)
obj.get()["Body"]

<botocore.response.StreamingBody at 0x1463120d0> `
So this makes me wonder if there is a similar mechanism that’s giving me this error.

  
  
Posted 2 years ago

BeefyHippopotamus73 are you running this using a ClearML Agent?

  
  
Posted 2 years ago

yes I did finally resolve this. If my memory serves, I think I needed to do some double checking that the clearml.conf file was in the proper location. It wasn’t where I thought I put it and I had to set all the proper environment variables in the container.

  
  
Posted 2 years ago

well, I can get this exact object from my machine, from the same script through boto3, so I’m not sure where the disconnect is AgitatedDove14 SuccessfulKoala55

  
  
Posted 2 years ago

I still think the issue is getting boto3 credentials

It might be the case
Are you using clearml-agent or are you running it manually ?

  
  
Posted 2 years ago

BeefyHippopotamus73 are you saying that on a remote machine you cannot set AWS_PROFILE ? or is it the clearml.conf is missing ? (not sure I follow how / who spins the remote machine)

  
  
Posted 2 years ago

BeefyHippopotamus73 indeed it seems like this is boto3 announcing it can't locate credentials on your machine

  
  
Posted 2 years ago

I think the issue is we do not pass a profile parameter to boto

  
  
Posted 2 years ago