Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All! I'M Trying To Create Dataset With Output_Uri Pointing To Google Storage And Got Weird Error:

Hi all! I'm trying to create Dataset with output_uri pointing to google storage and got weird error:

google.api_core.exceptions.NotFound: 404 GET 
<bucket_name>/<folder>/iam/testPermissions?permissions=storage.objects.get&permissions=storage.objects.update&prettyPrint=false: Not Found

and the code is:

dataset = Dataset.create(dataset_name="dataset_name", dataset_project="project_name",
                         output_uri=f'gs://<bucket_name>/<folder>')

Do you have any idea what am I doing wrong?
I've managed to debug to the point that if I remove folder and only keep output_uri=f'gs://<bucket_name> then my code passes just fine. However, I need to upload files to specific directory. Of course specific values are instead of <bucket_name> and <folder> Thanks in advance!

  
  
Posted one year ago
Votes Newest

Answers 11


Can you try to upgrade to the latest one? Try v1.10.1

  
  
Posted one year ago

Hi! Could anyone please take a look into my issue? This is still relevant for me

  
  
Posted one year ago

@<1544853721739956224:profile|QuizzicalFox36> what version of the ClearML SDK are you using?

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> 1.9.0

  
  
Posted one year ago

Hi @<1544853721739956224:profile|QuizzicalFox36> , from the error you're getting it looks like a permissions issue with the credentials. Check if your credentials have read/write/delete permissions

  
  
Posted one year ago

Yes, they are. I was using same service account from same code base with python sdk for download, delete, upload and so on.

  
  
Posted one year ago

Oh I misunderstood. It fails when you have the folder in the output_uri but runs all fine when it's without 🙂

  
  
Posted one year ago

@<1523701070390366208:profile|CostlyOstrich36> good to know. I hope you or someone else will be able to take a look at some point into my issue. Thanks!

  
  
Posted one year ago

Also, I've checked do the storageManager upload works for the same bucket and can confirm that it does, so the issue shouldn't be with permissions

  
  
Posted one year ago

You're totally right, if you managed to upload to a bucket then folder failure should be unrelated to permissions

  
  
Posted one year ago

yes, that's right. So do you have any ideas what might be wrong?

  
  
Posted one year ago