Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone. I’M Struggling To Setup Minio Storage. Below Is What I’M Adding In My Credentials And When I Try To Create A New Dataset Using Below Command; I Get Errors: Configs:

Hi everyone. I’m struggling to setup minio storage. Below is what I’m adding in my credentials and when I try to create a new dataset using below command; I get errors:

configs:

sdk {
    aws {
            s3 {
                # default, used for any bucket not specified below
                key: ""
                secret: ""
                region: ""
    
                credentials: [
                    {
                        # This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
                        host: "ostore.xyz.tech"
                        key: "my_access_key"
                        secret: "my_secret_key"
                        multipart: false
                        secure: false
                    }
                ]
            } 
    }
}

command: clearml-data create --storage " None " --project dummyproject --name testingds1
Note: I’ve already created a bucket in minio named clearml

Error:

Error: Insufficient permissions (write failed) for 

Am I missing anything?
cc: @<1523701435869433856:profile|SmugDolphin23> could you please help?

  
  
Posted 6 months ago
Votes Newest

Answers 33


@<1709740168430227456:profile|HomelyBluewhale47> , how did you set the output_uri ?

  
  
Posted 6 months ago

For below config and command it worked
command: clearml-data create --storage " None " --project dummyproject --name testingds2
image

  
  
Posted 6 months ago

Again throwing Insufficient permissions.

  
  
Posted 6 months ago

I passed only None

  
  
Posted 6 months ago

try None :port/bucket

  
  
Posted 6 months ago

Can you provide a full print of the terminal?

  
  
Posted 6 months ago

Try only like this ostore.xyz.tech:9000

  
  
Posted 6 months ago

I also tried passing the default port for minio for host in config
eg: host: “ostore.xyz.tech:9000”

But getting same error.

  
  
Posted 6 months ago

image

  
  
Posted 6 months ago

raise ValueError(“Insufficient permissions (delete failed) for {}“.format(base_url))
ValueError: Insufficient permissions (delete failed) for None

  
  
Posted 6 months ago

What is the combination of --storage and configuration that worked in the end?

  
  
Posted 6 months ago

With what host config were you trying the last attempts?

  
  
Posted 6 months ago

Hi @<1523701070390366208:profile|CostlyOstrich36>
After removing the port number from command given by you.
Re: your message
with the combination of None :port/bucket for --storage ?

  
  
Posted 6 months ago

Also tried changing files_server in config but didn’t work
image

  
  
Posted 6 months ago

still got same error

  
  
Posted 6 months ago

sure

  
  
Posted 6 months ago

Thanks a lot @<1523701070390366208:profile|CostlyOstrich36> emoji people:simple_smile 🙏

  
  
Posted 6 months ago

host: None

  
  
Posted 6 months ago

clearml-data create --storage "
" --project dummyproject --name testingds1
clearml-data - Dataset Management & Versioning CLI
Creating a new dataset:

Error: Insufficient permissions (write failed) for ostore.xyz.tech
  
  
Posted 6 months ago

Am I missing anything in config?

  
  
Posted 6 months ago

Try running the following script

from clearml import Task 
import time

task = Task.init(output_uri="
")

print("start sleep")
time.sleep(20)
print("end sleep")

Please add the logs

  
  
Posted 6 months ago

Yes, I tried with default port of minio (9000). didn’t work either.

  
  
Posted 6 months ago

it is picking http, it should https?

  
  
Posted 6 months ago

Also a paste of your config

  
  
Posted 6 months ago

Yeah, I missed that you defined the storage with --storage please try with the port as well there

  
  
Posted 6 months ago

yes

  
  
Posted 6 months ago

with the combination of None :port/bucket for --storage ?

  
  
Posted 6 months ago

Removed port and set secure as true.

  
  
Posted 6 months ago

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: “ None

  
  
Posted 6 months ago

FYI: I also changed the files_server in config

  
  
Posted 6 months ago
26K Views
33 Answers
6 months ago
6 months ago
Tags
aws