Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Everyone, I Have An Autoscaler Configuration That Runs Different Ec2 Instances. I Want The Ec2 Worker Launched By The Autoscaler Could Handle A Bucket With Different Aws Keys. The Configuration I Am Passing Is As Follows:

Hey everyone,
I have an autoscaler configuration that runs different ec2 instances.
I want the EC2 worker launched by the autoscaler could handle a bucket with different AWS keys.
The configuration I am passing is as follows:
` aws {
s3 {
# S3 credentials, used for read/write access by various SDK elements

   	    # default, used for any bucket not specified below
        key: "***"
        secret: "***"
        region: "us-east-1"

        credentials: [
             specifies key/secret credentials to use when handling s3 urls (read or write)
             {
                 bucket: "bucket-name"
                 key: "***"
                 secret: "***"
            },
           	# {
           	#     # This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
           	#     host: "my-minio-host:9000"
           	#     key: "12345678"
           	#     secret: "12345678"
           	#     multipart: false
           	#     secure: false
           	# }
        ]
} `But it still looks like the boto3 on the worker doesn’t use the key I passed for my specific bucket.

I would appreciate any help.

  
  
Posted 2 years ago
Votes Newest

Answers 8


I just ran into this too recently. Are you passing these also in the extra_clearml_conf for the autoscaler?

  
  
Posted 2 years ago

AFAIK that's the only way right now (see my comment here - https://clearml.slack.com/archives/CTK20V944/p1657720159903739?thread_ts=1657699287.630779&cid=CTK20V944 )
Or then if you have the ClearML paid service, I believe there is a "vaults" service, right AgitatedDove14 ?

Yep UnevenDolphin73 :)

  
  
Posted 2 years ago

I didn’t pass it through the extra_clearml_conf because I don’t want it to be exposed inside my code

  
  
Posted 2 years ago

Visibility, same issue as I had 😁

  
  
Posted 2 years ago

So yep, vault it is ...

  
  
Posted 2 years ago

I see, thanks for the help!

  
  
Posted 2 years ago

GrumpySeaurchin29 you can pass s3 credential for the autoscaler, but all the tasks will have them. Are you saying two diff sets of credentials is the issue, or is it the visibility?

  
  
Posted 2 years ago

AFAIK that's the only way right now (see my comment here - https://clearml.slack.com/archives/CTK20V944/p1657720159903739?thread_ts=1657699287.630779&cid=CTK20V944 )
Or then if you have the ClearML paid service, I believe there is a "vaults" service, right AgitatedDove14 ?

  
  
Posted 2 years ago