Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi There, I Have A Bit Of A Problem With Aws Secrets: I Pass Keys As Env Var To Clearml-Agents To Retrieve Data From A Bucket In Us-East-1 But I Use A Bucket To Store Task Artifacts In A Bucket In Eu-Central-1. So When I Pass Aws Keys As Env Vars, The Tas

Hi there, I have a bit of a problem with AWS secrets: I pass keys as env var to clearml-agents to retrieve data from a bucket in us-east-1 but I use a bucket to store task artifacts in a bucket in eu-central-1. So when I pass AWS keys as env vars, the tasks executed by the agent try to log artifacts on the us-east-1 bucket and if fails. Is there any way to set in the code the region to use for storing artifacts?

  
  
Posted 2 years ago
Votes Newest

Answers 14


But clearml does read from env vars as well right? It’s not just delegating resolution to the aws cli, so it should be possible to specify the region to use for the logger, right?

  
  
Posted 2 years ago

Traceback (most recent call last): File "devops/train.py", line 73, in <module> train(parse_args) File "devops/train.py", line 37, in train train_task.get_logger().set_default_upload_destination(args.artifacts + '/clearml_debug_images/') File "/home/machine/miniconda3/envs/py36/lib/python3.6/site-packages/clearml/logger.py", line 1038, in set_default_upload_destination uri = storage.verify_upload(folder_uri=uri) File "/home/machine/miniconda3/envs/py36/lib/python3.6/site-packages/clearml/storage/helper.py", line 494, in verify_upload log_on_error=log_on_error, File "/home/machine/miniconda3/envs/py36/lib/python3.6/site-packages/clearml/storage/helper.py", line 1424, in _test_bucket_config raise StorageError(msg) clearml.storage.helper.StorageError: Failed testing access to bucket clearml-artefacts: incorrect region specified for bucket clearml-artefacts (detected region eu-central-1)

  
  
Posted 2 years ago

JitteryCoyote63 Hello πŸ™‚

So That I understand correctly you have a task where you pull data from us-east-1 and then want to upload the data on eu-central-1 but it's failing due to the region change, correct? Can you please give an example of the failure in console?

  
  
Posted 2 years ago

Do you explicitly specify the region?
Also, please try specifying the buckets directly in your ~/clearml.conf

The relevant section is sdk.aws.s3.credentials

It would look something like this:

{ host: "<BUCKET_A_HOST>" key: "<KEY>" secret: "<SECRET>" region: "<REGION>" }

  
  
Posted 2 years ago

JitteryCoyote63 , can you try upgrading to the latest versions? Is there any reason why you stayed on older versions?

  
  
Posted 2 years ago

Yes that’s correct - the weird thing is that the error shows the right detected region

  
  
Posted 2 years ago

I am using clearml_agent v1.0.0 and clearml 0.17.5 btw

  
  
Posted 2 years ago

thanks Jake, good to know!

  
  
Posted 2 years ago

Cool πŸ™‚ - as for your question, clearml will use the env vars as the global settings, but if you set the configuration for a specific bucket, it should use that for that bucket

  
  
Posted 2 years ago

Yes, I stayed with an older version for a compatibility reason I cannot remember now πŸ˜„ - just tested with 1.1.2 and it’s the same
I tried specifying the bucket directly in my clearml.conf, same problem. I guess clearml just reads from the env vars first

  
  
Posted 2 years ago

JitteryCoyote63 using multiple regions while using env vars is something that's not even supported in the AWS CLI... πŸ™‚
If you want to mix regions, you'll have to use the clearml.conf configuration (as CostlyOstrich36 suggested)

  
  
Posted 2 years ago

Sorry both of you, my problem was actually lying somewhere else (both buckets are in the same region) - thanks for you time!

  
  
Posted 2 years ago

SuccessfulKoala55 I want to avoid writing creds in plain text in the config file

  
  
Posted 2 years ago

JitteryCoyote63 , well I'm happy to hear that everything worked out in the end πŸ™‚

  
  
Posted 2 years ago
657 Views
14 Answers
2 years ago
one year ago
Tags