Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Want To Run

I want to run clearml sdk to create task (without running it) and upload artifact in a container.
I have code to pull clearml API key from Azure Keyvault and auth with clearml using

    clearml.Task.set_credentials(
        api_host=creds['api_url'],
        web_host=creds['web_url'],
        key=creds['access_key'],
        secret=creds['secret_key']
    )

But how do I set the auth for our blob storage account ? I would like to not use clearml.conf

  
  
Posted 12 days ago
Votes Newest

Answers 6


Awesome. I will try that !
And also found this based on your suggestion that clearml use azure sdk underneath: None
Just not sure under which conditions from_config is actually called ...

  
  
Posted 10 days ago

Yes just use AZURE_STORAGE_ACCOUNT, it is used when no specific config exist.
Perfect reference BTW :)

  
  
Posted 9 days ago

Yes the easiest is os.environ call before the import
Regarding azure blob

General azure env vars should work because it configure the underlying azure sdk, but I would double check

Generally speaking
Generic Override Format
ClearML allows you to override any config entry using this format:

bash
CLEARML__<section>__<key>=<value>
Double underscores __ separate the hierarchy levels.

All keys and values are treated as strings.

This works for nested entries in clearml.conf.

  
  
Posted 11 days ago

So this can be translated to

CLEARML__SDK__AZURE__STORAGE__CONTAINERS__0__ACOUNT_NAME=abcd
  
  
Posted 11 days ago

Hi @<1576381444509405184:profile|ManiacalLizard2>
You can also use env vars, it might be easier, I'm assuming this is kind of CI/CD process
'''
export CLEARML_API_ACCESS_KEY="your-public-key"
export CLEARML_API_SECRET_KEY="your-private-secret"
export CLEARML_API_HOST=" https://api.clear.ml "
export CLEARML_WEB_HOST=" https://app.clear.ml "
export CLEARML_FILES_HOST=" https://files.clear.ml "

'''
https://clear.ml/docs/latest/docs/configs/env_vars/
Re aws s3 credentials
export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_DEFAULT_REGION="us-east-1"

Optional for S3-compatible storage:

export CLEARML_AWS_S3_ENDPOINT=" https://minio.mycompany.com "

  
  
Posted 12 days ago

@<1523701205467926528:profile|AgitatedDove14>
What is the env var name for Azure Blob storage ? That the one we use for our Artifiact.

Also, is there function call rather than env var ?
It would be simplier in our case to call a function to set credential for clearml rather than fetch secret and set env var prior to running the python code.

If there is only the option of using env var, I am thinking fetchcing secrets and set env var from python, eg: os.environ["MY_VARIABLE"] = "hello" prior to import clearml That should work too right ?

  
  
Posted 11 days ago
214 Views
6 Answers
12 days ago
8 days ago
Tags
Similar posts