Awesome. I will try that !
And also found this based on your suggestion that clearml use azure sdk underneath: None
Just not sure under which conditions from_config
is actually called ...
Yes just use AZURE_STORAGE_ACCOUNT, it is used when no specific config exist.
Perfect reference BTW :)
Yes the easiest is os.environ call before the import
Regarding azure blob
General azure env vars should work because it configure the underlying azure sdk, but I would double check
Generally speaking
Generic Override Format
ClearML allows you to override any config entry using this format:
bash
CLEARML__<section>__<key>=<value>
Double underscores __ separate the hierarchy levels.
All keys and values are treated as strings.
This works for nested entries in clearml.conf.
So this can be translated to
CLEARML__SDK__AZURE__STORAGE__CONTAINERS__0__ACOUNT_NAME=abcd
Hi @<1576381444509405184:profile|ManiacalLizard2>
You can also use env vars, it might be easier, I'm assuming this is kind of CI/CD process
'''
export CLEARML_API_ACCESS_KEY="your-public-key"
export CLEARML_API_SECRET_KEY="your-private-secret"
export CLEARML_API_HOST=" https://api.clear.ml "
export CLEARML_WEB_HOST=" https://app.clear.ml "
export CLEARML_FILES_HOST=" https://files.clear.ml "
'''
https://clear.ml/docs/latest/docs/configs/env_vars/
Re aws s3 credentials
export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_DEFAULT_REGION="us-east-1"
Optional for S3-compatible storage:
export CLEARML_AWS_S3_ENDPOINT=" https://minio.mycompany.com "
@<1523701205467926528:profile|AgitatedDove14>
What is the env var name for Azure Blob storage ? That the one we use for our Artifiact.
Also, is there function call rather than env var ?
It would be simplier in our case to call a function to set credential for clearml rather than fetch secret and set env var prior to running the python code.
If there is only the option of using env var, I am thinking fetchcing secrets and set env var from python, eg: os.environ["MY_VARIABLE"] = "hello"
prior to import clearml
That should work too right ?