Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Guys, Can Anyone Help Me?? I Got This Message When I Tried To Ran Script Through Clearml-Task

Hello guys, Can anyone help me??
I got this message when I tried to ran script through clearml-task
botocore.exceptions.NoCredentialsError: Unable to locate credentials
log message wrote here.

ClearML results page: 

/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/boto3/compat.py:88: PythonDeprecationWarning: Boto3 will no longer support Python 3.6 starting May 30, 2022. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.7 or later. More information can be found here: 

  warnings.warn(warning, PythonDeprecationWarning)
Traceback (most recent call last):
  File "/root/.clearml/venvs-builds/3.6/code/clearml_aws_connection.py", line 22, in <module>
    main()
  File "/root/.clearml/venvs-builds/3.6/code/clearml_aws_connection.py", line 18, in main
    print(s3.list_objects(Bucket=bucket_name, Prefix=prefix))
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/client.py", line 508, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/client.py", line 895, in _make_api_call
    operation_model, request_dict, request_context
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/client.py", line 917, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/endpoint.py", line 116, in make_request
    return self._send_request(request_dict, operation_model)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/endpoint.py", line 195, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/endpoint.py", line 134, in create_request
    operation_name=operation_model.name,
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/hooks.py", line 412, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/hooks.py", line 256, in emit
    return self._emit(event_name, kwargs)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/hooks.py", line 239, in _emit
    response = handler(**kwargs)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/signers.py", line 103, in handler
    return self.sign(operation_name, request)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/signers.py", line 187, in sign
    auth.add_auth(request)
  File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/auth.py", line 407, in add_auth
    raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials
  • aws part setting in clearml.conf
  aws {
        s3 {
            # S3 credentials, used for read/write access by various SDK elements

            # default, used for any bucket not specified below
            key: ""
            secret: ""
            region: ""

            credentials: [
                # specifies key/secret credentials to use when handling s3 urls (read or write)
                {
                    bucket: "mybucket"
                    key: "ITISTHEKEY"
                    secret: "MYSECRETKEY"
                    # use_credentials_chain: true,
                },
                # {
                #     # This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
                #     host: "my-minio-host:9000"
                #     key: "12345678"
                #     secret: "12345678"
                #     multipart: false
                #     secure: false
                # }
            ]
        }
        boto3 {
            pool_connections: 512
            max_multipart_concurrency: 16
        }
    }
  • clearml-agent command:
clearml-agent daemon --queue default --foreground --cpu-only --docker
  • clearml-task command:clearml-task --project test --name remote_test --script test/clearml_aws_connection.py --queue default --packages boto3
  • clearml_aws_connection.py
"""
Test script for connection between clearml and aws s3
"""
import boto3

from clearml import Task


def main():
    s3 = boto3.client('s3')
    bucket_name = "mybucket"
    prefix = "ml/"
    task = Task.init(project_name="test", task_name="test")
    print(s3.list_objects(Bucket=bucket_name, Prefix=prefix))


if __name__ == "__main__":
    main() 
  
  
Posted one year ago
Votes Newest

Answers 3


Hi @<1557899668485050368:profile|FantasticSquid9> , I see you're using boto3 directly - ClearML does used boto3, but it will not set it up for you (just when it uses it directly).
You can use the StorageManager to access your configured object storage using ClearML (which in case you're using AWS, will actually use boto3 under the hood and send the commands for you)

  
  
Posted one year ago

I setted default key and secret in aws part in clearml.conf
but it didn't work either

  
  
Posted one year ago

Thx for fast replying. It was what I look for!

  
  
Posted one year ago
1K Views
3 Answers
one year ago
one year ago
Tags
aws
Similar posts