Hello guys, Can anyone help me??
I got this message when I tried to ran script through clearml-task
botocore.exceptions.NoCredentialsError: Unable to locate credentials
log message wrote here.
ClearML results page:
/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/boto3/compat.py:88: PythonDeprecationWarning: Boto3 will no longer support Python 3.6 starting May 30, 2022. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.7 or later. More information can be found here:
warnings.warn(warning, PythonDeprecationWarning)
Traceback (most recent call last):
File "/root/.clearml/venvs-builds/3.6/code/clearml_aws_connection.py", line 22, in <module>
main()
File "/root/.clearml/venvs-builds/3.6/code/clearml_aws_connection.py", line 18, in main
print(s3.list_objects(Bucket=bucket_name, Prefix=prefix))
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/client.py", line 508, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/client.py", line 895, in _make_api_call
operation_model, request_dict, request_context
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/client.py", line 917, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/endpoint.py", line 116, in make_request
return self._send_request(request_dict, operation_model)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/endpoint.py", line 195, in _send_request
request = self.create_request(request_dict, operation_model)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/endpoint.py", line 134, in create_request
operation_name=operation_model.name,
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/hooks.py", line 412, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/hooks.py", line 256, in emit
return self._emit(event_name, kwargs)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/hooks.py", line 239, in _emit
response = handler(**kwargs)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/signers.py", line 103, in handler
return self.sign(operation_name, request)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/signers.py", line 187, in sign
auth.add_auth(request)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/botocore/auth.py", line 407, in add_auth
raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials
- aws part setting in clearml.conf
aws {
s3 {
# S3 credentials, used for read/write access by various SDK elements
# default, used for any bucket not specified below
key: ""
secret: ""
region: ""
credentials: [
# specifies key/secret credentials to use when handling s3 urls (read or write)
{
bucket: "mybucket"
key: "ITISTHEKEY"
secret: "MYSECRETKEY"
# use_credentials_chain: true,
},
# {
# # This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
# host: "my-minio-host:9000"
# key: "12345678"
# secret: "12345678"
# multipart: false
# secure: false
# }
]
}
boto3 {
pool_connections: 512
max_multipart_concurrency: 16
}
}
clearml-agent daemon --queue default --foreground --cpu-only --docker
- clearml-task command:
clearml-task --project test --name remote_test --script test/clearml_aws_connection.py --queue default --packages boto3
- clearml_aws_connection.py
"""
Test script for connection between clearml and aws s3
"""
import boto3
from clearml import Task
def main():
s3 = boto3.client('s3')
bucket_name = "mybucket"
prefix = "ml/"
task = Task.init(project_name="test", task_name="test")
print(s3.list_objects(Bucket=bucket_name, Prefix=prefix))
if __name__ == "__main__":
main()