Hey @<1523701070390366208:profile|CostlyOstrich36> could you help me out? I’m having trouble figuring out why it’s not working. I’d really appreciate your help!
After copying the clearml.conf
file to /opt/clearml/config/
, the error has disappeared, but the files still aren't being reflected on S3.
using below code to upload the file.
task = Task.init(project_name="Test Project", task_name="Test Task", output_uri="
")
# task = Task.init(project_name="Test Project", task_name="Test Task")
task.set_parameter(name='disable_caching', value=True, description='Disable caching for this task')
task.upload_artifact(name="/content/sample_data/README.md", artifact_object={"key": "value"})
# artifact = task.artifacts.get("/content/sample_data/README.md")
# print(artifact.get_local_copy())
Here’s the relevant portion of my clearml.conf
file located at ~/clearml.conf
:
sdk {
aws {
s3 {
key: "AWS_ACCESS_KEY"
secret: "AWS_SECRET_KEY"
region: "us-east-1"
use_credentials_chain: true
credentials: [
{
bucket: "clearml-s3"
key: "AWS_ACCESS_KEY"
secret: "AWS_SECRET_KEY"
region: "us-east-1"
use_credentials_chain: true
}
]
}
boto3 {
pool_connections: 512
max_multipart_concurrency: 16
}
}
storage {
default_output_uri: " [None](s3://clearml-s3/) "
}
}
I'm using a t3a.large
instance and running the default Docker Compose file on Ubuntu.
Despite following the documentation, I’m having trouble getting it to work. If anyone has experience setting up S3 with ClearML or can point me in the right direction, I would really appreciate your help!
Thanks in advance!