Based on this : it feels like S3 is supported
Hi @<1742355077231808512:profile|DisturbedLizard6> , why would you like to upload from s3? You can simply register the s3 link. If you want to upload, the model path only supports local files
How are you using the function update_output_model
?
You are right. Based on this page S3 is supported. My file with this function:
from clearml import Task
task = Task.init(project_name='s3_upload_models', task_name='sklearn', output_uri=True)
task.update_output_model(auto_delete_file=False, name='v0.0.1', model_path='s3://<BUCKET_NAME>/MLOps/models/sklearn/sklearn.pkl')
task.close()
my clearml.conf
aws {
s3 {
# default, used for any bucket not specified below
key: ""
secret: ""
region: ""
credentials: [
{
# This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
host: "<s3host>:443"
key: "<ACCESS_KEY>"
secret: "<SECRET_KEY>"
multipart: false
secure: false
verify: false # OR "/path/to/ca/bundle.crt" OR "
" OR false to not verify
}
]
}
boto3 {
pool_connections: 512
max_multipart_concurrency: 16
multipart_threshold: 8388608 # 8MB
multipart_chunksize: 8388608 # 8MB
}
}