Bump, still waiting, closing in on a month since we are unable to deploy. We have team of 10+ people

host: "my-minio-host:9000"

The port should be whatever port that is used by your S3 solution

we might as well have "s5" there but it is needed there

Hi @<1590514584836378624:profile|AmiableSeaturtle81> , the hotfix should right around the corner 🙂

This is the link generated

Adding bucket in clearml.conf causes the same error: clearml.storage - ERROR - Failed uploading: Could not connect to the endpoint URL: " None "

Hey, i see that 1.14.2 dropped
I tried it but the issue is still there, maybe the hotfix is in next patch?

Here is the setup so you can reproduce it (we dont have region field)
clearml.conf:
s3 {
use_credentials_chain: false
credentials: [
{
host: " s3.somehost.com "
key: "XXXXXXXXXXXXXXXXXXXX"
secret: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
bucket: "rnd-dev"
},
]
}

test.py

task: clearml.Task = clearml.Task.init(
project_name="project",
task_name="task",
output_uri=" None ",
)

2024-02-08 11:23:52,150 - clearml.storage - ERROR - Failed creating storage object None Reason: Missing key and secret for S3 storage access ( None )

Hi @<1590514584836378624:profile|AmiableSeaturtle81> self hosted S3 service must specify the protocol (http/https) and port, even for the default ones (80 / 443).

I dont have a region. I guess I will wait till tomarrow then?

This is an actual AWS S3 bucket?

Or whatever port you use

As I wrote, you need to remove the s3 from the start of the host section..

maybe someone on your end can try to parse such a config and see if they also have the same problem

Hi @<1590514584836378624:profile|AmiableSeaturtle81> ! To help us debug this: are you able to simply use the boto3 python package to interact with your cluster?
If so, how does that code look like? This would give us some insight on how the config should actually look like or what changes need to be made.

@<1590514584836378624:profile|AmiableSeaturtle81> ok, I think that your credentials from clearml.conf are actually working now. let's not change them.
Now let's try this simple code:

from clearml import Task
import numpy as np


if __name__ == "__main__":
    task = Task.init(task_name="test4", project_name="test4", output_uri="

")
    image = np.random.randint(0, 256, size=(500, 1000, 3), dtype=np.uint8)
    task.upload_artifact("image", image)

You should change the task_name and project_name from test just in case some object has been created previously

• Here is how client side clearml.conf looks like together with the script im using to create the tasks. Uploads seems to work and is fixed thanks to you guys 🙌
  
  
  

It looks like im moving forward

Setting url in clearml.conf without "s3" as suggested works (But I dont add port ther, not sure if it breaks something, we dont have a port)
host: " our-host.com "

Then in test_task.py
task: clearml.Task = clearml.Task.init(
project_name="project",
task_name="task",
output_uri=" None ",
)

I think connection is created
What im getting now is bucket error, i suppose I have to specify it somewhere?

Specifying it like this, gets me different error:

Exception has occurred: ValueError

• Insufficient permissions (delete failed) for None
  botocore.exceptions.ClientError: An error occurred (IllegalLocationConstraintException) when calling the DeleteObject operation: The me-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.

During handling of the above exception, another exception occurred:

File "/home/ma/src/clearml-server/task_test.py", line 10, in <module>
task: clearml.Task = clearml.Task.init(
ValueError: Insufficient permissions (delete failed) for None

This is unrelated to your routers. There are two things at play here. The configuration of WHERE the data will go - output_uri and the other is the clearml.conf that you need to setup with credentials. I am telling you, you are setting it wrong. Please follow documentation.