The code is run from another machine where clearml.conf configured to connect to ClearML server, no other configurations are provided
How can you have a certificate error if you're using S3? I'm sure their certificate is OK...
A bit overwhelmed by configuration, since it has an agent, a server and bunch of configuration files, easy to mess up
I meant the code where you upload an artifact, sorry
` s3 {
# S3 credentials, used for read/write access by various SDK elements
# default, used for any bucket not specified below
key: "mykey"
secret: "mysecret"
region: " ` ` "
credentials: [
{
bucket: "mybucket"
key: "mykey"
secret: "mysecret"
region: " ` ` "
}, `
SmugDolphin23 Got it. Now I am a bit confused about region parameter in s3 section. Amazon docs say that region could be a regular URL with protocol like https://etc.etc which my endpoint actually is. I plugged it in s3 section in clearml.conf. Should it stay that way?
@<1523701087100473344:profile|SuccessfulKoala55> Fixed it by setting env var with path to certificates. I was sure that wouldn't help since I can curl and python get request to my endpoint from shell just fine. Now it says I am missing security headers, seems it's something on my side. Will try to fix this
@<1523701435869433856:profile|SmugDolphin23> @<1523701087100473344:profile|SuccessfulKoala55>
2023-02-03 20:38:14,515 - clearml.metrics - WARNING - Failed uploading to <my-endpoint> (HTTPSConnectionPool(host=' e ndpoint', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)'))))
2023-02-03 20:38:14,517 - clearml.metrics - ERROR - Not uploading 1/2 events because the data upload failed
@<1526734383564722176:profile|BoredBat47> Yeah. This is an example:
s3 {
key: "mykey"
secret: "mysecret"
region: "us-east-1"
credentials: [
{
bucket: "
"
key: "mykey"
secret: "mysecret"
region: "us-east-1"
},
]
}
# some other config
default_output_uri: "
"
@<1526734383564722176:profile|BoredBat47> the bucket name in your case should just be somebucket
(and should not start with s3://
)
OK. Bt the way, you can find the region from the AWS dashabord
@<1526734383564722176:profile|BoredBat47> How would you connect with boto3
? ClearML uses boto3
as well, what it basically does is getting the key/secret/region from the conf file. After that it opens a Session
with the credentials. Have you tried deleting the region altogether from the conf file?
@<1523701304709353472:profile|OddShrimp85> I fixed my SSL error by putting REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
in .bashrc
file
May I know where to set the cert to in env variable?
` from random import random
from clearml import Task, TaskTypes
args = {}
task: Task = Task.init(
project_name="My Proj",
task_name='Sample task',
task_type=TaskTypes.inference,
auto_connect_frameworks=False
)
task.connect(args)
task.execute_remotely(queue_name="default")
value = random()
task.get_logger().report_single_value(name="sample_value", value=value)
with open("some_artifact.txt", "w") as f:
f.write(f"Some random value: {value}\n")
task.upload_artifact(name="test_artifact", artifact_object="some_artifact.txt") `
SmugDolphin23 I added a region, run experiment again. Didn't work
Could you try adding region
under credentials
as well?
Oh, it's configured o agent machine, got you
You need to specify it. Or you could specify this in your config: https://github.com/allegroai/clearml/blob/54c601eea2f9981bb8e360a8203bc36696a55cfd/clearml/config/default/sdk.conf#L164
Yeah, that's always the case with complex systems 😕
it's the same file you added your s3 creds to
I think that will work, but I'm not sure actually. I know for sure that something like us-east-2
is supported
The only expection is the models if I'm not mistaken, which are stored locally by default.
@<1526734383564722176:profile|BoredBat47> Just to check if u need to do update-ca-certificates or equivalent?
And I believe that by default we send artifacts to the clearml server if not specified
@<1523701435869433856:profile|SmugDolphin23> I didn't use a region at first and that was not working. Now I use a region and it still doesn't work.
From the boto3 inside a Python I could create a session where I specify ak and sk, and create a client from the session where I pass service_name and endpoint_url. It works just fine