@<1523701070390366208:profile|CostlyOstrich36> Old debug samples. My URL for files server has changed, and old debug sampled are not shown.

Oh, I see, it's there in UI

@<1523701087100473344:profile|SuccessfulKoala55> I provided following env vars:
CLEARML_HOST_IP: "<my_ip>"
CLEARML_WEB_HOST: " http://<my_ip>:8080 "
CLEARML_API_HOST: " http://<my_ip>:8008 "
CLEARML_FILES_HOST: " http://<my_ip>:8081 "
CLEARML_API_ACCESS_KEY: <my_access_key>
CLEARML_API_SECRET_KEY: <my_secret_key>
also I changed IP in entrypoint from apiserver:8008 to <my_ip>:8008

Yes, I run both commands from the same place — dedicated user on my worker m...

` s3 {
# S3 credentials, used for read/write access by various SDK elements

        # default, used for any bucket not specified below
        key: "mykey"
        secret: "mysecret"
        region: " ` ` "

        credentials: [

             {
                 bucket: "mybucket"
                 key: "mykey"
                 secret: "mysecret"
                 region: " ` ` "
              }, `

Docstring from inside the boto3 lib says:

:param endpoint_url: The complete URL to use for the constructed
    client. Normally, botocore will automatically construct the
    appropriate URL to use when communicating with a service.  You
    can specify a complete URL (including the "http/https" scheme)
    to override this behavior.  If this value is provided,
    then ``use_ssl`` is ignored.

I want ClearML to use my endpoint

session = boto3.Session(
    aws_access_key_id=self.access_key,
    aws_secret_access_key=self.secret_key)

Thanks for the reply! We have a custom S3 server, it has an URL — endpoint like https://<some-domain>.<sub-domain>. I've read in docs that when you provide credentials.host — port must be specified. @<1523701070390366208:profile|CostlyOstrich36>

@<1523701087100473344:profile|SuccessfulKoala55> Fixed it by setting env var with path to certificates. I was sure that wouldn't help since I can curl and python get request to my endpoint from shell just fine. Now it says I am missing security headers, seems it's something on my side. Will try to fix this

I figured it was something with syntax but I thought single ticks inside double quotes should work without escaping 😀

CostlyOstrich36 Any thoughts?

@<1722061389024989184:profile|ResponsiveKoala38> Sure, I'll get back to you as it finishes

Right, seems the lib was severely outdated

Also, previous problem was in incorrect proxy configuration on agent machine

Try to run docker ps and check if all of your clearml containers up and running (should be 8 total)

All log entries have "level": "INFO"

clearml-agent daemon --foreground

@<1523701304709353472:profile|OddShrimp85> I fixed my SSL error by putting REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt in .bashrc file

@<1523701070390366208:profile|CostlyOstrich36> Yes, I know. Above I posted a link where there's a solution. DB request to elastic to change those URLs. My question is: where to send this DB request? What endpoint? Request provided in FAQ in incomplete. It lacks URL where to send the request to.

curl --header "Content-Type: application/json" \
--request POST \
--data '{
    "script": {
        "source": "ctx._source.url = ctx._source.url.replace('

.<OLD_ADDRESS>', '

...

@<1523701087100473344:profile|SuccessfulKoala55> It's the URL I use when creating boto3 session from Python like this fro example

s3 = self.session.client(
    service_name='s3',
    endpoint_url=endpoint,
    verify=False
)

@<1523701087100473344:profile|SuccessfulKoala55> No port needed when accessing this URL from things like boto3 or s3-client CLI

@<1523701435869433856:profile|SmugDolphin23> Thanks a lot, that actually worked! It was very difficult to figure out you have to plug those exact values given you have https endpoint:

• Using s3 protocol instead of https together with bucket name in output URI
• Not providing a bucket name in credentials section where it is by default
• Providing default secure port for both host and output URI
• Disabling credentials chainI think a common use case for many people that they get S3 storage wi...

CostlyOstrich36
The error appears regardless of --foreground tag. This is not full stacktrace, I will provide it with the next message.
clearml 1.9.0
clearml-agent 1.5.1
Ubuntu1 8.04.6 LTS

@<1523701087100473344:profile|SuccessfulKoala55> So I have to provide a host for it to work and no other way around it?

curl -XPOST -H 'Content-Type: application/json' 'localhost:9200/events-training_debug_image-*/_update_by_query?conflicts=proceed' -d'{
    "script": {
        "source": "ctx._source.url = ctx._source.url.replace('http://<MY_OLD_ADDRESS>', '

.<NEW_ADDRESS>')",
        "lang": "painless"
    },
    "query": {"prefix": {"url": {"value": "http://<MY_OLD_ADDRESS>"}}}
}'

Console output of clearml-agent init with no clearml.conf:
...
ClearML Hosts configuration:
Web App: None
API: None
File Store: None

Verifying credentials ...
Error: could not verify credentials: key=ak secret=sk
...
Console output of clearml-agent daemon --foreground with clearml.conf created by clearml-init is missing. No output.
...

Thanks a lot. I see that ClearML apiserver is up for 7 months, could it be that it runs on a version that was recent 7 month ago?

@<1523701435869433856:profile|SmugDolphin23> @<1523701087100473344:profile|SuccessfulKoala55>
2023-02-03 20:38:14,515 - clearml.metrics - WARNING - Failed uploading to <my-endpoint> (HTTPSConnectionPool(host=' e ndpoint', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)'))))
2023-02-03 20:38:14,517 - clearml.metrics - ERROR - Not uploa...

@<1722061389024989184:profile|ResponsiveKoala38> Got following error. Sorry for a screenshot, I can't copy text from web terminal I am using right now.

it's pretty much the same