
Reputation
Badges 1
104 × Eureka!Try to run docker ps
and check if all of your clearml containers up and running (should be 8 total)
~/.local/bin/clearml-agent daemon --foreground
@<1523701070390366208:profile|CostlyOstrich36> Yes, I know. Above I posted a link where there's a solution. DB request to elastic to change those URLs. My question is: where to send this DB request? What endpoint? Request provided in FAQ in incomplete. It lacks URL where to send the request to.
curl --header "Content-Type: application/json" \
--request POST \
--data '{
"script": {
"source": "ctx._source.url = ctx._source.url.replace('
.<OLD_ADDRESS>', '
...
CostlyOstrich36 Yep, it seems it was the case. I did not provide credentials for API in docker compose. I did that but now agent-services just keeps restarting. I looked into containers logs and it seems to be a proxy error. Why this container is trying to connect somewhere?
@<1722061389024989184:profile|ResponsiveKoala38> Got following error. Sorry for a screenshot, I can't copy text from web terminal I am using right now.
You missed a ":" in your URL after "https"
SmugDolphin23 Sorry to bother again, output_uri should be a URI to S3 endpoint or clear ml fileserver? If it's not provided artifacts are stored locally, right?
@<1722061389024989184:profile|ResponsiveKoala38> Thanks a lot for the help. Keep up the good work!
@<1523701087100473344:profile|SuccessfulKoala55> Fixed it by setting env var with path to certificates. I was sure that wouldn't help since I can curl and python get request to my endpoint from shell just fine. Now it says I am missing security headers, seems it's something on my side. Will try to fix this
@<1523701070390366208:profile|CostlyOstrich36> My question basically is what host do I need to send POST request to change paths? curl to localhost:9200 (where elastic is running) is not working
I figured it was something with syntax but I thought single ticks inside double quotes should work without escaping 😀
clearml 1.9.0
clearml-agent 1.5.1
NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
@<1523701304709353472:profile|OddShrimp85> I fixed my SSL error by putting REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
in .bashrc
file
curl -XPOST -H 'Content-Type: application/json' 'localhost:9200/events-training_debug_image-*/_update_by_query?conflicts=proceed' -d'{
"script": {
"source": "ctx._source.url = ctx._source.url.replace('http://<MY_OLD_ADDRESS>', '
.<NEW_ADDRESS>')",
"lang": "painless"
},
"query": {"prefix": {"url": {"value": "http://<MY_OLD_ADDRESS>"}}}
}'
@<1722061389024989184:profile|ResponsiveKoala38> Sure, I'll get back to you as it finishes
@<1722061389024989184:profile|ResponsiveKoala38> My bad, I was indeed using double quotes. I fixed it and now script is working. I'll wait for results
@<1722061389024989184:profile|ResponsiveKoala38> Now I can see the images where previously it was placeholders with text "Unable to upload the images"
@<1523701070390366208:profile|CostlyOstrich36> Old debug samples. My URL for files server has changed, and old debug sampled are not shown.
SmugDolphin23 Thank you very much!
That's clearml.conf for ClearML end users right?
@<1722061389024989184:profile|ResponsiveKoala38>
@<1722061389024989184:profile|ResponsiveKoala38> Thank a lot! I am gonna upgrade ClearML using this link: None
@<1523701435869433856:profile|SmugDolphin23> @<1523701087100473344:profile|SuccessfulKoala55>
2023-02-03 20:38:14,515 - clearml.metrics - WARNING - Failed uploading to <my-endpoint> (HTTPSConnectionPool(host=' e ndpoint', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)'))))
2023-02-03 20:38:14,517 - clearml.metrics - ERROR - Not uploa...
Should I remove "case-sensitive" option from a query?
Thanks a lot. I see that ClearML apiserver is up for 7 months, could it be that it runs on a version that was recent 7 month ago?
So, right now I have old deployment. It's working good, it's not corrupted. Service versions I shared above (output of docker ps). My goal is to move everything to another machine. Yes, I want to have a new deployment with all previous data. Basically, it's backup and restore task. The problem was that old docker compose file doesn't work as is. Maybe because when I run it on a new machine clearml:1 is pulling the latest version and elastic version is set to one that is no longer supported.