AgitatedDove14 , I'll have a look at it and let you know. According to you the VPN shouldn't be a problem right?
Which would mean the error is because of a company firewall/self-signed certificate.
The easiest solution,Disable SSL certificate check for ClearML.
Create the ~/clearml.conf manually:
` #disable SSL certificate check
api.verify_certificate: False
copy paste the credentials section from the UI
it should look something like:
api {
# web_server on port 8080
web_server: " "
# Notice: 'api_server' is the api server (default port 8008), not the web server.
api_server: " ` ` "
# file server on port 8081
files_server: " ` ` "
# Credentials are generated using the webapp, ` `
credentials {"access_key": "EGRTCO8JMSIGI6S39GTP43NFWXDQOW", "secret_key": "x!XTov_G-#vspE*Y(h$Anm&DIc5Ou-F)jsl$PdOyj5wG1&E!Z8"}
} `
Seems lime someone sitting in the middle and reroutes the request (maybe both https and port) ?!
I tried it about 2-3 months ago with trains-init (same use-case as this one) and it failed that time too.
Could it be the credentials are actually incorrect?
Highly unlikely, like I said, I generated a new set of credentials from the Web-UI and it worked perfectly fine for an Azure VM (not under the VPN).
Yes, copied and pasted the configuration file correctly, points to the right server (running on Azure). I created a clearml.conf file on another Azure VM (not under the VPN) and there it worked fine. The 'on-premise' server fails to connect to the ClearML server because of the VPN I think
Ran it, but it gave no output (but it does say ran successfully)
Verifying credentials ... Error: could not verify credentials: key=xxxxxx secret=xxxxx Enter user access key:
Could it be the credentials are actually incorrect? because it seems like you can access the server? (I assume you were able to browse to it and generate credentials. right?)
The 'on-premise' server fails to connect to the ClearML server because of the VPN I think
I think you are correct.
You can quickly test it, try ti run curl
http://local-server:8008 see if that works
2021-02-08 18:49:25,036 - clearml - WARNING - InsecureRequestWarning: Certificate verification is disabled! Adding certificate verification is strongly advised. See:
Retrying (Retry(total=239, connect=240, read=239, redirect=240, status=240)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='X.Y.Z.W', port=8015): Read timed out. (read timeout=3.0)",)': /auth.login
Got this, times out after 5 tries
According to you the VPN shouldn't be a problem right?
Correct as long as all parties are on the same VPN it should work, all the connections are always http so basically trivial communication
It is http btw, i don't know why it logged https://
This is odd could it be it automatically forwards to https ?
I would try the certificate check thing first
` * Trying X.Y.Z.W:8080...
- TCP_NODELAY set
- Connected to X.Y.Z.W (X.Y.Z.W) port 8008 (#0)
GET /debug.ping HTTP/1.1
Host: X.Y.Z.W:8080
User-Agent: curl/7.68.0
Accept: /
- Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Location:
< Connection: close
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Content-Security-Policy: frame-ancestors
< - Closing connection 0 `
What if I just copy a clearml.conf file and edit out the tokens? Could that work?
It is http btw, i don't know why it logged https://
This is odd could it be it automatically forwards to https
I have to check with my IT Team if there's any forwarding
Yeah I noticed that too! Ports are configured properly in the conf file though
And is it talking to the correct server ?