seems true
root@9f6a74ab9a27:~/clearml# curl
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method is not allowed for the requested URL.</p>
root@9f6a74ab9a27:~/clearml# curl
curl: (7) Failed to connect to localhost port 8081: Connection refused
root@9f6a74ab9a27:~/clearml#
I don't thing WEB_HOST is important, but what about FILE_HOST?
do I need to change it accordingly?
yeah, I tried the following
None
but haven't managed yet to make it work
I wonder if I just need to join 2 docker-compose files to run everything in one session
Actually that could also work
But for reference, when I said IP i meant the actual host network IP not the 127.0.0.1 (which is the same as localhost)
seems like an issue about 2 compose apps using different networks which are not accessible from each other
I wonder if I just need to join 2 docker-compose files to run everything in one session
you are right, for some reason it doesn't resolve inside a container
root@dd0252a8f93e:~/clearml# curl
curl: (7) Failed to connect to localhost port 8008: Connection refused
root@dd0252a8f93e:~/clearml# curl
curl: (7) Failed to connect to 127.0.0.1 port 8008: Connection refused
root@dd0252a8f93e:~/clearml#
same thing
clearml-serving-inference | Retrying (Retry(total=236, connect=236, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899dc4e8b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /auth.login
Try the following example.env :
CLEARML_SERVING_PORT=9090
CLEARML_WEB_HOST="http://<IP>:8080"
CLEARML_API_HOST="http://<IP>:8008"
CLEARML_FILES_HOST="http://<IP>:8081"
(I think the localhost is resolved to inside the container and not the host machine, hence the error)
I have to step away for a couple of hours
please let me know if you find something wrong
my clearml.conf
api {
web_server:
api_server:
files_server:
# test 3
credentials {
"access_key" = "91SFEX4BYUQ9YCZ9V6WP"
"secret_key" = "4WTXT7tAW3R6tnSi8hzSKNjgkmgUoyv22lYT2FIzIfLoeGERRO"
}
}
maybe I'm missing something with credentials?
oh, I see one error, let me check fast
Okay this seems correct...
Can you share both yaml files (server & serving) and env file?
curl
{"meta":{"id":"59bbb55b6ddc456092658ae588c9a436","trx":"59bbb55b6ddc456092658ae588c9a436","endpoint":{"name":"auth.login","requested_version":"2.18","actual_version":"1.0"},"result_code":401,"result_subcode":20,"result_msg":"Unauthorized (missing credentials)","error_stack":null,"error_data":{}},"data":{}}
What are you getting with:
curl http://<ip>:8008/auth.login
I tried that, it didn't work. I was confused by the separate port parameter:
CLEARML_SERVING_PORT: ${CLEARML_SERVING_PORT:-8080}
which is only one port related in docker-compose-triton.yml
Can I test /auth.login somehow independently? Using curl or any other way. Which address does it suppose to have and which creds should I use?
@<1523706266315132928:profile|DefiantHippopotamus88> seems like you are missing the ports 🙂
CLEARML_WEB_HOST="
"
CLEARML_API_HOST="
"
CLEARML_FILES_HOST="
"
do I need to change anything else?
and my ~/clearml.conf
api {
web_server:
api_server:
files_server:
# test 3
credentials {
"access_key" = "91SFEX4BYUQ9YCZ9V6WP"
"secret_key" = "4WTXT7tAW3R6tnSi8hzSKNjgkmgUoyv22lYT2FIzIfLoeGERRO"
}
}
my example.env
CLEARML_WEB_HOST="
"
CLEARML_API_HOST="
"
CLEARML_FILES_HOST="
"
CLEARML_API_ACCESS_KEY="91SFEX4BYUQ9YCZ9V6WP"
CLEARML_API_SECRET_KEY="4WTXT7tAW3R6tnSi8hzSKNjgkmgUoyv22lYT2FIzIfLoeGERRO"
CLEARML_SERVING_TASK_ID="450231049bba42f69c6507cb774f7dc6
I changed port here:
clearml-serving-inference:
image: allegroai/clearml-serving-inference:latest
container_name: clearml-serving-inference
restart: unless-stopped
ports:
- "9090:8080"
Hi @<1523706266315132928:profile|DefiantHippopotamus88>
The idea is that clearml-server acts as a control plane and can sit on a different machine, obviously you can run both on the same machine for testing. Specifically it looks like the clearml-sering is not configured correctly as the error points to issue with initial handshake/login between the triton containers and the clearml-server. How did you configure the clearml-serving docker compose?