since this is an enterprise machine, and you don't have sudo/root, I am wondering if there is already other docker networks/composer setups running/in use

HugePelican43 did you try docker ps and see that the clearml-apiserver container is up? if so, can you docker exec -it cleaml-apiserver /bin/bash and try to do the curl http://localhost:8008 from inside it?

so I am not entirely sure what else you have changed Sir

I will get back to you tmr 😄 cheers

thank you so much for your effort

👍

"NetworkSettings": {
"Bridge": "",
"SandboxID": "xx",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"8008/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8008"
}
],
"8080/tcp": null,
"8081/tcp": null
},
"SandboxKey": "/var/run/docker/netns/xx",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"mlops_backend": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"apiserver",
"f86e5d0835f4"
],
"NetworkID": "xx",
"EndpointID": "xx",
"Gateway": "172.25.0.1",
"IPAddress": "172.25.0.6",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "xx",
"DriverOpts": null
},
"mlops_frontend": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"apiserver",
"f86e5d0835f4"
],
"NetworkID": "xx",
"EndpointID": "xx",
"Gateway": "172.26.0.1",
"IPAddress": "172.26.0.3",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "xxx",
"DriverOpts": null
}
}
}
}
]

I also run netstat inside the container:
netstat -an | grep LISTEN | grep 8008
tcp 0 0 0.0.0.0:8008 0.0.0.0:* LISTEN

I believe host address of apiserver is not set to 0.0.0.0

curl inside clearml-apiserver works just fine 😄

here is my network return after docker inspect

What am I supposed to set CLEARML_HOST_IP value?

SuccessfulKoala55

I assign 0.0.0.0

curl inside clearml-apiserver works just fine 

And from outside the docker, on the machine running the docker, it fails? That's very strange...

I think because clearml-apiserver is binded to 127.0.0.1 inside docker container instead of 0.0.0.0

yeah from the host curl return curl: (56) Recv failure: Connection reset by peer.

but the code binds it to 0.0.0.0 by default...

You mean for the agent-services?

SuccessfulKoala55

BTW, from the netstat output you sent it seems like the apiserver is binded correctly inside the docker

Seems to be some system/docker configuration issue - maybe related to you now having admin permissions?

Actually, it uses the apiserver.listen.ip configuration value (which is 0.0.0.0 by default)

You can look at the entrypoint inside the docker (it's called wrapper.sh )

😢 ah it's strange. I tried all the solutions I came up with

cannot fix it

do you think it happened because of container firewall?

Possibly...

oh it's not a problem.. if you can fling up the logs of ES after startup that's probably the next step.. along with a 'docker network list' output 👍

humhh I will ask the server admin