Perhaps your container is not part of the correct docker networks as the fileserver contains is?
@<1523701087100473344:profile|SuccessfulKoala55> right now it is configured with:
network_mode: service:apiserver
in docker-compose-win-10.yml
. I have also tried:
networks:
- frontend
- backend
But with this configuration, I was not even able to communicate with the apiserver.
Correction: I am able to authenticate with the api server with both configurations. However, with the first configuration, I can use the task cli to do for instance:
clearml-task --project examples --name remote_test --repo
--branch master --script /webinar-0620/keras_mnist.py --args batch_size=64 epochs=1 --queue default
However, this does not work with the second configuration (I get a connection refused error)
@<1523701087100473344:profile|SuccessfulKoala55> No, I am not able to upload a file with either configuration. However, the first configuration works better in that I can use the task CLI.
Just wanted to add that the above error api_server is misconfigured
seems to happen regardless of whether I modify the standard docker_compose.yml
@<1523701087100473344:profile|SuccessfulKoala55> I see the api_server is misconfigured
error in the logs for the clearml-fileserver
. Unfortunately I can't copy and paste the logs for policy reasons, but I'll paraphrase them below:
- config files loaded from
/opt/clearml/fileserver
- [ERROR] [werkzeug] (ip address) code 400, message bad request version
urllib3.connectionpool
Retrying after connection broken by NewConnectionError- Traceback: urllib3/connection.py: ConnectionRefusedError [Errno 111] Connection refused- During the handling of above exception, another error occured: MaxRetryError- During the handling of above exception: ConnectionError: HTTPConnectionPool(host='apiserver', port=8008): Max retries exceeded with url:
/auth.login
- During the handling of above exception:ValueError
: Connection Error: it seems api_server is misconfigured
Hi @<1722786133810614272:profile|LazyClams7> where exactly do you see the error? Can you send the complete logs?
@<1523701087100473344:profile|SuccessfulKoala55> I did as you suggested: from the fileserver container, I was able to authenticate with the API server and receive a token.
however, from my development container if I doclearml-data add --files Affairs.csv
I still get a connetion refused error.
I also tried using the ip address of the files_server
container, but that didn't seem to make a difference
Hi @<1722786133810614272:profile|LazyClams7> , if you see api_server is misconfigured
error in the logs for the clearml-fileserver
, this basically means the filesserver can't reach the apiserver (it needs to), this doesn't seem to be a client issue. The filessererver tried to reach the apiserver by default using the None endpoint, which should be available inside the docker backend
network - you can try to exec into the fileserver container and see if curl to that address works
Also, how are you configuring clearml.conf inside your container?