Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Folks,

Hi Folks,

  • I'm experimenting with a local deployment of ClearML server on my laptop. I'm developing on a docker container anyway, so I think it makes sense to add my dev container to the clearml docker-compose.yml so that my dev container can communicate with the ClearML server.
  • What is the best way to do this? - I tried:
services:
  devcontainer:
    build: Dockerfile
    ...
    network_mode: service:apiserver
    depends_on:
    - apiserver
    - redis
    - mongo
    - elasticsearch
    - fileserver

  apiserver:
...

With this approach I seem to be able to communicate with the apiserver. for instance, I can create a task with the python SDK. However, I don't seem to be able to communicate with the fileserver. if I do the dataset Hello World example I get connection refused when adding data to a dataset. If I inspect the logs for the fileserver I see an error similar to this one: None

  • I tried modifying the docker-compose.yml as follows:
services:
  devcontainer:
    build: Dockerfile
    ...
    networks: 
      - frontend
      - backend
...
  apiserver:
...

But with this configuration I can't even connect to the apiserver.

Thanks so much for your help

  
  
Posted one month ago
Votes Newest

Answers 12


Perhaps your container is not part of the correct docker networks as the fileserver contains is?

  
  
Posted 24 days ago

@<1523701087100473344:profile|SuccessfulKoala55> right now it is configured with:

network_mode: service:apiserver

in docker-compose-win-10.yml . I have also tried:

networks:
   - frontend
   - backend

But with this configuration, I was not even able to communicate with the apiserver.
Correction: I am able to authenticate with the api server with both configurations. However, with the first configuration, I can use the task cli to do for instance:

clearml-task --project examples --name remote_test --repo 
 --branch master --script /webinar-0620/keras_mnist.py --args batch_size=64 epochs=1 --queue default

However, this does not work with the second configuration (I get a connection refused error)

  
  
Posted 24 days ago

I used clearml-init to configure the clearml.conf . This used the following:

  • api_server : None
  • web_server: None
  • files_server : NoneJust to see if it would help, I tried changing localhost to host.docker.internal but it didn't help.
  
  
Posted one month ago

@<1523701087100473344:profile|SuccessfulKoala55> No, I am not able to upload a file with either configuration. However, the first configuration works better in that I can use the task CLI.

  
  
Posted 16 days ago

Just wanted to add that the above error api_server is misconfigured seems to happen regardless of whether I modify the standard docker_compose.yml

  
  
Posted one month ago

So the first configuration works?

  
  
Posted 17 days ago

@<1523701087100473344:profile|SuccessfulKoala55> I see the api_server is misconfigured error in the logs for the clearml-fileserver . Unfortunately I can't copy and paste the logs for policy reasons, but I'll paraphrase them below:

  • config files loaded from /opt/clearml/fileserver
  • [ERROR] [werkzeug] (ip address) code 400, message bad request version
  • urllib3.connectionpool Retrying after connection broken by NewConnectionError
  • Traceback: urllib3/connection.py: ConnectionRefusedError [Errno 111] Connection refused- During the handling of above exception, another error occured: MaxRetryError- During the handling of above exception: ConnectionError: HTTPConnectionPool(host='apiserver', port=8008): Max retries exceeded with url: /auth.login- During the handling of above exception: ValueError : Connection Error: it seems api_server is misconfigured
  
  
Posted one month ago

Hi @<1722786133810614272:profile|LazyClams7> where exactly do you see the error? Can you send the complete logs?

  
  
Posted one month ago

@<1523701087100473344:profile|SuccessfulKoala55> I did as you suggested: from the fileserver container, I was able to authenticate with the API server and receive a token.
however, from my development container if I do
clearml-data add --files Affairs.csv
I still get a connetion refused error.

  
  
Posted one month ago

I also tried using the ip address of the files_server container, but that didn't seem to make a difference

  
  
Posted one month ago

Hi @<1722786133810614272:profile|LazyClams7> , if you see api_server is misconfigured error in the logs for the clearml-fileserver , this basically means the filesserver can't reach the apiserver (it needs to), this doesn't seem to be a client issue. The filessererver tried to reach the apiserver by default using the None endpoint, which should be available inside the docker backend network - you can try to exec into the fileserver container and see if curl to that address works

  
  
Posted one month ago

Also, how are you configuring clearml.conf inside your container?

  
  
Posted one month ago
214 Views
12 Answers
one month ago
15 days ago
Tags
Similar posts