Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello ! I Am Having Some Issues While Running A Pipeline Remotely (Same Device But Queue + Agent) . Hitting The Following Issue On My First Step:

Hello ! I am having some issues while running a pipeline remotely (same device but queue + agent) . Hitting the following issue on my first step: 2024-01-24 14:38:47,129 - clearml.storage - ERROR - Could not download None , err: HTTPConnectionPool(host='localhost', port=8081): Max retries exceeded with url: /Full%20Overview/.datasets/Fashion%20MNIST/Fashion%20MNIST.cf1e886ab64d4f94b83712fae1afc036/artifacts/data/dataset.cf1e886ab64d4f94b83712fae1afc036.z0nngd18.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc043462f50>: Failed to establish a new connection: [Errno 111] Connection refused')) the host should be my local ip instead of localhost. it is configured correctly on the clearml.conf file. When my worker start, it shows

api.web_server = 
                                                                                                                                   
api.api_server = 
                                                                                                                                   
api.files_server = 
                                                                                                                                 
api.credentials.access_key = OLMJ45B0VVW04MRR9YIT                                                                                                                            
api.host = 
                                                                           

Anyone had a similar issue ?

  
  
Posted 9 months ago
Votes Newest

Answers 8


I believe i have found the issue. I previously had my clearml host as localhost While fixing my setup little by little, i changed this to the local ip of my machine. Unfortunnately as shown in this picture, the link was not updated, is there a way to achieve this ?
image

  
  
Posted 9 months ago

Hi @<1660817806016385024:profile|FantasticMole87> , you'll either have to re-run it or change something in the DB. I suggest the first option.

  
  
Posted 9 months ago

Understood, i am still experimenting with the tool, i will check with minio afterwards ton confirms ! Thanks a lot for the quick response !

  
  
Posted 9 months ago

That is indeed what i ended up doing, it does mean that any migration would lead to a corrupted DB just like mine. Wouldn't it make more sense to build this link from a env var instead in future versions ?

  
  
Posted 9 months ago

In the end the data is physically uploaded somewhere and ClearML keeps the link to it. What would you suggest?

  
  
Posted 9 months ago

The idea is that the server is not something that should be moved on a whim. Once you do setups you can test but once everything is setup it is expected that the server will remain in the same address.

Also, this is when you're using the files server. If you were using minio from the start this would be a non issue.

  
  
Posted 9 months ago

Haven't played enough with it to have a clear answer to that. Haven't played with storage on minio yet, maybe this would fix it. But I found it misleading to have a link to something that does not exist anymore. As the fix is either to manually update the all database or to repush the dataset again, my opinion is that it would make more sense to have the standard adress configurable (let's say None ) and only store only the location to be happened Full%20Overview/.datasets/Fashion%20MNIST/Fashion%20MNIST.cf1e886ab64d4f94b83712fae1afc036/artifacts/data/dataset.cf1e886ab64d4f94b83712fae1afc036.z0nngd18.zipas example

  
  
Posted 9 months ago

What do you think ?

  
  
Posted 9 months ago
543 Views
8 Answers
9 months ago
9 months ago
Tags