Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
We’Re Randomly Getting The Following Message -

We’re randomly getting the following message -

2023-03-07 11:07:26,123 [WARNING] [urllib3.connectionpool]: Retrying (Retry(total=237, connect=237, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x14e80c430>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /v2.20/projects.get_all

We’re running ClearML on K8s. Not sure how to debug this. It’s not consistent unfortunately.
Any ideas?

  
  
Posted one year ago
Votes Newest

Answers 5


But this is not a matter of the server itself being configured or not (I'm sure it is), but something in the SDK changing the SDK's configuration regarding the server's address

  
  
Posted one year ago

@<1523701083040387072:profile|UnevenDolphin73> when do you mean exactly? This error indicates a mis-configured server URL, I can't imagine how this would be sporadic...

  
  
Posted one year ago

But there's nothing of that sort happening. The process where it's failing is on getting tasks for a project.

  
  
Posted one year ago

Ah. Apparently getting a task ID while it’s running can cause this behaviour 🤔

  
  
Posted one year ago

That's what I thought @<1523701087100473344:profile|SuccessfulKoala55> , but the server URL is correct (and WebUI is functional and responsive).
In part of our code, we look for projects with a given name, and pull all tasks in that project. That's the crash point, and it seems to be related to having running tasks in that project.

  
  
Posted one year ago