Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Team! I Am Using

Hey team!
I am using clearml-serving to serve a custom pipeline and I was testing the offline scenario when the internet goes off. Everything works well in terms of uploading the data once the internet is back, but I have an issue with logging. My logger gets quite spammed with the retries to reconnect - any ideas how I could suppress them on the ClearML side?
Logs:

Retrying (Retry(total=237, connect=237, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7c6c748eafb0>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /v2.23/events.add_batch
Retrying (Retry(total=236, connect=236, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7c6d2c6c63b0>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /v2.23/tasks.get_all
Retrying (Retry(total=236, connect=236, read=240, redirect=240, status=240)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7c6c748d41f0>, 'Connection to XXX(host IP) timed out. (connect timeout=10.0)')': /v2.23/events.add_batch
Retrying (Retry(total=235, connect=235, read=240, redirect=240, status=240)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7c6d2c6c7e80>, 'Connection to XXX(host IP) timed out. (connect timeout=10.0)')': /v2.23/tasks.get_all

I have tried the following code but still didn't help with these logs:

self._task = Task.init(project_name=project_name,
    task_name=task_name,
    task_type=task_type,
    auto_connect_streams=False)

Thanks for the help! 🙌

  
  
Posted 4 months ago
Votes Newest

Answers 2


Hi @<1817731756720132096:profile|WickedWhale51> , are you using the Logger module from the SDK?

  
  
Posted 4 months ago

Yes, using that one

  
  
Posted 4 months ago