Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, From Time To Time Due To Connectivity Issues My Tasks Can'T Report To The Server For 5-20Mins And Fail Because Of That. Is There Any Way To Adjust Something In The Configuration File To Deal With That?

Hi, from time to time due to connectivity issues my tasks can't report to the server for 5-20mins and fail because of that. is there any way to adjust something in the configuration file to deal with that?

  
  
Posted one year ago
Votes Newest

Answers 11


2022-06-03 02:09:52 Jun 02 19:09:52 - Retrying (Retry(total=143, connect=143, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5cb1263940>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /v2.13/tasks.ping 2022-06-03 02:09:54 Jun 02 19:09:54 - Retrying (Retry(total=143, connect=143, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5ca95efac0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /v2.13/events.add_batch 2022-06-03 02:09:57 Jun 02 19:09:57 - Retrying (Retry(total=143, connect=143, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5ca95ef310>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /v2.13/events.add_batch 2022-06-03 02:09:58 2022-06-02 19:09:58,321 - clearml.Task - WARNING - ### TASK STOPPED - USER ABORTED - STATUS CHANGED ###these were the latest ones.

  
  
Posted one year ago

ContemplativeGoat37 I think there was an issues just lije you described and it was solved in later versions, upgrade to the latest clearml package version, you should be fine 🙂

  
  
Posted one year ago

ContemplativeGoat37 what are the exact errors you're getting?

  
  
Posted one year ago

yes it's manual one it was aborted by itself 1.1.2

  
  
Posted one year ago

the update helped in my situation

  
  
Posted one year ago

Ok, thank you. Will try

  
  
Posted one year ago

ContemplativeGoat37 and new

  
  
Posted one year ago

what was the old version?

  
  
Posted one year ago

1.1.2. the new one I don’t remember, you can check the version that was in the last one in pypi 3 months ago

  
  
Posted one year ago

ContemplativeGoat37
http://1.it seems the DNS resolving to the server fails? (Temporary failure in name resolution) Is this running on an agent, or manually ? "clearml.Task - WARNING - ### TASK STOPPED - USER ABORTED - STATUS CHANGED ###" Is this you manually aborting the Task or is it aborting itslef due to the connectivity ?
4. what's the clearml/clearml-agent versions ?

  
  
Posted one year ago

ContemplativeGoat37 hi, any updates? I have a similar issue due executing clearml-data create command, also the status is stuck in “uploading”

And when I’m trying to add a file to dataset, this happens:
Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f20d7231430>: Failed to establish a new connection: [Errno 111] Connection refused')': / Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f20d7231670>: Failed to establish a new connection: [Errno 111] Connection refused')': /

  
  
Posted one year ago