Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Guys, Is There A Way To Timeout (From Clearml) A Task If That Running Too Long?

hi guys,
is there a way to timeout (from clearml) a task if that running too long?

  
  
Posted 8 months ago
Votes Newest

Answers 4


it should be fairly easy to write such a daemon

from clearml.backend_api.session.client import APIClient
client = APIClient()
timestamp = time() - 60 * 60 * 2  # last 2 hours
tasks = client.tasks.get_all(
            status=["in_progress"],
            only_fields=["id"],
            order_by=["-last_update"],
            page_size=100,
            page=0,
            created =[">{}".format(datetime.utcfromtimestamp(timestamp))],
        )
...

references:
None
None

  
  
Posted 8 months ago

my case more like there is a task/process that running but somehow its takes too long to completed. it can be because connection issue forgot to put connection timeout, a problem connection database, etc that makes status still running, but its traped in a situation like that.

so i want to force shutdown a task to failed if that happen

  
  
Posted 8 months ago

Hi @<1523701260895653888:profile|QuaintJellyfish58>
You mean some "daemon service" aborting Tasks that do not end after X hours? or is it based on CPU/GPU utilization?

  
  
Posted 8 months ago

i see thanks for the answer, i will read that reference.

  
  
Posted 8 months ago
572 Views
4 Answers
8 months ago
8 months ago
Tags
Similar posts