Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Am Trying To Using Clearml Agent Version 0.16 ("Trains_Agent"), And I Always Get A Silent Exit After Starting Task Execution:

Hi,
I am trying to using ClearML agent version 0.16 ("trains_agent"), and I always get a silent exit after starting task execution:

Environment setup completed successfully
Starting Task Execution:

After this point, the Task becomes Completed, and the rest of the code is ignored.
Any idea what caused this, or how to debug it?

  
  
Posted 3 years ago
Votes Newest

Answers 6


Hi StaleMole4 ,
Can you share the segment of your task where you used ClearML? What code gets ignored?
Also, can you try using the latest ClearML Agent version (0.17.x)?

  
  
Posted 3 years ago

OK, and where do you call this function?

  
  
Posted 3 years ago

For debug purposes, in main:
if __name__ == "__main__": prms = enqueue_task({}, "Test task", Task.TaskTypes.custom) print(prms) print("Finished main")And the "print" gets ignored in the enqueued task

  
  
Posted 3 years ago

Hi SuccessfulKoala55 , thanks for the quick reply!

This is a stand-alone code, that creates a task, clones it, and enqueues the cloned task:
` def enqueue_task(trains_prms, task_name, task_type):

# Connect to trains
task = Task.init("OMD", task_name='Temp controller for ' + task_name, task_type=task_type) # , task_type=Task.TaskTypes.controller

# Select queue
trains_queue = "cpu_queue"

# Connect parameters
assert "is_controller_run" not in trains_prms
trains_prms["is_controller_run"] = True
trains_prms = task.connect(trains_prms)
print("Connected to trains")

if trains_prms["is_controller_run"]:

    # Controller work
    task.close()

    # Get a reference to the task to pipe to.
    next_task = Task.get_task(task_id=task.id)

    # Clone the task to pipe to. This creates a task with status Draft whose parameters can be modified.
    cloned_task = Task.clone(source_task=next_task, name=task_name)

    # Get the original parameters of the Task, modify the value of one parameter,
    #   and set the parameters in the next Task
    cloned_task_parameters = cloned_task.get_parameters()
    cloned_task_parameters["is_controller_run"] = False
    cloned_task.set_parameters(cloned_task_parameters)

    # Enqueue the Task for execution. The enqueued Task must already exist in the clearml platform
    print('Enqueue task: ' + task_name)
    Task.enqueue(cloned_task.id, queue_name=trains_queue)
    print('Enqueued task: ' + task_name)
    is_controller_run = True

    # Controller is finished
    return is_controller_run, trains_prms, -1

# Not the controller
return trains_prms["is_controller_run"], trains_prms, task `
  
  
Posted 3 years ago

It appears to be working now (though I'm still debugging).
Setting the conda environment directly appeared to have solved it:
task.set_base_docker(conda_env)

  
  
Posted 3 years ago

Thanks for the help!

  
  
Posted 3 years ago
512 Views
6 Answers
3 years ago
one year ago
Tags