Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello All, I'M Trying To Queue A Task In Python But I'D Like To Reuse The Prior Task Id. In The Webapp You Can

Hello all,

I'm trying to queue a task in python but I'd like to reuse the prior task ID. In the webapp you can Reset and Enqueue a task and it'll reuse the task ID. I'm trying to do the same thing with the Python SDK (in the code example below, I'd like to reuse the task ID when reuse_task = True .

Code:

def run_task(
    task_name: str,
    project_name: str,
    execution_queue_name: str,
    production: bool = False,
    reuse_task: bool = False,
    parameter_overrides: DictConfig = None,
):
    # check that parameter_overrides is a DictConfig or dict
    if parameter_overrides:
        assert isinstance(
            parameter_overrides, (DictConfig, dict)
        ), "parameter_overrides must be a DictConfig or dict."

        # if parameter_overrides is a DictConfig convert it to a dict
        if isinstance(parameter_overrides, DictConfig):
            parameter_overrides = OmegaConf.to_container(
                parameter_overrides, resolve=True
            )

    # get the most recent task
    task_id = get_most_recent_task(
        task_name=task_name, project_name=project_name, production=production
    )

    if task_id is None:
        logger.error(f"Task {task_name} not found in ClearML.")
        return

    # grab the correct task
    source_task: Task = Task.get_task(task_id=task_id)

    if reuse_task:
        source_task.reload()
        # if reuse_task is True, pipe to the same task
        target_task: Task = source_task
    else:
        # Clone the task to pipe to. This creates a task with status Draft whose parameters can be modified.
        target_task: Task = Task.clone(
            source_task=source_task, comment="Scheduled Task."
        )

    # if parameter_overrides is not None, update the parameters of the cloned task
    if parameter_overrides:
        # Get the original parameters of the Task, modify the value of one parameter,
        #   and set the parameters in the next Task
        target_task_parameters: dict = target_task.get_parameters()

        target_task_parameters.update(parameter_overrides)

        target_task.set_parameters(target_task_parameters)

    # remove all tags from the task
    target_task.set_tags([])

    # Enqueue the cloned task to the queue.
    logger.info(f"Enqueueing [{task_name}] to: [{execution_queue_name}] queue.")

    target_task.execute_remotely(queue_name=execution_queue_name)

I've tried using source_task.reload() and source_task.reset() . I've also tried using target_task.execute_remotely(queue_name=execution_queue_name) and Task.enqueue(task=target_task.id, queue_name=execution_queue_name) . I've also tried using force=true in reset() and in enqueue. What am I missing? 🫠

  
  
Posted 11 months ago
Votes Newest

Answers 5


I'm trying to queue a task in python but I'd like to reuse the prior task ID.

is it your own Task? i,,e, enqueue yourself, if this is the case use task.execute_remotely it will do just that.
If this is another Task, then if it is aborted then you can just enqueue it, by definition it will continue with the Same Task ID.

  
  
Posted 11 months ago

This turns out to be a layer-8 error . task.execute_remotely does work but there was a bug in my code and I wasn't correctly setting the reuse_task flag when run. Sorry to bother the both of you with my mistake.

  
  
Posted 11 months ago

No worries, glad to hear you found it 😄

  
  
Posted 11 months ago

No error. Just a new task each time.

  
  
Posted 11 months ago

Hi @<1545216070686609408:profile|EnthusiasticCow4> , what error are you getting exactly? I would assume reset() should do the job...

  
  
Posted 11 months ago
780 Views
5 Answers
11 months ago
11 months ago
Tags
Similar posts