Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi All, I Created A Pipeline That Creates A Task With A Script That Runs Some Hugingface Code And Then Runs A Slurm Batch That Executes "Clearml-Agent Exe" To Run The Task. But When The Script Runs, It Gives The Following Error: Valueerror: Task Object

Hi All,

I created a pipeline that creates a task with a script that runs some HugingFace code and then runs a slurm batch that executes "clearml-agent exe" to run the task. But when the script runs, it gives the following Error:

ValueError: Task object can only be updated if created or in_progress [status=completed fields=['hyperparams']]

The error raises when I execute trainer.train()

Please see the pipline and task code, and the entire output log attached

Would appreciate your help,


Posted one year ago
Votes Newest

Answers 5

Hi @<1577468638728818688:profile|DelightfulArcticwolf22> , it looks like you're trying to update some parameter of a task after it finished running...

Posted one year ago

OK I will try that.

Posted one year ago

I would suggest adding print outs during the code to better understand when this happens

Posted one year ago

Hi John, but the task is running the script code, so how can that be?

Posted one year ago

Hi John. It turns out that the task that ended was the pipeline step. Apparently since I create the task that I then run with clearml-agent execute, in the step, it has to remain running. I added task.wait_for_status() at the end of the step and now my pipeline works.

I have another question: Do you know if it is possible to use PipelineDecorator.component as a class decorator?

Posted one year ago