Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi All, I Created A Pipeline That Creates A Task With A Script That Runs Some Hugingface Code And Then Runs A Slurm Batch That Executes "Clearml-Agent Exe" To Run The Task. But When The Script Runs, It Gives The Following Error: Valueerror: Task Object

Hi All,

I created a pipeline that creates a task with a script that runs some HugingFace code and then runs a slurm batch that executes "clearml-agent exe" to run the task. But when the script runs, it gives the following Error:

ValueError: Task object can only be updated if created or in_progress [status=completed fields=['hyperparams']]

The error raises when I execute trainer.train()

Please see the pipline and task code, and the entire output log attached

Would appreciate your help,


Posted 5 months ago
Votes Newest

Answers 5

Hi John, but the task is running the script code, so how can that be?

Posted 5 months ago

Hi John. It turns out that the task that ended was the pipeline step. Apparently since I create the task that I then run with clearml-agent execute, in the step, it has to remain running. I added task.wait_for_status() at the end of the step and now my pipeline works.

I have another question: Do you know if it is possible to use PipelineDecorator.component as a class decorator?

Posted 5 months ago

OK I will try that.

Posted 5 months ago

Hi @<1577468638728818688:profile|DelightfulArcticwolf22> , it looks like you're trying to update some parameter of a task after it finished running...

Posted 5 months ago

I would suggest adding print outs during the code to better understand when this happens

Posted 5 months ago