I also have a pipelines.yaml which i convert to a pipeline
AgitatedDove14 - are there cases when it tries to skip steps?
From the code - it’s supposed to not cache if task override is different? I also have task_override that adds a version which changes each run
` pipe = PipelineController(
default_execution_queue="minerva-default",
add_pipeline_tags=True,
target_project=pipelines_project,
)
for step in self.config["steps"]:
name = self._experiment_name(step)
pipe.add_step(
name=name,
base_task_project=pipelines_project,
base_task_name=name,
parents=self._get_parents(step),
task_overrides={
"tags": [new_version],
},
)
pipe.start()
pipe.wait()
pipe.stop() `
Hi TrickySheep9
Could you post the pipeline code here?
Also which clearml version are you using ?
Hmm there was this one:
https://github.com/allegroai/clearml/commit/f3d42d0a531db13b1bacbf0977de6480fedce7f6
Basically always caching steps (hence the skip), you can install from the main branch to verify this is the issue. an RC is due in a few days (it was already supposed to be out but got a bit delayed)
I also have task_override that adds a version which changes each run
It's just a tag, so no real difference
Yes there was a bug that it was always cached, just upgrade the clearmlpip install git+