Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Am Seeing That Some Steps In A Pipeline Are Being Skipped. Like For Example, In A Pipeline With 4 Steps, It’S Directly Starting At Step 3. Is There Some Reason For This, Some Optimization Kicking In?

I am seeing that some steps in a pipeline are being skipped. Like for example, in a pipeline with 4 steps, it’s directly starting at step 3. Is there some reason for this, some optimization kicking in?

  
  
Posted 3 years ago
Votes Newest

Answers 12


I also have task_override that adds a version which changes each run

It's just a tag, so no real difference

  
  
Posted 3 years ago

Yes there was a bug that it was always cached, just upgrade the clearml
pip install git+

  
  
Posted 3 years ago

1.0.3

  
  
Posted 3 years ago

Hi TrickySheep9
Could you post the pipeline code here?
Also which clearml version are you using ?

  
  
Posted 3 years ago

` pipe = PipelineController(
default_execution_queue="minerva-default",
add_pipeline_tags=True,
target_project=pipelines_project,
)

    for step in self.config["steps"]:
        name = self._experiment_name(step)
        pipe.add_step(
            name=name,
            base_task_project=pipelines_project,
            base_task_name=name,
            parents=self._get_parents(step),
            task_overrides={
                "tags": [new_version],
            },
        )

    pipe.start()
    pipe.wait()
    pipe.stop() `
  
  
Posted 3 years ago

AgitatedDove14 - are there cases when it tries to skip steps?

  
  
Posted 3 years ago

This is my code, but it’s pretty standard

  
  
Posted 3 years ago

pipeline code itself is pretty standard

  
  
Posted 3 years ago

Hmm there was this one:
https://github.com/allegroai/clearml/commit/f3d42d0a531db13b1bacbf0977de6480fedce7f6
Basically always caching steps (hence the skip), you can install from the main branch to verify this is the issue. an RC is due in a few days (it was already supposed to be out but got a bit delayed)

  
  
Posted 3 years ago

From the code - it’s supposed to not cache if task override is different? I also have task_override that adds a version which changes each run

  
  
Posted 3 years ago

I also have a pipelines.yaml which i convert to a pipeline

  
  
Posted 3 years ago

Can I disable this cache?

  
  
Posted 3 years ago