Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi

Hi ,
Join us today for our community talk, where we'll discuss our updated roadmap, give an overview of our new pipeline V2 features and celebrate Hacktoberfest with some easy-to-contribute issues!
We'll meet on 17:30 (GMT+3)  https://us02web.zoom.us/j/82227800084?pwd=VVc1ZVUvL0FCT2R6ektDT3FUaU9pdz09 !

  
  
Posted 2 years ago
Votes Newest

Answers 11


Hi MelancholyElk85 , documentation is a soft spot for us, we're trying to be better but we aren't always 🙂 https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py this is an example, if you check the pipeline folder you'll see others (like pipeline_from_function.py

  
  
Posted 2 years ago

There are some questions in this channel already regarding pipeline V2. Is there any tutorial or changelog or examples I can refer to?

  
  
Posted 2 years ago

I could insert some updated info to my conference talk if you share the recording by tomorrow morning 😄

  
  
Posted 2 years ago

SubstantialElk6 Yes! We should have it up, together with the previous one uploaded soon!

  
  
Posted 2 years ago

Hi i missed this. Is there a recording?

  
  
Posted 2 years ago

MelancholyElk85 , yes nesting pipelines is possible. as for flatting it afterwords, maybe AgitatedDove14 knows? I'm pretty sure it can't be done though

  
  
Posted 2 years ago

Very nice for small pipelines (where every step could be put into a function in a single repository)

  
  
Posted 2 years ago

AnxiousSeal95 We can make a nested pipeline, right? Like if the top pipeline calls add_step to create steps from tasks, and then we decompose any single step further and create a sub-pipeline from decorators there. We should be able to do that, because PipelineController is itself a task, right?

Also, is there a way to unfold such nested pipeline into a flat pipeline? So that only a single pipeline task is created, and it draws a single detailed DAG in PLOTS tab?

  
  
Posted 2 years ago

MelancholyElk85 I'll talk to the guy in charge and try! You can find main points here 🙂 https://docs.google.com/document/d/1QlPiDO2EzDq_HRvuVhYwPh5F_IsjgmmRTlPNdHik10k/edit

  
  
Posted 2 years ago

It's decorators. It's functions as steps, It's the ability to add metric tracking for pipelines. It's a better way to move artifacts between steps (way less boilerplate code). It's pipeline instance versioning (for the upcoming UI changes), it's better step parallelizations (If steps are not dependent on each other, they will automatically be parallelized) and I've probably missed some features....

  
  
Posted 2 years ago

Ah, ok, pipeline v2 is just about decorators? This example has already been there for a while

  
  
Posted 2 years ago