Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey, Is There A Way To Set Pipeline Component Return Artifact Compression At A Pipeline Level ? It Would Allow To Make Big Dataframes Flow Across Component Without Having To Resort To Define Temporary Datasets, Currently It'S Generating Only Raw Pickles.

Hey, is there a way to set pipeline component return artifact compression at a pipeline level ? It would allow to make big dataframes flow across component without having to resort to define temporary datasets, currently it's generating only raw pickles.

  
  
Posted 11 months ago
Votes Newest

Answers 6


Well at this point I might as well try to write a PR implementing the behavior I described above

  
  
Posted 11 months ago

Hi @<1523702000586330112:profile|FierceHamster54> ! This is currently not possible, but I have a workaround in mind. You could use the artifact_serialization_function parameter in your pipeline. The function should return a bytes stream of the zipped content of your data with whichever compression level you have in mind.
If I'm not mistaken, you wouldn't even need to write a deserialization function in your case, because we should be able to unzip your data just fine.
Wdyt?

  
  
Posted 11 months ago

Hmm, in that case you might need to write it. Doesn’t hurt trying eitherway

  
  
Posted 11 months ago

That would be much appreciated

  
  
Posted 11 months ago

Thanks @<1523701435869433856:profile|SmugDolphin23> , tho are you sure I don't need to override the deserialization function even if I pass multiple distinct objects as a tuple ?

  
  
Posted 11 months ago

Linked github issue: None

  
  
Posted 11 months ago
708 Views
6 Answers
11 months ago
11 months ago
Tags