Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey, Is There A Way To Set Pipeline Component Return Artifact Compression At A Pipeline Level ? It Would Allow To Make Big Dataframes Flow Across Component Without Having To Resort To Define Temporary Datasets, Currently It'S Generating Only Raw Pickles.

Hey, is there a way to set pipeline component return artifact compression at a pipeline level ? It would allow to make big dataframes flow across component without having to resort to define temporary datasets, currently it's generating only raw pickles.

  
  
Posted one year ago
Votes Newest

Answers 6


Thanks @<1523701435869433856:profile|SmugDolphin23> , tho are you sure I don't need to override the deserialization function even if I pass multiple distinct objects as a tuple ?

  
  
Posted one year ago

That would be much appreciated

  
  
Posted one year ago

Well at this point I might as well try to write a PR implementing the behavior I described above

  
  
Posted one year ago

Linked github issue: None

  
  
Posted one year ago

Hi @<1523702000586330112:profile|FierceHamster54> ! This is currently not possible, but I have a workaround in mind. You could use the artifact_serialization_function parameter in your pipeline. The function should return a bytes stream of the zipped content of your data with whichever compression level you have in mind.
If I'm not mistaken, you wouldn't even need to write a deserialization function in your case, because we should be able to unzip your data just fine.
Wdyt?

  
  
Posted one year ago

Hmm, in that case you might need to write it. Doesn’t hurt trying eitherway

  
  
Posted one year ago
1K Views
6 Answers
one year ago
one year ago
Tags