Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Have A Task Which Uses Hydra For Configuration. I Want To Add This Taks To A Pipeline, And Pass The Full Hydra Config Objects To The Task. Is There A Way To Do It? I Get “Parameters Should Be In The Form Of “`Section-Name`/Parameter”, Example: “Args

hi, I have a task which uses hydra for configuration.
I want to add this taks to a pipeline, and pass the full hydra config objects to the task. is there a way to do it?
I get “parameters should be in the form of “section-name/parameter”, example: “Args/param”"

  
  
Posted 2 years ago
Votes Newest

Answers 10


CostlyOstrich36
pipe.add_step(name='train', parents=['data_pipeline', ], base_task_project='xxx', base_task_name='yyy', parameter_override={'OmegaConf': cfg.trainer})

  
  
Posted 2 years ago

and for the record - to override hydra params the syntax is :
parameter_override={'Hydra/x.y':1234}where x.y=1234 is how you would override the param via the cli

  
  
Posted 2 years ago

RoughTiger69 , Hi!

Can you please add a snippet of how you're trying to add it?

  
  
Posted 2 years ago

I want to pass the entire hydra omegaconf as a (nested) dictionary

  
  
Posted 2 years ago

thanks

  
  
Posted 2 years ago

Okay this is a bit tricky (and come to think about it, we should allow a more direct interface):
pipe.add_step(name='train', parents=['data_pipeline', ], base_task_project='xxx', base_task_name='yyy', task_overrides={'configuration.OmegaConf': dict(value=yaml.dump(MY_NEW_CONFIG), name='OmegaConf', type='OmegaConf YAML')} )Notice that if you had any other configuration on the base task, you should add them as well (basically it overwrites the configuration section)
Do you think this will work?

  
  
Posted 2 years ago

RoughTiger69 the easiest thing would be to use the override option of Hydra:
parameter_override={'Args/overrides': '[the_hydra_key={}]'.format(a_new_value)})wdyt?

  
  
Posted 2 years ago

I think it works.
small correction - use slash and not dot in configuration/OmegaConf:
parameter_override={'configuration/OmegaConf': dict...')})

  
  
Posted 2 years ago

the above only passes the overrides if I am not mistaken

  
  
Posted 2 years ago

I see... let me check something

  
  
Posted 2 years ago