Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
A Question On Pipeline Artifacts And Their Use In Steps. I Want To Add An Artifact To The Pipeline (See Below), And Use It As Input To A Function Step. But The Following Does Not Work Because

A question on pipeline artifacts and their use in steps. I want to add an artifact to the pipeline (see below), and use it as input to a function step. But the following does not work because step name 'pipeline' is invalid Does anyone know if this is possible at all? Thanks for your help :)

pipe = PipelineController(name="my_pipeline", project="my_project")
pipe.upload_artifact(name="my_artifact", artifact_object=my_artifact)
pipe.add_function_step(
    name="step",
    function=step_function,
    function_kwargs=dict(param="${pipeline.artifacts.my_artifact}"),
)
  
  
Posted 3 months ago
Votes Newest

Answers 3


I think this is just a feature that is not currently supported, but was wondering if there was a way around it?

  
  
Posted 3 months ago

Hi @<1649221402894536704:profile|AdventurousBee56> , I'm not sure I understand. Can you add the full log and explain step by step what's happening?

  
  
Posted 3 months ago

Hi @<1523701070390366208:profile|CostlyOstrich36> , here's a more full example of what I would like to be able to do:

some_object_instance = dict(a=1, b=2)

def step_func(some_object):
    print(some_object)

pipe = PipelineController(name="my_pipeline", project="my_project")
pipe.upload_artifact(name="some_object", artifact_object=some_object_instance)
pipe.add_function_step(
    name="step",
    function=step_func,
    function_kwargs=dict(param="${pipeline.some_object}"),
)
pipe.start_locally(run_pipeline_steps_locally=True)

And the error is:

"/home/xxx/site-packages/clearml/automation/controller.py", line 2177, in _add_function_step
    self.__verify_step_reference(node=self.Node(name=name), step_ref_string=v)
  File "/home/xxx/site-packages/clearml/automation/controller.py", line 3116, in __verify_step_reference
    raise ValueError("Node '{}', parameter '{}', step name '{}' is invalid".format(node.name, v, prev_step))
ValueError: Node 'step', parameter '${pipeline.some_object}', step name 'pipeline' is invalid
  
  
Posted 3 months ago
206 Views
3 Answers
3 months ago
3 months ago
Tags
Similar posts