Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Everyone! I'M Trying To Use Arguments With The Pipeline But Can Not Make It Work. My Case Is

Hello everyone! I'm trying to use arguments with the pipeline but can not make it work. My case is
defined PipelineController add_parameter("docker_image") when I try to pass docker_image to the functional_step of the pipeline I always get the default value, however I did the update in the UI
Does anyone had similar issues?

  
  
Posted 2 years ago
Votes Newest

Answers 18


` pipe = PipelineController(
project='examples',
name='Pipeline demo',
version='1.1',
add_pipeline_tags=False,
)

set the default execution queue to be used (per step we can override the execution)

pipe.set_default_execution_queue('default')

add pipeline components

pipe.add_parameter(
name='url',
description='url to pickle file',
default=' '
)
pipe.add_function_step(
name='step_one',
function=step_one,
function_kwargs=dict(pickle_data_url='${pipeline.url}'),
function_return=['data_frame'],
cache_executed_step=True,
) `This is in the examples

  
  
Posted 2 years ago

Try this, I tested it and it works:
docker=pipe._parse_step_ref("${pipeline.url}")It's hack-ish but it should work. I'll try and get a fix in one of the upcoming SDK releases that supports parsing references for parameters other than kwargs

  
  
Posted 2 years ago

I have another issue with pipelines, I have described it in the another thread would you mind if I tag you there? because no solution 😞

  
  
Posted 2 years ago

no, because I want to use pipe.add_parameter in the docker field of the pipe.add_function_step not in the function_kwargs

  
  
Posted 2 years ago

AnxiousSeal95 Thank you so much! I will use it.

  
  
Posted 2 years ago

Can you try: '${pipeline.docker_image}' ?

  
  
Posted 2 years ago

Just checking, are you just trying to use a different docker image in a task? Because then you might want to use this: https://clear.ml/docs/latest/docs/apps/clearml_task/#docker
https://clear.ml/docs/latest/docs/clearml_agent#docker-mode

  
  
Posted 2 years ago

Tried and as output clearml-agent is trying to pull image '${pipeline.docker_image}' can not convert it to the value

  
  
Posted 2 years ago

AHHHHHHHHHHHH! That makes more sense now 😄 😄
Checking 🙂

  
  
Posted 2 years ago

AnxiousSeal95 Do you know this one?

  
  
Posted 2 years ago

Yeah, with pleasure 🙂

  
  
Posted 2 years ago

Checking 🙂

  
  
Posted 2 years ago

pipe.add_function_step(
name='step_one',
function=step_one,
function_kwargs=dict(pickle_data_url='${pipeline.url}'),
function_return=['data_frame'],
cache_executed_step=True,
#############
docker='${pipeline.url}' !!!!!! this does not work
) ``

  
  
Posted 2 years ago

Hey GrotesqueDog77 , so it seems like references only works on "function_kwargs" and not other function step parameter.
I'm trying to figure out if there's some workaround we can offer 🙂

  
  
Posted 2 years ago

function_kwargs they work , but docker parameter not

  
  
Posted 2 years ago

TimelyMouse69 the main problem is the arguments here is the code snippet
pipeline = PipelineController( name="Awesome Pipeline") pipeline.add_parameter( "docker_image", default="DEFAULT_DOCKER")And the I have functional step, where I want to use the argument
pipeline.add_function_step(name="best_step", docker="${pipeline.docker_image}"And also I tried
parameters = pipeline.get_parameters() pipeline.add_function_step(name="best_step", docker=parameters["docker_image"])But when I execute this in the first case it passes string literal ${pipeline.docker_image} and on the second version always default value.

  
  
Posted 2 years ago

Did you try with function_kwargs?

  
  
Posted 2 years ago

But I want to use value from the arguments.

  
  
Posted 2 years ago
1K Views
18 Answers
2 years ago
one year ago
Tags
Similar posts