Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Everyone! I'M Trying To Use Arguments With The Pipeline But Can Not Make It Work. My Case Is

Hello everyone! I'm trying to use arguments with the pipeline but can not make it work. My case is
defined PipelineController add_parameter("docker_image") when I try to pass docker_image to the functional_step of the pipeline I always get the default value, however I did the update in the UI
Does anyone had similar issues?

  
  
Posted one year ago
Votes Newest

Answers 18


AHHHHHHHHHHHH! That makes more sense now 😄 😄
Checking 🙂

  
  
Posted one year ago

Just checking, are you just trying to use a different docker image in a task? Because then you might want to use this: https://clear.ml/docs/latest/docs/apps/clearml_task/#docker
https://clear.ml/docs/latest/docs/clearml_agent#docker-mode

  
  
Posted one year ago

function_kwargs they work , but docker parameter not

  
  
Posted one year ago

Try this, I tested it and it works:
docker=pipe._parse_step_ref("${pipeline.url}")It's hack-ish but it should work. I'll try and get a fix in one of the upcoming SDK releases that supports parsing references for parameters other than kwargs

  
  
Posted one year ago

pipe.add_function_step(
name='step_one',
function=step_one,
function_kwargs=dict(pickle_data_url='${pipeline.url}'),
function_return=['data_frame'],
cache_executed_step=True,
#############
docker='${pipeline.url}' !!!!!! this does not work
) ``

  
  
Posted one year ago

Hey GrotesqueDog77 , so it seems like references only works on "function_kwargs" and not other function step parameter.
I'm trying to figure out if there's some workaround we can offer 🙂

  
  
Posted one year ago

TimelyMouse69 the main problem is the arguments here is the code snippet
pipeline = PipelineController( name="Awesome Pipeline") pipeline.add_parameter( "docker_image", default="DEFAULT_DOCKER")And the I have functional step, where I want to use the argument
pipeline.add_function_step(name="best_step", docker="${pipeline.docker_image}"And also I tried
parameters = pipeline.get_parameters() pipeline.add_function_step(name="best_step", docker=parameters["docker_image"])But when I execute this in the first case it passes string literal ${pipeline.docker_image} and on the second version always default value.

  
  
Posted one year ago

Tried and as output clearml-agent is trying to pull image '${pipeline.docker_image}' can not convert it to the value

  
  
Posted one year ago

But I want to use value from the arguments.

  
  
Posted one year ago

Can you try: '${pipeline.docker_image}' ?

  
  
Posted one year ago

AnxiousSeal95 Do you know this one?

  
  
Posted one year ago

Checking 🙂

  
  
Posted one year ago

Did you try with function_kwargs?

  
  
Posted one year ago

AnxiousSeal95 Thank you so much! I will use it.

  
  
Posted one year ago

I have another issue with pipelines, I have described it in the another thread would you mind if I tag you there? because no solution 😞

  
  
Posted one year ago

Yeah, with pleasure 🙂

  
  
Posted one year ago

no, because I want to use pipe.add_parameter in the docker field of the pipe.add_function_step not in the function_kwargs

  
  
Posted one year ago

` pipe = PipelineController(
project='examples',
name='Pipeline demo',
version='1.1',
add_pipeline_tags=False,
)

set the default execution queue to be used (per step we can override the execution)

pipe.set_default_execution_queue('default')

add pipeline components

pipe.add_parameter(
name='url',
description='url to pickle file',
default=' '
)
pipe.add_function_step(
name='step_one',
function=step_one,
function_kwargs=dict(pickle_data_url='${pipeline.url}'),
function_return=['data_frame'],
cache_executed_step=True,
) `This is in the examples

  
  
Posted one year ago
645 Views
18 Answers
one year ago
one year ago
Tags
Similar posts