Hey, Can Someone Help With Using Parameter In The Pipeline? According To The Documentation:

Answered

Hey, can someone help with using parameter in the pipeline?
According to the documentation:

pipe.add_parameter(
    name='pickle_url',
    description='url to pickle file', 
    default='

'
)

and then access it with:

"${pipeline.<parameter_name>}"

however it is not working, I am running:

pipe.add_parameter(
     name='branch',
     description='choose the branch for the URL',
     default='main'
) 
...
pipe.add_function_step(
    name='step4-test4',
    repo='<my repo>',
    repo_branch='${pipeline.branch}',
    function=Test4,
    function_return=['step4'],
    parents=['step3-test3'],
)

and the pipelines sees it as

branch = ${pipeline.branch}

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					WorriedSwan6
				
					0
					 × 1

Votes Newest

Answers 3

Hi @<1726047624538099712:profile|WorriedSwan6> ! At the moment, only the function_kwargs and queue parameters accept such references. We will consider supporting them for other fields as well in the near future

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					SmugDolphin23
				
					0

Hi @<1726047624538099712:profile|WorriedSwan6> , can you share the log?

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

Hey @<1523701087100473344:profile|SuccessfulKoala55>
The relevant log from the pipeline shows:

error: pathspec 'origin/${pipeline.branch}' did not match any file(s) known to git
Repository cloning failed: Command '['git', 'checkout', 'origin/$%7Bpipeline.branch%7D', '--force']' returned non-zero exit status 1.

and I believe this is because the params cannot be injected in such a way, not sure if this is a bug or normal behaviour.
I am able to to use it like so:

pipe.add_parameter(
     name='branch',
     description='github url',
     default='main'
) 

pipe.add_function_step(
     name='step1 - get data',
     function=Get_data,
     function_return=['data'],
     function_kwargs=dict(branch='${pipeline.branch}'),
     repo='<my repo>',
)

. but not like so:

pipe.add_parameter(
     name='branch',
     description='github url',
     default='main'
) 

pipe.add_function_step(
     name='step1 - get data',
     function=Get_data,
     function_return=['data'],
     repo_branch='${pipeline.branch}',
     repo='>my repo>',
)

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					WorriedSwan6
				
					0
					 × 1

Write your answer

319 Views

3 Answers

3 months ago