Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Given I Want To Run A Task In A Pipeline Using A Base Task Id. One Of My Steps Just Finds The Latest Results To Use. I Want The Task To Output The Id Of The Results And The Next Step To Use It. How Would I Go About Doing This? Is There A Way To Pass Just

Given I want to run a task in a pipeline using a base task id. One of my steps just finds the latest results to use. I want the task to output the id of the results and the next step to use it. How would I go about doing this? Is there a way to pass just a string between tasks in pipeline?

  
  
Posted one year ago
Votes Newest

Answers 4


Hi SmugTurtle78 , I think you can set it up as follows (or something similar):
pipe.add_step( name="stage_train", parents=["stage_process"], base_task_project="examples", base_task_name="Pipeline step 3 train model", parameter_override={"General/dataset_task_id": "${stage_process.id}"}, )Note that in parameter_override I take a task id from a previous step and insert it into the configuration/parameters of the current step. Is that what you're looking for?

  
  
Posted one year ago

actually not exactly, The parameter I wanted to pass is not an input parameter of the parent task, I would like to save it as an artifact or something like that.

  
  
Posted one year ago

Well if you save it as an artifact, that artifact is accessible by other tasks and passable via the pipeline with monitor_artifacts parameter in add_step()
https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#add_step

  
  
Posted one year ago

Dont sure if this a good solution for me since I wanted to add it as a paramater....
For example:
In dataset_creation utility I do this upload:
task.upload_artifact('id_of_running_creation', artifact_object=id_of_running)In Inference utility I do this upload:
task.upload_artifact('id_of_running_inference', artifact_object=id_of_running)and I would like to use them as a parameters to the next step of the pipeline run:
like this:
pipe.add_step( name='post_processing',...., parameter_override= {"General/id_of_running_creation": "${dataset_creation.artifacts.id_of_running_creation}", "General/id_of_running_inference": "${inference.artifacts.id_of_running_inference}"}...but it doesn't work since the artifact is a string ( I tried also with dataframe but then I cant change it to string before I execute the next step)

  
  
Posted one year ago