Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, I Have A Query Regarding The Retrieval Of Pipeline Details. I Have Already Created A Pipeline Under The Project Name "<Project_Name>" And The Pipeline Name Is "<Pipeline_Name>". I Would Like To Retrieve The Version Of This Pipeline. I Tried Using

Hi All,
I have a query regarding the retrieval of pipeline details. I have already created a pipeline under the project name "<project_name>" and the pipeline name is "<pipeline_name>". I would like to retrieve the version of this pipeline. I tried using the PipelineController class and its get method, but I was unable to find an attribute or method that provides the pipeline version.

Could anyone please provide guidance on how to retrieve the "Pipeline version" or find the latest version of a pipeline using the project and pipeline name in ClearML?
Thank you

  
  
Posted 2 months ago
Votes Newest

Answers 8


thank you @<1523701435869433856:profile|SmugDolphin23>

  
  
Posted 2 months ago

Hi, @<1523701435869433856:profile|SmugDolphin23> , the provided SDK still doesn't provide the version details, once the pipeline created from the draft state.

I’m encountering an issue that I hope you can assist with.

Issue #1:
As discussed in previous comments, we have a scenario where we create a draft pipeline and enqueue it for later. The problem arises when the pipeline is triggered - the version details are not being added to the pipeline that is enqueued from the draft state.

Could you guide me on how the version number can be reflected in the pipeline created from the draft state? Currently, it’s causing confusion as the pipeline doesn’t have a version, and we need to keep track of whether the pipeline was executed from the draft version or not.

Expectation: Pipeline created from the draft version with the version number assigned to it.

Issue #2:
After a pipeline is created in the draft state, since it doesn’t contain any version details. When the SDK method p._task.get_user_properties().get("version") is used to retrieve the version details of the pipeline, it returns “None”.

Expectation: The desired outcome is to obtain at least the most recent version of the pipeline from the project of the pipeline.

Please refer to the pipeline that doesn't have a version number
image

  
  
Posted 2 months ago

Hi, @<1523701435869433856:profile|SmugDolphin23> Good morning,

I am currently working with the ClearML pipeline and have automated the process of retrieving the pipeline version using a specific provided method, which is then added to the pipeline controller.

While the pipeline was in a draft state, this method worked perfectly fine. However, once the pipeline was created in the draft state and enqueued to the queue, I encountered an issue. Specifically, I was unable to retrieve the pipeline version using the same method. Instead of returning the version, it returned “None”.
Here is the method I used:

p = PipelineController.get(pipeline_project=project_name, pipeline_name=pipeline_name)
p._task._get_runtime_properties().get("version")

Could you please assist me in resolving this issue? I would like to get the version details of the pipeline, regardless of the pipeline’s state.

Thank you

  
  
Posted 2 months ago

Hi @<1626028578648887296:profile|FreshFly37> ! You could try getting the version via user properties as well: None .
so something like p._task.get_user_properties().get("version")

  
  
Posted 2 months ago

Hi @<1626028578648887296:profile|FreshFly37> ! Indeed, the pipeline gets tagged once it is running. Actually, it just tags itself. That is why you are encountering this issue. The version is derived in 2 ways: either you manually add the version using the version argument in the PipelineController , or the pipeline fetches the latest version out of all the pipelines that have ran, and auto-bumps that.
Please reference this function: None
Now that I think about it, I believe you could just call it on your draft pipeline and it should work.

We will handle the versions better in a future patch. Thank you

  
  
Posted 2 months ago

Hi @<1626028578648887296:profile|FreshFly37> ! You can get the version by doing:

p = Pipeline.get(...)
p._task._get_runtime_properties().get("version")

We will make the version more accessible in a future version

  
  
Posted 2 months ago

T hank you @<1523701435869433856:profile|SmugDolphin23> I got the info. I missed to update you here Im handling manually using the version argument.

  
  
Posted 2 months ago

great, glad you found a work-around

  
  
Posted 2 months ago
158 Views
8 Answers
2 months ago
2 months ago
Tags
Similar posts