Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, Is There Anyway To Get The Id Of The Pipeline Using Pipeline Name? I Need The Id Of The Pipeline So That I Can Schedule The Pipeline To Run Via

Hi all,

Is there anyway to get the id of the pipeline using pipeline name? I need the id of the pipeline so that I can schedule the pipeline to run via TaskScheduler

  
  
Posted 8 months ago
Votes Newest

Answers 16


@<1523701087100473344:profile|SuccessfulKoala55> @<1523701205467926528:profile|AgitatedDove14> Same results, it's not able to find the Project

tasks = Task.get_tasks(project_name=task_to_schedule.get_project_name(),
                           task_name=task_to_schedule.name,
                           task_filter={"search_hidden": True})
print([task.name for task in tasks])
# Output: 
# 2023-07-09 11:08:11,693 - clearml - WARNING - No projects were found with name(s): Project Team NASA/.pipelines/NASA Pipeline
# []
  
  
Posted 7 months ago

Hi @<1523701205467926528:profile|AgitatedDove14> , It works if I dont specify the project name and just give the task name. So I modified the above code a bit and now I'm able to get the id of the latest pipeline run.

task_to_schedule: Task = Task.get_task(task_id='b0732c334115432f8dc0cec0dbfdbdcb')
print(task_to_schedule.get_project_name())
# Output: Project Team NASA/.pipelines/NASA Pipeline
print(task_to_schedule.name)
# Output: NASA Pipeline #11


# task: Task = Task.get_task(project_name=task_to_schedule.get_project_name(),
#                            task_name=task_to_schedule.name)
# Error: ValueError: No projects found when searching for `Project Team NASA/.pipelines/NASA Pipeline`

tasks = Task.get_tasks(task_name= task_to_schedule.name,
                       task_filter={
                           "status": ["completed"],
                           "order_by": ["-last_update"]
                       })
# Returns multiple tasks by the same name 
# (The pipeline was run by scheduler multiple times, so same name)

required_task: Task = tasks[0]
# Here I'm picking the latest completed pipeline

print(required_task.id)
# Output: 80836a14e8b0482cab14fa58484b99d0 (id of latest pipeline run)
  
  
Posted 7 months ago

then will have to rerun the pipeline code then manually get the id and update the task.

Makes total sense to me!

Failed auto-generating package requirements: _PyErr_SetObject: exception SystemExit() is not a BaseException subclass

Not sure why you are getting this one?!

ValueError: No projects found when searching for MyProject/.pipelines/PipelineName

hmm, what are you getting with:

task = Task.get_task(pipeline_uid_here)
print(task.get_project_name())
  
  
Posted 7 months ago

Same error

task_to_schedule: Task = Task.get_task(task_id='b0732c334115432f8dc0cec0dbfdbdcb')
print(task_to_schedule.get_project_name())
# Output: Project Team NASA/.pipelines/NASA Pipeline
print(task_to_schedule.name)
# Output: NASA Pipeline #11


task: Task = Task.get_task(project_name=task_to_schedule.get_project_name(),
                           task_name=task_to_schedule.name,
                           task_filter={"search_hidden": True})
print(task.id)
# Error: ValueError: No projects found when searching for `Project Team NASA/.pipelines/NASA Pipeline`
  
  
Posted 7 months ago

Try passing search_hidden: True in the tasks filer

  
  
Posted 7 months ago

@<1523701205467926528:profile|AgitatedDove14> I think this.kigjt be since the project itself is hidden?

  
  
Posted 7 months ago

Hi

, It works if I dont specify the project name and just give the task name

But now it searches for it globally , which is not very stable:

Let me check why it fails to find the project...

  
  
Posted 7 months ago

Hi @<1587615463670550528:profile|DepravedDolphin12> , it seems this is an internal issue in the SDK (resolving the project is done without using the task filter) - we will fix this for the next version

  
  
Posted 7 months ago

Hi @<1587615463670550528:profile|DepravedDolphin12>

Is there anyway to get the id of the pipeline using pipeline name?

In the UI top right "details" panel should have the Pipeline ID
Is this what you are looking for ?

  
  
Posted 8 months ago

Hey @<1523701205467926528:profile|AgitatedDove14> I need to access it programmatically. Just like we get the task id by Task.get_task().id

  
  
Posted 8 months ago

So inside the pipeline logic you can do Task.current_task().id
Or inside a component Task.current_task().parent

  
  
Posted 8 months ago

Oh I see
but now I'm confused if this is from code, why aren't you coping the Pipeline ID from the UI?
regrading the query, it should be something like

task_to_schedule = Task.get_task(project_name='MyProject/.pipelines/PipelineName',                                  task_name='PipelineName')
  
  
Posted 7 months ago

why aren't you coping the Pipeline ID from the UI? --> I have tried this and it works but let's say if I updated the pipeline code, then will have to rerun the pipeline code then manually get the id and update the task. But if there is way in which I can directly fetch the id using pipeline name, it'll allow me to get the latest pipeline to be scheduled without manual intervention. Please correct if my understanding is wrong.

task_to_schedule = Task.get_task(project_name='MyProject/.pipelines/PipelineName',                                  task_name='PipelineName')

I have tried the given code Snippet and it throws below error:

raise ValueError('No {entity}s found when searching for `{query}`'.format(**locals()))
ValueError: No projects found when searching for `MyProject/.pipelines/PipelineName`
Failed auto-generating package requirements: _PyErr_SetObject: exception SystemExit() is not a BaseException subclass
  
  
Posted 7 months ago

Hi @<1523701205467926528:profile|AgitatedDove14> , The code returns the project, but if I use the same project in get_task it's throwing ValueError

task_to_schedule: Task = Task.get_task(task_id='b0732c334115432f8dc0cec0dbfdbdcb')
print(task_to_schedule.get_project_name())
# Output: Project Team NASA/.pipelines/NASA Pipeline
print(task_to_schedule.name)
# Output: NASA Pipeline #11


task: Task = Task.get_task(project_name=task_to_schedule.get_project_name(),
                           task_name=task_to_schedule.name)
# Error: ValueError: No projects found when searching for `Project Team NASA/.pipelines/NASA Pipeline` 
  
  
Posted 7 months ago

But this is a different script for TaskScheduler , something like below:

scheduler = TaskScheduler(...)

task_to_schedule = Task.get_task(project_name='MyProject',
                                  task_name='MyTask')
scheduler.add_task(schedule_task_id=task_to_schedule.id, ...)

# Do we have something similar to this? Like below
# pipeline_to_schedule = Task.get_task(project_name='MyProject',
                                #   task_name='MyPipeline')

# scheduler.add_task(schedule_task_id=pipeline_to_schedule.id, ...)
  
  
Posted 8 months ago
477 Views
16 Answers
8 months ago
7 months ago
Tags
Similar posts