Is it possible to trigger a pipeline run via API?
Yes! a pipeline is at the end a Task, you can take the pipeline ID and clone and enqueue it
pipeline_task = Task.clone("pipeline_id_here")
Task.enqueue(pipeline_task, queue_name="services")
You can also monitor the pipeline with the same Task inyerface.
wdyt?
I suspect there's an easier way to do this, as running the pipeline results in this. ClearML only picks up the script but not the modules in the repo:
Environment setup completed successfully
Starting Task Execution:
Traceback (most recent call last):
File "/root/.clearml/venvs-builds/3.10/code/main_clearml.py", line 5, in <module>
from clearml_pipeline.pipeline import executing_pipeline
ModuleNotFoundError: No module named 'clearml_pipeline'
Leaving process id 428
DONE: Running task 'df90d78f3d1b48ebaaac095cfcb04331', exit status 1
Process failed, exit code 1
Thanks AgitatedDove14 ! I can try it out and get back to you
AgitatedDove14 that works. Many thanks!
Is there any way to make that increment from last run?
pipeline_task = Task.clone("pipeline_id_here", name="new execution run here")
Task.enqueue(pipeline_task, queue_name="services")
wdyt?
I'm able to clone and run a pipeline! However this cloning using the ID still retains the pipeline run #number.
Is there any way to make that increment from last run?