Reputation
Badges 1
19 × Eureka!on a nother machine with the relevant queue
currently yes.
but as I said I want to create a pipeline of pipelines
let's say I have pipeline_1 which has 10 tasks in it.
and I also have pipeline_2 with has 10 tasks in it.
I want a new_pipeline that will run pipeline_1 and after that will run pipeline_2.
If I cannot do pipeline of pipelines, it means that I need to create a pipeline containing 20 tasks using add_step !
and In the future if I would like to add pipeline_3 to all of this, I'll need again to use add_step to all the 20 task + the ones of pipeline_3.
I makes all the process very uncomfort...
how?
because I tried using PipelineController to create new pipeline and on add_step I just added as base_task_id the id of the pipeline. but it failed with an error 😕
am I doing something wrong ?
what I mean is that I want to create a new pipeline of few existing pipelines:
ex:
new_pipeline : input
-> existing_pipeline_1
-> existing_pipeline_2
can I do such a thing?
this errors only appears to occur only when I enter on add_step an id of a pipeline. If I enter an id of single task no error occur.
earlier you've said I can managed to do so with some hacks. do u have an example or can u explain what I need to do 🙂 ?
maybe some hacks will be ok for now too
I'm using the PipelineController to ctreate a pipeline.
now I want to add 2 other pipelines to this main-pipeline using add_step (so instead of task-id I'm giving it a pipeline-id.
but it doesn't work and always throws error.
If I'm doing the same thing with task ids all works.. so I guess there is another correct way to do a pipeline of pipelines
not a memo issue.. I also tried to switch the queue to another new machine with new clearml-agent installation
But I get the same result ..
if I use start_locally instead of start this issue doesn't occur
so same queue for multiple agents will do the work? why do I need to split into steps? so each step will run parallel to the other ?
Can't a task/pipeline's specific node run on multiple agents?
@<1523701070390366208:profile|CostlyOstrich36> I still didn't managed to solved this issue .. can u please help me with that or send and working example of valid pipeline of pipelines (if possible ofc 🙂 )?
found it.my_task.data.runtime.get("version")
exactly because of the reason I described above:
None