Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Trying To Make Use Of New Capabilities Of Dag Creation In Clearml. Seems That Api Has Changed Pretty Much Since A Few Versions Back. There Seems To Be No Need In

Hi, I'm trying to make use of new capabilities of DAG creation in ClearML. Seems that api has changed pretty much since a few versions back. There seems to be no need in controller_task anymore, right? Is Clear Show on Youtube deprecated now? 🙂 Also, in new examples on Github, like this one https://github.com/allegroai/clearml/blob/master/examples/pipeline/step1_dataset_artifact.py ,
task.execute_remotely is called everywhere. Is it required now?

The pipeline creates the tasks, but never executes/enqueues them (they are all in Draft mode). No DAG graph appears in RESULTS/PLOTS tab.

AgitatedDove14 Please, could you help troubleshoot this?

  
  
Posted 2 years ago
Votes Newest

Answers 21


👍
Okay But we should definitely output an error on that

  
  
Posted 2 years ago

Seems that api has changed pretty much since a few versions back.

Correct, notice that your old pipelines Tasks use the older package and will still work.

There seems to be no need in 

controller_task

 anymore, right?

Correct, you can just call pipeline.start() 🙂

The pipeline creates the tasks, but never executes/enqueues them (they are all in 

Draft

 mode). No DAG graph appears in 

RESULTS/PLOTS

 tab.

Which version are we talking here ? v1.1.2? or the latest from GitHub ?

  
  
Posted 2 years ago

Hmm that is odd.
Can you verify with the latest from GitHub?
Is this reproducible with the pipeline example code?

  
  
Posted 2 years ago

Agent 1.1.0
Python client 1.1.1

  
  
Posted 2 years ago

Sorry for the delay

Not reproduced, but caught another error when running pipeline_from_tasks.py
Traceback (most recent call last): File "pipeline_from_tasks.py", line 31, in <module> pipe.add_step(name='stage_data', base_task_project='examples', base_task_name='pipeline step 1 dataset artifact') File "/home/kirillfish/.local/lib/python3.6/site-packages/clearml/automation/controller.py", line 276, in add_step base_task_project, base_task_name)) ValueError: Could not find base_task_project=examples base_task_name=pipeline step 1 dataset artifact

  
  
Posted 2 years ago

OK, I managed to launch the example and it works

  
  
Posted 2 years ago

👍

  
  
Posted 2 years ago

MelancholyElk85 I'm assuming you have the agent setup and everything in the example code works, is that correct ?
Where is it failing on your pipelines ?

  
  
Posted 2 years ago

But I still cannot launch my own pipelines

  
  
Posted 2 years ago

but never executes/enqueues them (they are all in 

Draft

 mode).

All pipeline steps are not enqueued ?
Is the pipeline controller itself running?

  
  
Posted 2 years ago

pipeline controller itself is stuck at running mode forever all step tasks are created but never enqueued
I can share some code

  
  
Posted 2 years ago

The pipeline is initialized like this
pipe = PipelineController(project=cfg['pipe']['project_name'], name='pipeline-{}'.format(name_postfix), version='1.0.0', add_pipeline_tags=True) pipe.set_default_execution_queue('my-queue')
Then for each step I have a base task which I want to clone

step_base_task = Task.get_task(project_name=cfg[name]['base_project'], task_name=cfg[name]['base_name'])
I want to modify a lot of params (docker image, docker params, task params, requirements, repository, branch...). From what I saw, the API of how this can be done has changed multiple times over the last few versions. In short, I was not able to do it with Task.clone and Task.create , the behavior differs from what is described in docs and docstrings (this is another story - I can submit an issue on github later)

I ended up exporting task configuration, modifying it in-place and then importing it to another task

` export_data = step_base_task.export_task()
export_data['name'] = '{} {}'.format(name, name_postfix)
export_data['project_name'] = cfg[name]['base_project']
export_data['script']['diff'] = ''
export_data['script']['branch'] = 'main'
export_data['script']['repository'] = cfg[name]['repo']
export_data['script']['requirements'] = {}
export_data['container']['image'] = 'registry.gitlab.com/my-image:my-tag'
export_data['container']['arguments'] = '-v /root/data:/data'

task = Task.import_task(export_data) Then I take this newly created task and add it as a pipeline step (I don't want to clone it one more time though, thus clone_base_task=False ` )

pipe.add_step( name=name, base_task_id=task.id, parents=parents, parameter_override=dict(flattener(cfg[name]['override'])), cache_executed_step=True, clone_base_task=False )
After adding all the steps, I simply run

pipe.start() pipe.wait() pipe.stop()
And we have what we have

  
  
Posted 2 years ago

I can share some code

Please do 🙂

  
  
Posted 2 years ago

AgitatedDove14 any ideas? 😀

  
  
Posted 2 years ago

. In short, I was not able to do it with 

Task.clone

 and 

Task.create

 , the behavior differs from what is described in docs and docstrings (this is another story - I can submit an issue on github later)

The easiest is to use task_ task_overrides
Then pass:
task_overrides = dict('script': dict(diff='', branch='main'))

  
  
Posted 2 years ago

MelancholyElk85 assuming we are running with clearml 1.1.1 , let's debug the pipeline and instead of pipeline start/wait/stop :
Let's do:
pipeline.start_locally(run_pipeline_steps_locally=False)

  
  
Posted 2 years ago

pipeline launches on the server anyway (appears on the web UI)

  
  
Posted 2 years ago

it has the same effect as start/wait/stop, kinda weird

  
  
Posted 2 years ago

What's the difference between the example pipeeline and this code ?
Could it be the "parents" argument ? what is it?

  
  
Posted 2 years ago

creates all the step tasks in draft mode and then stucks

  
  
Posted 2 years ago

You are right, I had [None] as parents in one of the tasks. Now this error is gone

  
  
Posted 2 years ago