Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, Is There A Way To Create A Pipeline From Files Which Are Already Tasks In Clearml (With Task Init And Their Own Parameters Which Are Connected From The Task)? I'Ve Read About Pipeline_From_Task But As I Seen All The Tasks Must Be Already Tasks Whi

Hi all,
Is there a way to create a pipeline from files which are already tasks in clearml (with task init and their own parameters which are connected from the task)?
I've read about pipeline_from_task but as I seen all the tasks must be already tasks which are created in the clearml server and cannot be created during runtime.

  
  
Posted 9 months ago
Votes Newest

Answers 8


@<1523701087100473344:profile|SuccessfulKoala55> What I'm trying to do is connect 3 different tasks into 1 pipeline but still being able to run each task as an individual when needed but without changing the tasks code. for example i have a training.py file which runs task.init in the start and creates a task in the server for training a new model, but i want also to create a pipeline that will run that training.py and other tasks together, is that more clear now?

  
  
Posted 9 months ago

Well, I guess you can try using Task.current_task() and checking the return value - it it's None, it should be safe to do Task.init

  
  
Posted 9 months ago

Hi @<1523701295830011904:profile|CluelessFlamingo93> , what is the exact flow you're considering?

  
  
Posted 9 months ago

@<1523701295830011904:profile|CluelessFlamingo93> I'm not sure I understand what you're trying to do - in order to run a pipeline, you need to define it somehow. You can build a pipeline that will import your existing modules and each defined step will call one of your functions - is that what you're tyring to do?

  
  
Posted 9 months ago

Hi @<1523701295830011904:profile|CluelessFlamingo93> , you can use pipelines from functions or from decorators also

  
  
Posted 9 months ago

Ok cool, I'll try that, Thanks

  
  
Posted 9 months ago

when i tried doing with the decorators it threw me an error that it cannot run task init in side a working task (the pipe lines task)

  
  
Posted 9 months ago

The flow is: Training.py (which creates and runs a training task) -> conversion_task.py (converts the outputs of the models into a format of our choosing) -> testing.py (testing the model after conversion).
I tried using the decorators and fucntions but they both threw me errors that i cannot do task init in side a running task.

  
  
Posted 9 months ago
473 Views
8 Answers
9 months ago
9 months ago
Tags
Similar posts