Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
How To Use

How to use task_overrides in PipelineController.add_step method to set latest commit in some branch?

  
  
Posted one year ago
Votes Newest

Answers 15


When I launch tasks with a pipeline, they keep complaining about missing pip packages. I run it inside a docker container, and I'm sure these packages are present inside it (when I launch the container locally, run python3 and import them, it works like charm). Any ideas how to fix this?

  
  
Posted one year ago

MelancholyElk85 can you provide some logs on the pip install fails ?

  
  
Posted one year ago

It doesn't install anything with pip during launch, I'm assuming it should take everything from the container itself (otherwise there would be a huge overhead). It simply fails trying to import things in the script

File "preprocess.py", line 4, in <module> from easydict import EasyDict as edict ModuleNotFoundError: No module named 'easydict'

  
  
Posted one year ago

I specifically set is as empty with export_data['script']['requirements'] = {} in order not to reduce overhead during launch. I have everything installed inside the container

  
  
Posted one year ago

This is odd, Can you send the full Task log? (remove any pass/user/repo that you think is sensitive)

  
  
Posted one year ago

MelancholyElk85 what do you have under "Installed Packages" for this specific Task ?

  
  
Posted one year ago

I specifically set is as empty with 

export_data['script']['requirements'] = {}

 in order not to reduce overhead during launch. I have everything installed inside the container

Do you have everything inside the container Inside a venv ?

  
  
Posted one year ago

This doesn't seem to be running inside a container...
What's the clearml-agent launch command you are using ? (i.e. do you have --docker flag)

  
  
Posted one year ago

I understand but how do you launch the cleaml-agent itself:
clearml-agent daemon --detached --queue default --docker

  
  
Posted one year ago

You're right, I re-launched the agents, now it pulls images

  
  
Posted one year ago

MelancholyElk85 , did you start the task using task.init() or did you start it through the pipeline?

  
  
Posted one year ago

👍

  
  
Posted one year ago

It worked, thank you!

  
  
Posted one year ago

I have a base task for each pipeline step. When I initialize a pipeline, for each step I clone the corresponding task, modify it and add it as a step. Tasks are launched from a pipeline, not cli. I'm absolutely sure docker argument is not empty (I specify it with export_data['container']['image'] = ' http://registry.gitlab.com/cherrylabs/ml/clearml-demo:clearml ' , and it shows on Web UI)

  
  
Posted one year ago

MelancholyElk85 , please try the following:
task_overrides={'script.branch': 'main', 'script.version_num': '', }

  
  
Posted one year ago
206 Views
15 Answers
one year ago
8 months ago
Tags