Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
How To Use

How to use task_overrides in PipelineController.add_step method to set latest commit in some branch?

  
  
Posted 2 years ago
Votes Newest

Answers 15


MelancholyElk85 can you provide some logs on the pip install fails ?

  
  
Posted 2 years ago

When I launch tasks with a pipeline, they keep complaining about missing pip packages. I run it inside a docker container, and I'm sure these packages are present inside it (when I launch the container locally, run python3 and import them, it works like charm). Any ideas how to fix this?

  
  
Posted 2 years ago

It doesn't install anything with pip during launch, I'm assuming it should take everything from the container itself (otherwise there would be a huge overhead). It simply fails trying to import things in the script

File "preprocess.py", line 4, in <module> from easydict import EasyDict as edict ModuleNotFoundError: No module named 'easydict'

  
  
Posted 2 years ago

MelancholyElk85 , did you start the task using task.init() or did you start it through the pipeline?

  
  
Posted 2 years ago

It worked, thank you!

  
  
Posted 2 years ago

MelancholyElk85 , please try the following:
task_overrides={'script.branch': 'main', 'script.version_num': '', }

  
  
Posted 2 years ago

MelancholyElk85 what do you have under "Installed Packages" for this specific Task ?

  
  
Posted 2 years ago

This doesn't seem to be running inside a container...
What's the clearml-agent launch command you are using ? (i.e. do you have --docker flag)

  
  
Posted 2 years ago

I understand but how do you launch the cleaml-agent itself:
clearml-agent daemon --detached --queue default --docker

  
  
Posted 2 years ago

You're right, I re-launched the agents, now it pulls images

  
  
Posted 2 years ago

👍

  
  
Posted 2 years ago

I specifically set is as empty with export_data['script']['requirements'] = {} in order not to reduce overhead during launch. I have everything installed inside the container

  
  
Posted 2 years ago

This is odd, Can you send the full Task log? (remove any pass/user/repo that you think is sensitive)

  
  
Posted 2 years ago

I have a base task for each pipeline step. When I initialize a pipeline, for each step I clone the corresponding task, modify it and add it as a step. Tasks are launched from a pipeline, not cli. I'm absolutely sure docker argument is not empty (I specify it with export_data['container']['image'] = ' http://registry.gitlab.com/cherrylabs/ml/clearml-demo:clearml ' , and it shows on Web UI)

  
  
Posted 2 years ago

I specifically set is as empty with 

export_data['script']['requirements'] = {}

 in order not to reduce overhead during launch. I have everything installed inside the container

Do you have everything inside the container Inside a venv ?

  
  
Posted 2 years ago
496 Views
15 Answers
2 years ago
one year ago
Tags