Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, Is It Possible To Query All Experiments In A Project And Get The Best Performing One (Sorted By One Metric)? Something Similar As Search_Runs In Mlflow (

Hi, is it possible to query all experiments in a project and get the best performing one (sorted by one metric)? Something similar as search_runs in mlflow ( https://www.mlflow.org/docs/latest/python_api/mlflow.tracking.html )

  
  
Posted 3 years ago
Votes Newest

Answers 11


Still feels super hacky tho, think it would be nice to have a simplier way or atleast some nice documentation 

YES you are absolutely correct, we should add it to the Task interface.
Any chance you add a GitHub issue so we do not forget ?

  
  
Posted 3 years ago

Is this still the best way? Is not working for me 😞

  
  
Posted 3 years ago

Hi GreasyPenguin14
Sure you can, although a bit convoluted (I'll make sure we have a nice interface 🙂 )
import hashlib title = hashlib.md5('epoch_accuracy_title'.encode('utf-8')).hexdigest() series = hashlib.md5('epoch_accuracy_series'.encode('utf-8')).hexdigest() task_filter = { 'page_size': 2, 'page': 0, 'order_by': ['last_metrics.{}.{}'.format(title, series)] } queried_tasks = Task.get_tasks(project_name='examples', task_filter=task_filter)

  
  
Posted 3 years ago

👍

  
  
Posted 3 years ago

Still feels super hacky tho, think it would be nice to have a simplier way or atleast some nice documentation 🙂
Thanks for your help!

  
  
Posted 3 years ago

nice, will give it a try

  
  
Posted 3 years ago

Well I figured it out, thats what your - was for :D

  
  
Posted 3 years ago

I'll try to test in a new project. Will they get ordered ascending or descending? Does documentation beside your example exist?

  
  
Posted 3 years ago

Will they get ordered ascending or descending?

Good point, I'll check the docs... but I think they do not specify
https://clear.ml/docs/latst/docs/references/sdk/task#taskget_tasks
From the code it seems the ordered is not guaranteed.
You can however pass '-last_update' : order_by which will give you the latest updated first
task_filter = { 'page_size': 2, 'page': 0, 'order_by': ['last_metrics.{}.{}'.format(title, series), '-last_update'] } Task.get_tasks(project_name='examples', task_filter=task_filter)

  
  
Posted 3 years ago

But now you're talking about the case two Tasks have the same metric, right?
I mean in general, whether the task with the largest metric is first, or smallest, because I'd need largest metric, but to me it seems like smallest metric is first. Can the order be inversed?

  
  
Posted 3 years ago

NastyOtter17 can you provide some more info ?

  
  
Posted 3 years ago
1K Views
11 Answers
3 years ago
one year ago
Tags