Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey - I'M Trying To Compare Voxel Versus Clear Ml In Image Data Exploration.

Hey - I'm trying to compare voxel versus clear ml in image data exploration.
Is there a way in clear ml to interactively view subsets of images based on a lasso of embedding plots? lik: is there a way in clear ml to calculate out metrics like recall / precision ...etc for multi class is there a way in clear ml to plot a confusion matrix, click on a grid cell and see only the images that are relevant to that grid cell?

  
  
Posted one year ago
Votes Newest

Answers 7


Hi @<1523701111020589056:profile|DefiantSpider5>
So there are two answers here, I'll start with the open-source version of both

Is there a way in clear ml to interactively view subsets of images based on a lasso of embedding plots

The ClearML Datasets have no "query" capabilities of the data inside the dataset. That means you can see preview images, statistics and download the datasets, but no query capabilities. On the other hand, there is no limitation on the type and format of metadata, and no limitation on the formatting options for both raw data and metadata in your dataset.
The ClearML HyperDataset (paid version) has these query capabilities as well as differentiable storage on top of the metadata, raw/meta data separation etc. Unfortunately I do not think it has this specific forensics capabilities of "lasso-ing" you mentioned. I would contact sales for a more updated answer on what they do support these days.

is there a way in clear ml to calculate out metrics like recall / precision ...etc for multi class

When you create the dataset itself you can store it ontop of the dataset as part of the dateset creation tool, and this will be part of the dataset catalog. That said it is not calculated automatically as clearml is not aware of what is considered "match" in your formatting so it cannot calculate it...

is there a way in clear ml to plot a confusion matrix, click on a grid cell and see only the images that are relevant to that grid cell?

I do not think so 😞

From your question I would suggest using CleaML as a data catalog for results of your experiments (different version for each experiment), then use voxel to do live forensics on the results. This achieves both scale as the clearml dataset versioning will only store the changes, i.e. the metadata files, and the easy with the help of the CLI you will be able to easily (i.e. everything is cached) pull the results on-top your machine to further analyse with the a local run of voxel. wdyt ?

  
  
Posted one year ago

@<1523701205467926528:profile|AgitatedDove14> thanks!
yeah. i talked to their sales rep and he swore that they have feature parity with voxel, but i couldn't find anything that resembles what you refer to as forensics.

  
  
Posted one year ago

I'm hoping i can find an end to end solution that also includes experiment management

  
  
Posted one year ago

thanks

  
  
Posted one year ago

I'm hoping i can find an end to end solution that also includes experiment management

Well of course biased here, but ClearML with the hyperdatasets is probably the most complete one.
Specifically with model performance analysis I would add voxel open-source to dissect specific results. but the combination of the abstraction and query capabilities of hyperdatasets, orchestration and experiment management are really unmatched for.
(and again of course I'm biased, but really there is no tool that can combine data+experiment management+orchestration on any compared level, feel free to correct me if you have thoughts )

  
  
Posted one year ago

computer vision tasks in agriculture

  
  
Posted one year ago

Yeah I think using voxel for forensics makes sense. What's your use case ?

  
  
Posted one year ago
608 Views
7 Answers
one year ago
one year ago
Tags