Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi Everyone! I’M Building Notification\Cleaning Service On Top Of Clearml Saved Data. I Use

Hi everyone!
I’m building notification\cleaning service on top of ClearML saved data. I use https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py as an inspiration. And I wonder - is there any way to programatically get the size(storage consumption) for any Tasks metrics\artifacts\models(together or separately). Because I want to notify(and then remove) only some really large Tasks, and don’t touch those that don’t require a lot of space.
I’ve browsed through the history but failed to find something related to the topic..

Posted one year ago
Votes Newest

Answers 3

Well, you'll need to be familiar with the ES documents structure, and use the _size https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-size-usage.html (which would probably require you to reindex the data). That's for ES.
For MongoDB, you'll need to use the $bsonSize https://www.mongodb.com/docs/upcoming/reference/operator/aggregation/bsonSize/ on the task documents.

Posted one year ago

Hi GreasyRaven35 , this actually takes some more infrastructure, and can only be done using ES plugins and specific MongoDB queries

Posted one year ago

Thank you for fast reply. Could you suggest any opensourced examples even for such maybe a little bit complicated pipeline?

Posted one year ago