Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi Everyone! I’M Building Notification\Cleaning Service On Top Of Clearml Saved Data. I Use

Hi everyone!
I’m building notification\cleaning service on top of ClearML saved data. I use https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py as an inspiration. And I wonder - is there any way to programatically get the size(storage consumption) for any Tasks metrics\artifacts\models(together or separately). Because I want to notify(and then remove) only some really large Tasks, and don’t touch those that don’t require a lot of space.
I’ve browsed through the history but failed to find something related to the topic..

Posted one year ago
Votes Newest

Answers 3

Well, you'll need to be familiar with the ES documents structure, and use the _size https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-size-usage.html (which would probably require you to reindex the data). That's for ES.
For MongoDB, you'll need to use the $bsonSize https://www.mongodb.com/docs/upcoming/reference/operator/aggregation/bsonSize/ on the task documents.

Posted one year ago

Thank you for fast reply. Could you suggest any opensourced examples even for such maybe a little bit complicated pipeline?

Posted one year ago

Hi GreasyRaven35 , this actually takes some more infrastructure, and can only be done using ES plugins and specific MongoDB queries

Posted one year ago