Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hello, I Having Been Exhausting The Metrics Quota Way To Fast For The Current Use That I Am Making Of Clearml. Is The Quota Cumulative ? I.E. Do We Get 1G Per Month ? I Am Concerned Because If We Upgrade And Need To Pay

I having been exhausting the metrics quota way to fast for the current use that I am making of clearml. Is the quota cumulative ? i.e. do we get 1G per month ? I am concerned because if we upgrade and need to pay $0.01/1MB Metric Events , the pricing will go up quickly.

Is it possible to use our own S3 storage for those ?

Posted one month ago
Votes Newest

Answers 8

Hey @<1644147961996775424:profile|HurtStarfish47> , you can use S3 for debug images specifically , see here: https://clear.ml/docs/latest/docs/references/sdk/logger/#set_default_upload_destination but the metrics (everything you report like scalars, single values, histograms, and other plots) are stored in the backend. The fact that you are almost running out of storage could be because of either too many experiments (in which case consider cleaning up your projects) or excessive reporting , in which case check that your code is not overly verbose

Posted one month ago

Thanks, that is exactly the kind of info I was looking for ! If debug images are counting in the metrics quota that would explain how we reached the limit so quickly.

Posted one month ago

And the quota is not cumulative , otherwise we’d run out of storage with the oldest accounts 😃

Posted one month ago

@<1537605940121964544:profile|EnthusiasticShrimp49> A follow up question about metrics - My pytorch (lightning) experiments are logging to tensorboard and ClearML is automatically picking this up and uploading scalars and debug_images. If I use the set_default_upload_destination that you mentionned, would that still properly use my URI even though I am not calling Logger.current_logger().report_image directly ?

Also, I reset than deleted ~80% of the experiment that I had 2 days ago and my metric quota hasn't gone down a MB yet, is it supposed to take longer to update ?

Posted one month ago

I made sure to delete them from the archived tab

Posted one month ago

@<1644147961996775424:profile|HurtStarfish47> did you see any result for the delete operation? Did you get any error or similar?

Posted one month ago

No ! The way I delete those is like so:
Experiment view -> Reset (one or more) experiment -> expriment is now in draft
Archive experiment
Open archive -> Delete

I get no feedback at all from the operation, but I can see the experiments are no longer available on clearml

Posted one month ago

About the first question - yes, it will use the destination URI you set.

About the second point - did you archive or properly delete the experiments?

Posted one month ago
8 Answers
one month ago
one month ago