Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! We Use Clearml Self-Hosted On A K8S Cluster. Great Work Btw

Hi!

We use ClearML self-hosted on a k8s cluster. Great work btw 😉

We are currently experiencing a problem with the debug samples of tasks. They are not shown when going to the tab in the detail view of the task. In the logs of the Apiserver we see that there is an AuthorizationException 403 from Elasticsearch. My question is not regarding the exception itself, I think we can work this out but more on the general coupling between ClearML and Elasticsearch.
Why is the debug sample at all coupled with Elasticsearch? For me it is some sort of file I would have suspected to be stored on the Fileserver and things like the console output of a task would be stored on Elasticsearch. Why can then problems with Elasticsearch lead to problems with showing debug samples?

And if there is this tight connection to Elasticsearch: Would you recommend to have a dedicated Elasticsearch instance for ClearML? We are currently using a more global Elasticsearch instance where other systems are also logging to and problems there may directly impact ClearML if I see it right?

  
  
Posted 2 months ago
Votes Newest

Answers 2


@<1523701070390366208:profile|CostlyOstrich36> Thanks for the clarification 👍

  
  
Posted 2 months ago

Hi @<1649221394904387584:profile|RattySparrow90> , events and console logs are logged to elastic so they can be fetched. Debug samples are also events so they are saved to elastic (A link to the debug is saved in the event itself).

It is suggested to keep a dedicated elastic to ClearML

  
  
Posted 2 months ago
150 Views
2 Answers
2 months ago
2 months ago
Tags