Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I’M Using Catboost For Training, But Sadly It Does Not Have A Native Integration With Clearml (Xgboost And Lightgbm Do Have Integrations). But Catboost Writes Down Training Logs In Tensorboard Format (Into A

Iā€™m using catboost for training, but sadly it does not have a native integration with clearml (xgboost and lightgbm do have integrations). But catboost writes down training logs in tensorboard format (into a .tfevents file). How can I integrate this file into clearml?

  
  
Posted 3 years ago
Votes Newest

Answers 15


Yes, but as you mentioned everything is created inside the lib, which means the python is not able to intercept the metrics so that clearml can send them to the backend.

  
  
Posted 3 years ago

as I understand, it uses tensorboard from C++ code

  
  
Posted 3 years ago

Hi FiercePenguin76
Is catboost actually using TB or is it just writing to .tfevent on its own ?

  
  
Posted 3 years ago

Yep šŸ˜ž

  
  
Posted 3 years ago

it certainly does not use tensorboard python lib

Hmm, yes I assume this is why the automagic is not working šŸ˜ž

Does it have a pythonic interface form the metrics ?

  
  
Posted 3 years ago

Hmm I think everything is generated inside the c++ library code, and python is just an external interface. That means there is no was to collect the metrics as they are created (i.e. inside the c++ code), which means the only was to collect them is to actively analyze/read the tfrecord created by catboost šŸ˜ž
Is there a python code that does that (reads the tfrecords it creates) ?

  
  
Posted 3 years ago

it certainly does not use tensorboard python lib

  
  
Posted 3 years ago

Although it is only for model tracking, autologging is yet to be implemented there

  
  
Posted 3 years ago

looking into the output folder of catboost, I see 3 types of metrics outputs:
tfevents (can be read by tensorboard) catboost_training.json (custom (?) format). Is read here to be shown as an ipython widget: https://github.com/catboost/catboost/blob/c2a6ed0cb85869a73a13d08bf8df8d17320f8215/catboost/python-package/catboost/widget/ipythonwidget.py#L93 learn_error.tsv, test_error.tsv, time_left.tsv which have the same data as json. Apparently they are to be used with this stale metrics viewer project: https://github.com/catboost/catboost-viewer

  
  
Posted 3 years ago

nope, catboost docs offer to manually run tensorboard against the output folder https://catboost.ai/docs/features/visualization_tensorboard.html

  
  
Posted 3 years ago

Wanted to check if MLFlow supports catboost. Apparently, it does. Pull request was merged 16 hours ago. Nice timing šŸ˜ƒ

  
  
Posted 3 years ago

Actually that is less interesting, as it is quite straight forward

  
  
Posted 3 years ago
780 Views
15 Answers
3 years ago
one year ago
Tags