HappyDove3 Hi 🙂
Well, since all the data is logged you can simply use the API to retrieve it and create the tables yourself quite easily!
I thought I'll have a way to compare my models accuracy, precision, recall etc through the web UI
You can do it by comparing experiments, what is your use case? I think I might be missing something. Can you please elaborate?
I started to explore clearml as an experiment tracker and I run a simple example of sklearn randomForest. I want to log the accuracy score to compare few experiments based on this index and I'm not sure how to log it.
This is a simple use case not a scalar and iterations.
The webUI won't calculate anything for you, so if you want to see accuracy, precision etc. you'll have to calculate those in your code and then report them as scalers for the UI to know what they are.
As you said, sklearn models don't work with iterations, so when using the logger to report the scalar, you can just set the iteration nr to 0.
The downside of that is that your scalars will still be plotted, only as a single point in the graph instead of in a table. This is a known open issue for now ( https://github.com/allegroai/clearml/issues/400 ), but would be cool if you could build it and contribute it yourself if you can 😄
As you can see in the issue comments, for now you can report it using iteration=0 and then adding the value as a field in the experiment overview (as leaderboard). This will give you a quick overview of your metrics per experiment in the main experiment list 🙂
Thanks! I'll try runing it with iteration=0 and then adding the value as a field in the experiment overview.
If I'll have time I will take a look 🙂 https://github.com/allegroai/clearml/issues/400
Seems like a quiet important feature for basic matrices comparison no ?
I agree, I came across the same issue too. But your post helps make it clear, so hopefully it can be pushed! 🙂
HappyDove3 Notice that in https://github.com/allegroai/clearml/issues/400 the goal is to see a table plot in the UI scalars tab for a specific experiment (with additional discussions on how these will be addressed when comparing experiments).
Note that once you take the approach you suggested of logging your metrics single values, you can configure your experiment comparison scalars view to show single values instead of the time-series graph which I think will provide you with the matrix comparison you're looking for. Does that work?
