AgitatedDove14 I'm looking for something else. Basically, I want to compare the accuracy with respect to another variable. I can get the accuracy of each experiment (see picture). Additionally, I can log the other variable (name it T) and show it as a scalar per experiment. However, I don't see an option How I can simply plot the graph of accuracy versus T, which is different for every experiment. The solution that loops thru all tasks might work anyway.
SoreDragonfly16 . In the hyper parameters Tab, you have "parallel coordinates" (next to the "add experiment" the button saying "values" press on it and there should be " parallel coordinates")
Is that it?
because comparing experiments using graphs is very useful. I think it is a nice to have feature.
So currently when you compare the graphs you can select the specific scalars to compare, and it Update in Real Time!
You can also bookmark the actual URL and it is fully reproducible (i.e. full state is stored)
You can also add custom columns to the experiment table (with the metrics) and sort / filter based on them, and create a summary dashboard (again like ll pages in the web app, URL is fully reproducible, so you can bookmark this dashboard)
SoreDragonfly16 wdyt ?
SoreDragonfly16 as SmallDeer34 mentioned, you can iterate over the Tasks, pull metrics (with either task.get_last_scalar_metrics or task.get_reported_scalar ) then report them on the Task that runs the Loop itself with the Logger.
wdyt?
So presumably you could write a Python loop that goes through and pulls the metrics into a list, then make a plot locally. Not sure about creating a Dashboard within the ClearML web interface though!
AgitatedDove14 That would work ofcourse. I wonder what is the best practice to do such things because comparing experiments using graphs is very useful. I think it is a nice to have feature.
This discussion might be relevant, it shows how to query a Task for metrics in code: https://clearml.slack.com/archives/CTK20V944/p1626992991375500?thread_ts=1626981377.374400&cid=CTK20V944