Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Pytorch Lightning Question About Logging A Figure. I Have The Following Code:

PyTorch Lightning question about logging a figure.
I have the following code:
` # turn confusion matrix into a figure (Tensor cannot be logged as a scalar)
fig = plt.figure()
plt.imshow(confmat_tensor.cpu().numpy())

log figure

self.logger.experiment.add_figure('epoch_confmat', fig, global_step=self.global_step) `This shows in the "Images" tab of TensorBoard as Confusion Matrix with a slider for the different steps.
(Based on this code: https://github.com/PyTorchLightning/pytorch-lightning/pull/4348#issuecomment-721914572 )

However, the TRAINS server (v1.6.1) page stays empty (scalars are properly logged). My Python conda env uses v1.6.3 for Trains.
Any idea how I would log figures into Trains server with PyTorch Lightning?

  
  
Posted 3 years ago
Votes Newest

Answers 18


Aah, I couldn't find it under PLOTS, but indeed it's there under DEBUG SAMPLES.

  
  
Posted 3 years ago

I mean using Trains:
Logger.current_logger().report_confusion_matrix(...)

  
  
Posted 3 years ago

AgitatedDove14 There is only a events.out.tfevents.1604567610.system.30991.0 file.
If I open this with a text editor, most is unreadable, but I do find a the letters "PNG" close to the name of the confusion matrix. So it looks like the image is encoded inside the TB log file?

  
  
Posted 3 years ago

DefeatedCrab47 yes that is correct. I actually meant if you see it on the tensorboard's UI 🙂
Anyhow if it there, you should find it in the Tasks Results Debug Samples

  
  
Posted 3 years ago

DefeatedCrab47 if TB has it as image, you should find it under "debug_samples" as image.
Can you locate it there ?

  
  
Posted 3 years ago

AgitatedDove14 TB has the confusion matrix like this:

  
  
Posted 3 years ago

So if I want it under plots, I would need to call e.g. report_confusion_matrix right?

  
  
Posted 3 years ago

Correct 🙂

  
  
Posted 3 years ago

That's useful to know! But actually in this case I want to just test if the code works (run 2 epochs and see if it works). I don't want this to be logged, so I don't Task.init in those cases.
I don't want the code to crash on Trains in those cases.

I see that Task.current_task() returns None if no task is running, so I can use that with an if statement 🙂

  
  
Posted 3 years ago

With offline mode,
Later if you need you can actually import the execution (including artifacts etc.) you just need the zip file it creates when you are done.

  
  
Posted 3 years ago

Yey 🙂

  
  
Posted 3 years ago

The reason is because it is logged as an image, not a plot 🙂

  
  
Posted 3 years ago

With PyTorch Lightning, I only use this line at the beginning of a Jup Notebook:
Task.init(project_name=project_name, task_name=task_name)The code to log the confusion matrix is in some .py file though that does not have any Trains code.

Is it possible to log it in a TB compatible way, that will be automatically picked up by Trains? I prefer to keep the .py Trains free.

  
  
Posted 3 years ago

And you cannot see it in Trains UI?

  
  
Posted 3 years ago

I'm not sure TB support confusion matrix regardless, from anywhere in your code you can do:from trains import Task Task.current_task().get_logger().report_confusion_matrix(...)

  
  
Posted 3 years ago

Based on your code snippet:
Logger.current_logger().report_confusion_matrix(title='confusion', series=confusion', value=confmat_tensor.cpu().numpy(), iteration=i)or Task.current_task().get_logger()
which is the same as Logger.current_logger()

  
  
Posted 3 years ago

Good news, there is an offline mode.
Task.set_offline(True)
If you want your code to be aware, you can do:
from trains import Task if Task.current_task(): Task.current_task().get_logger().report_confusion_matrix(...)

  
  
Posted 3 years ago

I have a numpy array, but I indeed didn't see a TB way of doing it. I guess that's not really an issue to add. The code should also be usable without Trains. How should I test if there is a current task? (I need a VPN on to log to TRAINS, which can be annoying for small tests)

  
  
Posted 3 years ago