Further to this, I have inspected further. This is working as expected for ClearML 1.8.3 but not for ClearML 1.9.0.
I looked at the commits and found that a change had been made to the
This aligns with the error message I'm seeing:
2023-02-08 15:17:25,539 - clearml - WARNING - Error: I/O operation on closed file.
Can this be actioned for the next release please?
Or is it suitable for me to raise my own PR?
The code is quite nested by I've tried to extract out the important parts (
summmary_writer is a tensorboard logger).
self.figure, (ax1, ax2, axc) = plt.subplots(1, 3, figsize=(total_width, total_height), facecolor="white") self.summary_writer = self.tb_logger.experiment self.summary_writer.add_figure(Partition.TRAINING.value, train_plot.figure, global_step=self.current_epoch + 1)
train_plot.figure is a matplotlib figure created using seaborn.
Let me know if this isn't useful and you'd like me to create a proper example.
I am using
tensorboard==2.11.2 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1