/home/ubuntu/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/utilities/plotlympl/mpltools.py:371: MatplotlibDeprecationWarning: The is_frame_like function was deprecated in Matplotlib 3.1 and will be removed in 3.3.
This is the last print statement before it hangs
I dont think its that. its a 20kb file upload. This was the last message just printedClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-star
it just hangs when trying to upload. maybe that is the reason that the plots are not logging?
I removed it and I still get the same error 😞
No I mean configure the files_server
in the clearml.conf
could it be how I am trying to log the figure manually?
So do I set the report_image to False for the plots to appear in the plots tab?
If uploaded as image, what is the target destination for logging?
I will try add some print statements to test the hanging issue
The task is dependent on a few artefacts from another task. Is there anything else I can do here?
Hmm could you try to upload to your files server (not the S3)
Maybe some credentials error ?
Hangs there ? could it be that it's uploading slowly ?
Can you check the network ?
Can you try to run the example code, see if that works for you?
Just upgraded matplotlib, going to test now
its a seaborn heatmap that needs to be plotted. not sure if that is useful at all
2021-03-01 20:51:55,655 - clearml.Task - INFO - Completed model upload to s3://15gifts-clearml/artefacts/pre-engine-traits/logistic-regression-paths-and-sales-tfidf-device-brand.8d68e9a649824affb9a9edf7bfbe157d/models/tfidf-logistic-regression-1614631915-8d68e9a649824affb9a9edf7bfbe157d.pkl *****
2021-03-01 20:52:01
2021-03-01 20:51:57,207 - clearml.Task - INFO - Waiting to finish uploads
Could you see if that makes a difference ?
Actually unless you specifically detached the matplotlib automagic, any plt.show() will be automatically reported.
Hi TenseOstrich47 whats the matplotlib version and clearml version you are using ?
TenseOstrich47 notice:task.logger.report_matplotlib_figure( title=f"Performance Heatmap - {name}", series="Device Brand Predictions", iteration=0, figure=figure, **report_image=True,** )
report_image=True means it will be uploaded as an image not a plot (like imshow), the default is False , which would put it under Plots section
Code you add a few prints, and see where it hangs ? there's no reason for it to hang (even the plot upload is done in the background)
` # Plot the confusion matrix for predictions
sns.heatmap(
preds_confusion_percentage, annot=True, fmt=".3f", linewidths=.5,
square=True, cmap='Blues_r'
)
plt.ylabel('Actual label')
plt.xlabel('Predicted label')
title_str = f'Accuracy Score: {round(score, 2)}\n{TRANSFORM_TYPE}'
plt.title(title_str, size=15)
task.logger.report_matplotlib_figure(
title=f"Performance Heatmap - {model_export_name}",
series="Device Brand Predictions",
iteration=0,
figure=plt,
report_image=False,
)
plt.show() `
Seems like it is working (including seaborn)
Yes this seems like it is stuck, could you test with the demo server ?
(basically remove the clearml.conf it will connect automatically)
(with matplotlib 3.2+ I get no warning, let me check with 3.1)