/home/ubuntu/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/utilities/plotlympl/mpltools.py:371: MatplotlibDeprecationWarning: The is_frame_like function was deprecated in Matplotlib 3.1 and will be removed in 3.3.
This is the last print statement before it hangs
I dont think its that. its a 20kb file upload. This was the last message just printedClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-star
it just hangs when trying to upload. maybe that is the reason that the plots are not logging?
I removed it and I still get the same error 😞
No I mean configure the files_server
in the clearml.conf
could it be how I am trying to log the figure manually?
(with matplotlib 3.2+ I get no warning, let me check with 3.1)
So do I set the report_image to False for the plots to appear in the plots tab?
If uploaded as image, what is the target destination for logging?
I will try add some print statements to test the hanging issue
Yes this seems like it is stuck, could you test with the demo server ?
(basically remove the clearml.conf it will connect automatically)
The task is dependent on a few artefacts from another task. Is there anything else I can do here?
Hmm could you try to upload to your files server (not the S3)
Maybe some credentials error ?
Hangs there ? could it be that it's uploading slowly ?
Can you check the network ?
TenseOstrich47 notice:task.logger.report_matplotlib_figure( title=f"Performance Heatmap - {name}", series="Device Brand Predictions", iteration=0, figure=figure, **report_image=True,** )
report_image=True means it will be uploaded as an image not a plot (like imshow), the default is False , which would put it under Plots section
Code you add a few prints, and see where it hangs ? there's no reason for it to hang (even the plot upload is done in the background)
Can you try to run the example code, see if that works for you?
Just upgraded matplotlib, going to test now
` # Plot the confusion matrix for predictions
sns.heatmap(
preds_confusion_percentage, annot=True, fmt=".3f", linewidths=.5,
square=True, cmap='Blues_r'
)
plt.ylabel('Actual label')
plt.xlabel('Predicted label')
title_str = f'Accuracy Score: {round(score, 2)}\n{TRANSFORM_TYPE}'
plt.title(title_str, size=15)
task.logger.report_matplotlib_figure(
title=f"Performance Heatmap - {model_export_name}",
series="Device Brand Predictions",
iteration=0,
figure=plt,
report_image=False,
)
plt.show() `
its a seaborn heatmap that needs to be plotted. not sure if that is useful at all
Seems like it is working (including seaborn)
2021-03-01 20:51:55,655 - clearml.Task - INFO - Completed model upload to s3://15gifts-clearml/artefacts/pre-engine-traits/logistic-regression-paths-and-sales-tfidf-device-brand.8d68e9a649824affb9a9edf7bfbe157d/models/tfidf-logistic-regression-1614631915-8d68e9a649824affb9a9edf7bfbe157d.pkl *****
2021-03-01 20:52:01
2021-03-01 20:51:57,207 - clearml.Task - INFO - Waiting to finish uploads
Could you see if that makes a difference ?
Actually unless you specifically detached the matplotlib automagic, any plt.show() will be automatically reported.
Hi TenseOstrich47 whats the matplotlib version and clearml version you are using ?