Hangs there ? could it be that it's uploading slowly ?
Can you check the network ?
/home/ubuntu/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/utilities/plotlympl/mpltools.py:371: MatplotlibDeprecationWarning: The is_frame_like function was deprecated in Matplotlib 3.1 and will be removed in 3.3.This is the last print statement before it hangs
Seems like it is working (including seaborn)
So do I set the report_image to False for the plots to appear in the plots tab?
If uploaded as image, what is the target destination for logging?
I will try add some print statements to test the hanging issue
` # Plot the confusion matrix for predictions
sns.heatmap(
preds_confusion_percentage, annot=True, fmt=".3f", linewidths=.5,
square=True, cmap='Blues_r'
)
plt.ylabel('Actual label')
plt.xlabel('Predicted label')
title_str = f'Accuracy Score: {round(score, 2)}\n{TRANSFORM_TYPE}'
plt.title(title_str, size=15)
task.logger.report_matplotlib_figure(
title=f"Performance Heatmap - {model_export_name}",
series="Device Brand Predictions",
iteration=0,
figure=plt,
report_image=False,
)
plt.show() `
Yes this seems like it is stuck, could you test with the demo server ?
(basically remove the clearml.conf it will connect automatically)
Hi TenseOstrich47 whats the matplotlib version and clearml version you are using ?
(with matplotlib 3.2+ I get no warning, let me check with 3.1)
TenseOstrich47 notice:task.logger.report_matplotlib_figure( title=f"Performance Heatmap - {name}", series="Device Brand Predictions", iteration=0, figure=figure, **report_image=True,** )report_image=True means it will be uploaded as an image not a plot (like imshow), the default is False , which would put it under Plots section
Code you add a few prints, and see where it hangs ? there's no reason for it to hang (even the plot upload is done in the background)
I dont think its that. its a 20kb file upload. This was the last message just printedClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-star
Could you see if that makes a difference ?
Just upgraded matplotlib, going to test now
2021-03-01 20:51:55,655 - clearml.Task - INFO - Completed model upload to s3://15gifts-clearml/artefacts/pre-engine-traits/logistic-regression-paths-and-sales-tfidf-device-brand.8d68e9a649824affb9a9edf7bfbe157d/models/tfidf-logistic-regression-1614631915-8d68e9a649824affb9a9edf7bfbe157d.pkl *****
2021-03-01 20:52:01
2021-03-01 20:51:57,207 - clearml.Task - INFO - Waiting to finish uploads
its a seaborn heatmap that needs to be plotted. not sure if that is useful at all
Hmm could you try to upload to your files server (not the S3)
Maybe some credentials error ?
I removed it and I still get the same error 😞
Actually unless you specifically detached the matplotlib automagic, any plt.show() will be automatically reported.
No I mean configure the files_server in the clearml.conf
Can you try to run the example code, see if that works for you?
could it be how I am trying to log the figure manually?
it just hangs when trying to upload. maybe that is the reason that the plots are not logging?
The task is dependent on a few artefacts from another task. Is there anything else I can do here?