Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All. I Am Struggling With Integrating Plots Into My Task. Without The Plotting Code, The Task Never Completes The Execution And Seems To Hang. Also, The Plots Are Not Visible In The Plots Tab. I Am Running A For Loop For Different Models And Attemptin

Hi all. I am struggling with integrating plots into my task. Without the plotting code, the task never completes the execution and seems to hang. Also, the plots are not visible in the plots tab.

I am running a for loop for different models and attempting to export the plots for each iteration.

Any ideas on why this is happening?

  
  
Posted 3 years ago
Votes Newest

Answers 30


Hmm could you try to upload to your files server (not the S3)
Maybe some credentials error ?

  
  
Posted 3 years ago

` # Plot the confusion matrix for predictions
sns.heatmap(
preds_confusion_percentage, annot=True, fmt=".3f", linewidths=.5,
square=True, cmap='Blues_r'
)
plt.ylabel('Actual label')
plt.xlabel('Predicted label')
title_str = f'Accuracy Score: {round(score, 2)}\n{TRANSFORM_TYPE}'
plt.title(title_str, size=15)

task.logger.report_matplotlib_figure(
title=f"Performance Heatmap - {model_export_name}",
series="Device Brand Predictions",
iteration=0,
figure=plt,
report_image=False,
)
plt.show() `

  
  
Posted 3 years ago

Seems like it is working (including seaborn)

  
  
Posted 3 years ago

TenseOstrich47 notice:
task.logger.report_matplotlib_figure( title=f"Performance Heatmap - {name}", series="Device Brand Predictions", iteration=0, figure=figure, **report_image=True,** )report_image=True means it will be uploaded as an image not a plot (like imshow), the default is False , which would put it under Plots section

Code you add a few prints, and see where it hangs ? there's no reason for it to hang (even the plot upload is done in the background)

  
  
Posted 3 years ago

and clearml version ?

  
  
Posted 3 years ago

matplotlib == 3.1.3

  
  
Posted 3 years ago

0.17.5rc5

  
  
Posted 3 years ago

2021-03-01 20:51:55,655 - clearml.Task - INFO - Completed model upload to s3://15gifts-clearml/artefacts/pre-engine-traits/logistic-regression-paths-and-sales-tfidf-device-brand.8d68e9a649824affb9a9edf7bfbe157d/models/tfidf-logistic-regression-1614631915-8d68e9a649824affb9a9edf7bfbe157d.pkl *****
2021-03-01 20:52:01
2021-03-01 20:51:57,207 - clearml.Task - INFO - Waiting to finish uploads

  
  
Posted 3 years ago

Yes this seems like it is stuck, could you test with the demo server ?
(basically remove the clearml.conf it will connect automatically)

  
  
Posted 3 years ago

it just hangs when trying to upload. maybe that is the reason that the plots are not logging?

  
  
Posted 3 years ago

Can you try to run the example code, see if that works for you?

  
  
Posted 3 years ago

hmmm How do I reproduce it ?

  
  
Posted 3 years ago

No I mean configure the files_server in the clearml.conf

  
  
Posted 3 years ago

Upload how? Via SSH?

  
  
Posted 3 years ago

reproduced with matplotlib 3.1

  
  
Posted 3 years ago

(with matplotlib 3.2+ I get no warning, let me check with 3.1)

  
  
Posted 3 years ago

I removed it and I still get the same error 😞

  
  
Posted 3 years ago

Hi TenseOstrich47 whats the matplotlib version and clearml version you are using ?

  
  
Posted 3 years ago

Just upgraded matplotlib, going to test now

  
  
Posted 3 years ago

The task is dependent on a few artefacts from another task. Is there anything else I can do here?

  
  
Posted 3 years ago

its a seaborn heatmap that needs to be plotted. not sure if that is useful at all

  
  
Posted 3 years ago

Hangs there ? could it be that it's uploading slowly ?
Can you check the network ?

  
  
Posted 3 years ago

Actually unless you specifically detached the matplotlib automagic, any plt.show() will be automatically reported.

  
  
Posted 3 years ago

could it be how I am trying to log the figure manually?

  
  
Posted 3 years ago

Could you see if that makes a difference ?

  
  
Posted 3 years ago

I dont think its that. its a 20kb file upload. This was the last message just printed
ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-star

  
  
Posted 3 years ago

/home/ubuntu/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/utilities/plotlympl/mpltools.py:371: MatplotlibDeprecationWarning: The is_frame_like function was deprecated in Matplotlib 3.1 and will be removed in 3.3.This is the last print statement before it hangs

  
  
Posted 3 years ago

So do I set the report_image to False for the plots to appear in the plots tab?
If uploaded as image, what is the target destination for logging?

I will try add some print statements to test the hanging issue

  
  
Posted 3 years ago
1K Views
30 Answers
3 years ago
one year ago
Tags