Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Everyone, I Have An Video Classification Model. I Have A Lot Of Metadata About The Videos In A Csv File, I First Preprocess The Metadata And In Pandas Df. (There May Be Lost Videos, Invalid Videos Etc, So I Remove Those From The Df). I Want To Log T

Hello Everyone, I have an video classification model. I have a lot of metadata about the videos in a csv file, I first preprocess the metadata and in pandas df. (There may be lost videos, invalid videos etc, so I remove those from the df). I want to log the last clean df statistics into clearml (Something like StatisticsGen of TFX). How can I log my data statistics?

Thank you in advance.

  
  
Posted one month ago
Votes Newest

Answers 19


I also use this: None
Which can give more control

  
  
Posted one month ago

None

  
  
Posted one month ago

you can upload the df as artifact.
Or the statistics as a DataFrame and upload as artifact ?

  
  
Posted one month ago

if you want plot, you can simply generate plot with matplotlib and clearml can upload them in the Plot or Debug Sample section

  
  
Posted one month ago

Pefect, thanks. I will take a look at that.

  
  
Posted one month ago

report_table seems to be the most straight forward without matplotlib integration. Do you think it has plotting features?

  
  
Posted one month ago

because, the param name that takes in the df is table_plot

  
  
Posted one month ago

with

    df = pd.DataFrame({'num_legs': [2, 4, 8, 0],
                       'num_wings': [2, 0, 0, 0],
                       'num_specimen_seen': [10, 2, 1, 8]},
                      index=['falcon', 'dog', 'spider', 'fish'])

    import clearml
    task = clearml.Task.current_task()
    task.get_logger().report_table(title='table example', series='pandas DataFrame', iteration=0, table_plot=df)
    # logger.report_table(title='table example',series='pandas DataFrame',iteration=0,table_plot=df)
    task.close()

image

  
  
Posted one month ago

never tried 😛

  
  
Posted one month ago

No worries. Thank you so much.

  
  
Posted one month ago

I mean, depend on what do you want to report ... if you want to stick to table, I suggest earlier to gather your stats in table format ...
Otherwise, matplotlib seems to be the most user friendly way

  
  
Posted one month ago

Yeah actually you are right. I can report the stats as a table, not the whole data.

  
  
Posted one month ago

yup, you have the flexibility and option, that what so nice with ClearML

  
  
Posted one month ago

Oh, I think that is for a very small data. I don't think it works for me.

  
  
Posted one month ago

I will have to go for matplotlib or seaborn options.

  
  
Posted one month ago

I know. And the very fast help 🙏 😀

  
  
Posted one month ago

and just came across this: None
That sounds like what you may be looking for

  
  
Posted one month ago

Can I see visualization of for example categorical columns as bar graphs?

  
  
Posted one month ago

At least I can do that along with matplotlib

  
  
Posted one month ago
312 Views
19 Answers
one month ago
one month ago
Tags