Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
So I'M In A Colab Notebook, And After Running My Trainer(), How Do I Upload My Test Metrics To Clearml? Clearml Caught These Metrics And Uploaded Them:

So I'm in a Colab notebook, and after running my Trainer(), how do I upload my test metrics to ClearML?
ClearML caught these metrics and uploaded them:
` trainer = Trainer(model=model,
args=training_args,
data_collator=data_collator,
train_dataset=dep['train'],
eval_dataset=dep['validation'],
compute_metrics=compute_metrics,
callbacks=[
EarlyStoppingCallback(early_stopping_patience=10),
]
# pretrained_bert=None,
)

trainer.train() But not these from the next cell:
posttrain_metrics = trainer.predict(dep['test']).metrics
print(posttrain_metrics) `

  
  
Posted 3 years ago
Votes Newest

Answers 8


Hi SmallDeer34
Did you call Task.init ?

  
  
Posted 3 years ago

(second cell)

  
  
Posted 3 years ago

AgitatedDove14 yes, I called init and tensorboard is installed. It successfully uploaded the metrics from trainer.train(), just not from the next cell where we do trainer.predict

  
  
Posted 3 years ago

No, they're not in Tensorboard

  
  
Posted 3 years ago

By default the pl Trainer will output everything to TB, which we automatically store. But verify that TB is installed

  
  
Posted 3 years ago

This seems to work:

` from clearml import Logger
for test_metric in posttrain_metrics:
print(test_metric, posttrain_metrics[test_metric])

#report_scalar(title, series, value, iteration)
Logger.current_logger().report_scalar("test", test_metric, posttrain_metrics[test_metric], 0) `

  
  
Posted 3 years ago

No, they're not in Tensorboard

Yep that makes sense

Logger.current_logger().report_scalar("test", test_metric, posttrain_metrics[test_metric], 0)

That seems like a great solution

  
  
Posted 3 years ago

can you see these metric on TB ?

  
  
Posted 3 years ago
1K Views
8 Answers
3 years ago
one year ago
Tags