Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi, I'Ve Just Started To Evaluate Clearml For Internal Use At My Org And Am Wondering If There'S Anyway To Import Data From Old Experiments Into The Dashboard. Anyone Have Any Thoughts On This?

I've just started to evaluate ClearML for internal use at my org and am wondering if there's anyway to import data from old experiments into the dashboard. Anyone have any thoughts on this?

Posted 2 years ago
Votes Newest

Answers 10

Hi JumpyPig73

import data from old experiments into the dashboard.

what do you mean by "old experiments" ?

Posted 2 years ago

We have run experiments in the past (before I put ClearML into my code) which has logged scalars, plots etc. to local tensorboard. Is there any way to import this data to ClearML cloud for tracking, visualization and comparison?

Posted 2 years ago

So basically take data from tensorboard read it, and report it to the cloud ?

Posted 2 years ago


Posted 2 years ago

I guess this is doable:
You can get the entire set of scalars like as pandas DF: https://www.tensorflow.org/tensorboard/dataframe_api
(another example: https://stackoverflow.com/a/45899735 )
Then iterate over the different runs and create + report scalars)
` from clearml import Task

for run in runs:
task = Task.create_task(...)
logger = task.get_logger()

not real code, just example:

w_times, step_nums, vals = zip(*event_acc.Scalars('Accuracy'))
for step, val in zip(step_nums, vals):
logger.report_scalar(title='Accuracy', series='Accuracy', iteraiton=step, value=val)

we are done

task.close() `Obviously this is missing the link to the code repository + python packages + console logs etc. But it will get you the scalars 🙂

Posted 2 years ago

This is great! Thanks!
If I have access to the logs, python env and git commits, is there an API to log those to the experiments too?

Posted 2 years ago

If I have access to the logs, python env and git commits, is there an API to log those to the experiments too?

task.update_task see here:
task.update_task(task_data={'script': {'branch': 'new_branch', 'repository': 'new_repo'}})The easiest way to get all the different sections (they should be relatively self explanatory) is calling task.export_task() which returns a dict with all the fields you can manually configure.

Posted 2 years ago

For reporting the console logs you can use :
logger.report_text("my log line here", print_console=False)https://github.com/allegroai/clearml/blob/b4942321340563724bc16f60ea5dd78c9161778d/clearml/logger.py#L120

Posted 2 years ago

This is great! Thanks for the example Martin, much appreciated!

Posted 2 years ago

With pleasure 🙂

Posted 2 years ago
10 Answers
2 years ago
one year ago