For reporting the console logs you can use :logger.report_text("my log line here", print_console=False)https://github.com/allegroai/clearml/blob/b4942321340563724bc16f60ea5dd78c9161778d/clearml/logger.py#L120
This is great! Thanks!
If I have access to the logs, python env and git commits, is there an API to log those to the experiments too?
If I have access to the logs, python env and git commits, is there an API to log those to the experiments too?
Sure:task.update_task see here:
https://clear.ml/docs/latest/docs/references/sdk/task#update_task
example:task.update_task(task_data={'script': {'branch': 'new_branch', 'repository': 'new_repo'}})The easiest way to get all the different sections (they should be relatively self explanatory) is calling task.export_task() which returns a dict with all the fields you can manually configure.
https://clear.ml/docs/latest/docs/references/sdk/task#export_task
So basically take data from tensorboard read it, and report it to the cloud ?
Hi JumpyPig73
import data from old experiments into the dashboard.
what do you mean by "old experiments" ?
I guess this is doable:
You can get the entire set of scalars like as pandas DF: https://www.tensorflow.org/tensorboard/dataframe_api
(another example: https://stackoverflow.com/a/45899735 )
Then iterate over the different runs and create + report scalars)
` from clearml import Task
for run in runs:
task = Task.create_task(...)
logger = task.get_logger()
not real code, just example:
w_times, step_nums, vals = zip(*event_acc.Scalars('Accuracy'))
for step, val in zip(step_nums, vals):
logger.report_scalar(title='Accuracy', series='Accuracy', iteraiton=step, value=val)
we are done
task.close() `Obviously this is missing the link to the code repository + python packages + console logs etc. But it will get you the scalars 🙂
This is great! Thanks for the example Martin, much appreciated!
We have run experiments in the past (before I put ClearML into my code) which has logged scalars, plots etc. to local tensorboard. Is there any way to import this data to ClearML cloud for tracking, visualization and comparison?