Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All. What Is The Most Straightforward Way To Download All Experiment-Related Data (Configs, Scalars, Debug Samples Etc.) For One Experiment From Clearml Server (Self-Hosted)? I Want To Backup Some Chosen Experiments And Also Poke Around The Experimenta

Hi all. What is the most straightforward way to download all experiment-related data (configs, scalars, debug samples etc.) for one experiment from ClearML server (self-hosted)? I want to backup some chosen experiments and also poke around the experimental data locally.

  
  
Posted 5 months ago
Votes Newest

Answers 4


Hi @<1529633475710160896:profile|ThickChicken87> , I would suggest opening developer tools (F12) and observing what api calls go out when going over the experiment object. This way you can replicate the api calls to pull all the relevant data. I'd suggest reading more here - None

  
  
Posted 5 months ago

Great suggestions, thanks guys

  
  
Posted 5 months ago

We use task.export_task() and a hacked version to get console log:


def save_console_log(task: clearml.Task, fs, remote_path, number_of_reports=10000):
    from clearml.backend_api.services import events
    from clearml.backend_api import Session

    # Stollen from Task.get_reported_console_output()
    if Session.check_min_api_version('2.9'):
        request = events.GetTaskLogRequest(
            task=task.id,
            order='asc',
            navigate_earlier=True,
            batch_size=number_of_reports)
    else:
        request = events.GetTaskLogRequest(
            task=task.id,
            order='asc',
            from_='tail',
            batch_size=number_of_reports)

    res = task.send(request)
    response = res.wait()
    if not response.ok() or not response.response_data.get('events'):
        print("Warning: No Clearml Console log retrieved")
        return

    logs = response.response_data['events']
    json_data = json.dumps(logs, indent=2)
    print(f"Saving ClearML task console logs to {remote_path}")
    with fs.open(remote_path, "w") as json_file:
        json_file.write(json_data)
  
  
Posted 5 months ago

not sure how for debug sample and scalars ....
But theorically, with the above, one should be able to fully reproduce a run

  
  
Posted 5 months ago
342 Views
4 Answers
5 months ago
5 months ago
Tags