Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Trains-Riders, Is There A Way From The Web Ui, To Automate Multiple Experiments, Each With A Different Configuration? Let Me Clarify What I Mean: Say I Have An Experiment Already Executed And Visible In The Experiments View. I Can Manually Duplicate I

Hey Trains-riders,
Is there a way from the web UI, to automate multiple experiments, each with a different configuration?
Let me clarify what I mean:
Say I have an experiment already executed and visible in the experiments view.
I can manually duplicate it multiple times, and each time modify one or more arguments and send it to the queue again.
Now I have a file with many experiment configurations, for simplicity let's assume that each configuration is simply a string.
(My experiment code knows how to parse this string and convert it to a proper experiment configuration, of course)
What I am looking for, is a way to parse this file, and for each line: duplicate an experiment; set this line as an argument, send it to the queue.

The idea is that I will be able to change/replace this configuration file from time to time, as my list of experiments evolves.

Note: it doesn't have to be a text file with a line per configuration, I don't mind converting it to any format (json, python, etc)

thanks

  
  
Posted 3 years ago
Votes Newest

Answers 6


Otherwise I might as well replace it with my own script that simply sends the configurations list to some random training server, which in turn will execute a version of TimelyPenguin76 's script. And the user will not even be aware of that. From his point of view the experiments will miraculously appear in the UI 🙂

That's an optional plan B

  
  
Posted 3 years ago

I don't mind writing JS or other scripts for that, if there's a hook waiting for me 🙂

  
  
Posted 3 years ago

Hi ColossalDeer61 ,

Not from the UI, but you can run a simple script to do that (assuming you can parse your configuration file), here is an example:

` from trains import Task

configuration_file = {
"stage_1": {
"batch_size": 32,
"epochs": 10
},
"stage_2": {
"batch_size": 64,
"epochs": 20
},
}

template_task = Task.get_task(task_id=<YOUR TEMPLATE TASK>)

for name, params in configuration_file.items():
# clone the template task into a new write enabled task (where we can change parameters)
cloned_task = Task.clone(source_task=template_task,
name=template_task.name + ' {}'.format(name),
parent=template_task.id)

# get the original template parameters
cloned_task_parameters = cloned_task.get_parameters()

# override
for k, cloned_params in params.items():
    cloned_task_parameters[k] = cloned_params

# put back into the new cloned task
cloned_task.set_parameters(cloned_task_parameters)
print('Experiment {} set with parameters {}'.format(name, cloned_task_parameters))

# enqueue the task for execution
Task.enqueue(cloned_task.id, queue_name="default")  # <--- Change "default" to your queue name
print('Experiment id={} enqueue for execution'.format(cloned_task.id)) `can this to the trick?
  
  
Posted 3 years ago

This is certainly a good way, which we use today.
I am looking for an even simpler way, for less technical people, who could apply this remotely, using the UI.

edit : the issue is less concerning technical level, but rather access to training machines

  
  
Posted 3 years ago

interesting, so you actually enqueue a task-generating task, which will eventually, once executed, enqueue all the configuration task (as you proposed earlier).

  
  
Posted 3 years ago

ColossalDeer61 we can add to the script Task.init and we will have this service in the UI, with connection to the configuration, users can change the configuration from the UI and send this script to execute (in services queue for example). This way others can change the configuration and run this script.

` from trains import Task

service_task = Task.init(project_name="services", task_name="clone and enqueue by configuration")

configuration_file = {
"stage_1": {
"batch_size": 32,
"epochs": 10
},
"stage_2": {
"batch_size": 64,
"epochs": 20
},
}

service_task.connect(configuration_file)
template_task = Task.get_task(task_id=<YOUR TEMPLATE TASK>)

for name, params in configuration_file.items():
# clone the template task into a new write enabled task (where we can change parameters)
cloned_task = Task.clone(source_task=template_task,
name=template_task.name + ' {}'.format(name),
parent=template_task.id)

# get the original template parameters
cloned_task_parameters = cloned_task.get_parameters()

# override with random samples form grid
for k, cloned_params in params.items():
    cloned_task_parameters[k] = cloned_params

# put back into the new cloned task
cloned_task.set_parameters(cloned_task_parameters)
print('Experiment {} set with parameters {}'.format(name, cloned_task_parameters))

# enqueue the task for execution
Task.enqueue(cloned_task.id, queue_name="default")  # <--- Change "default" to your queue name
print('Experiment id={} enqueue for execution'.format(cloned_task.id)) `
  
  
Posted 3 years ago