Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Another Conundrum: I Have A Single Script That Launches Training Jobs For Various Models. It Does This By Accepting A Flag Which Is The Model Name, And Dynamically Loading The Module To Train It. This Didn'T Mesh Well With Trains, Because The Project And

Another conundrum: I have a single script that launches training jobs for various models. It does this by accepting a flag which is the model name, and dynamically loading the module to train it. This didn't mesh well with Trains, because the project and task names were dynamically generated based on the flag. Since trains only populates flags after initialization, these flags were always empty when running the job on trains-agent. I'm not sure if there's a good workaround for this. For now, I'm back to using separate scripts with hard-coded names for the project and task.

  
  
Posted 4 years ago
Votes Newest

Answers 4


Hi SillyPuppy19
I think I lost you half way through.

I have a single script that launches training jobs for various models.

Is this like the automation example on the Github, i.e. cloning/enqueue experiments?

flag which is the model name, and dynamically loading the module to train it.

a Model has a UUID in the system as well, so you can use that instead of name (which is not unique), would that solve the problem?

This didn't mesh well with Trains, because the project and task names were dynamically generated based on the flag.

Not sure I follow, but you can always change a Task name, even after initialization of cloning. Is this what you mean?

  
  
Posted 4 years ago

AgitatedDove14 sorry if that wasn't clear. I think the issue is that when trains-agent runs the script, none of the flag values are set until the Task object is initialized. For that to happen, the task object needs to know what project/task to connect to, which I presume is via the project_name and task_name parameters.

If those parameters are themselves dependent on flags, then they will be uninitialized when trains-agent runs the script, as it does not run it with any command line flags.

  
  
Posted 4 years ago

I presume is via the 

project_name

  and 

task_name

 parameters.

You are correct in your assumption, it only happens when you call Task.init but two distinctions:
ArgParser arguments are overridden (with trains-agent) even before Task.init is called Task.init when running under trains-agent will totally ignore the project/task name, it receives a pre-made task id, and uses it. So the project name and experiment are meaningless if you are running the task with trains-agent

A few implementation details to complete the image for you. Task.init creates a new Task when running manually, and puts data into the created task (e.g. ArgParser dictionary params etc)
When called under trains-agent, the Task.init will do the opposite. It will not create a Task but it receives a Task id, from it it will pull all the parameters into the code, including argparser etc.
The exception is ArgParser, since it can be called before Task.init is called. As long as someone imported the trains package, the auto-magic will make sure that in trains-agent mode, the argparser will get the correct arguments before the call for Task.init

  
  
Posted 4 years ago

That's great to know. Thank you AgitatedDove14 . I might have gone wrong somewhere else, so I'll double-check.

  
  
Posted 4 years ago
1K Views
4 Answers
4 years ago
one year ago
Tags