Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
How Do I Get The Parsed Args Object Of A Task? I Ran A Task, Using The Argparser To Set Its Hyperparams. I Now Get The Task Using Task.Get_Task, And I Want To Instantiate An Args Namespace Which Has The Same Values As When The Task Ran The Parsing Code, A

How do I get the parsed args object of a task? I ran a task, using the argparser to set its hyperparams. I now get the task using Task.get_task, and I want to instantiate an args Namespace which has the same values as when the task ran the parsing code, args = parser.parse_args().

I managed to get the parameters using either task.get_parameters or task.get_parameters_as_dict, but this yields all params as Strings, rather than the original param types. What's the cleanest way to reconstruct the args Namespace as it was when the task originall ran?

  
  
Posted 2 years ago
Votes Newest

Answers 12


SuccessfulKoala55 I ran a training task. I now wish to run inference on some data. My model's init function expects the parsed args Namespace as an argument. In order to load the weights of the saved model I need to instantiate the class for which I need the args variable.

  
  
Posted 2 years ago

ThickDove42 what is the use-case, exactly?

  
  
Posted 2 years ago

SuccessfulKoala55 on a different question that came up in the context of this use case, I want to use Task.init(continue_last_task=id) in order to add the inference results to the original task that ran the inference.

I was able to do this for a single task. I locally fetch the task ids, then for the first task I run Task.init with continue_last_task and subsequently I run task.execute_remotely() and the task is then run remotely with the outputs appended to the original training task.

When doing this to a single task, I see that the task's script is being replaced with the current inference script and that's working fine. How can I do this for multiple tasks? I tried runnitn Task.enqueue after the task.init with continue but that doesn't work.

  
  
Posted 2 years ago

args_string = ['--{} {}'.format(k, v) for k, v in task.get_parameters_as_dict().get('Args', {}).items()] args_strings = [a.replace("'", '').replace("[", '').replace("]", '').replace(",", '') for a in reduce(lambda l, s: l + s.split(' '), args_string, []) if a]and then just parser.parse_args(ars_strings)

It's not even very clean, I could have replaced the multiple replace calls with a regex etc. just a quick hack to work around it.

Actually, I found out that if I use execute_remotely I don't even need this, since the parser arguments are apparently reconstructed. so just instantiating the parser and running parser.parse_args() is enough

  
  
Posted 2 years ago

This is what's working for me.
for task in tasks: subprocess.run(['python', './test_task.py', '--task_id', task.id])where in test_task.py I have the following:
` parser = ArgumentParser()

if Task.running_locally():
parser.add_argument('--task_id',type=str)
args=parser.parse_args()

task = Task.init(
    continue_last_task=args.task_id
)
task.execute_remotely(queue_name='default') `and the rest is the inference code, which is only run on the remote, and includes populating the argparaser instance with the necessary arguments
  
  
Posted 2 years ago

SuccessfulKoala55 I managed to find a workaround, by instantiating a new parser and feeding it the string values. if there'd be a cleaner solution I'll be happy to hear about it.

  
  
Posted 2 years ago

can you give me a snippet? when I tried it didn't work, since execute_Remotes terminates the local task, unless you tell it to clone, which is not what I need here.

  
  
Posted 2 years ago

Can you share a code snippet? 🙂

  
  
Posted 2 years ago

why not use execute_remotely as well?

  
  
Posted 2 years ago

SuccessfulKoala55 this seems to work, but I would have preferred a continue_last_task version of enqueue, which would handle things for me, instead of introducing another level of hierarchy

  
  
Posted 2 years ago

the only solution I see is to have another script that would run the test task using (for example) popen in a separate process.

  
  
Posted 2 years ago

Hi ThickDove42 , let me check that...

  
  
Posted 2 years ago
565 Views
12 Answers
2 years ago
one year ago
Tags
Similar posts