Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Encountered A Few Problems:

Hi, I encountered a few problems:
Which files will be uploaded by trains according to the task's output_uri ? model, text or images? Is the judgment based on the file extension? If I have a file that I don't want to add to git control (I will frequently modify it), is there a way for trains to access this file and call it in the repository? Because I use docker to run my worker, can I make the worker not run the virtual environment after receiving the task? When I use torch.save() , will trains change the name of my model before uploading? Is it possible not to change the name?Thanks in advance πŸ™

  
  
Posted 3 years ago
Votes Newest

Answers 8


  1. Artifacts and models will be uploaded to the output URI, debug images are uploaded to the default file server. It can be changed via the Logger.
  2. Hmm is this like a configuration file?
    You can do.
    local_text_file = task.connect_configuration('filenotingit.txt')
    Then open the 'local_text_file' it will create a local copy of the data in runtime, and the content will be stored on the Task itself.
  3. This is how the agent installs the python packages, but if the docker already contactains them, then they will be used so not reinstalled. This gives you the flexibility of changing version or packages but still have all the base stuff preinstalled.
  4. No, it will keep the same file name, that means that if the file name is fixed you are also overwriting the file on you file server
  
  
Posted 3 years ago

Hi FierceFly22 I'll answer according to q order?

  
  
Posted 3 years ago

Hi AgitatedDove14 , I try to use your method( local_text_file = task.connect_configuration('filenotingit.txt') ) to access local_text_file.
But when I write with open(local_text_file,'r') as f: print(f.read()) , it print nothing at console... This's part of my code:
` task = Task.init(project_name='Example',task_name='Test')
task.execute_remotely(queue_name="Important_Queue", clone=False, exit_process=True)

config_file = os.path.join('cfg_test','cfg.txt')
config_file = task.connect_configuration(config_file)

with open(config_file, "r") as f:
print(f.read()) `Did i make any mistake?
Thanks πŸ™

  
  
Posted 3 years ago

Hi FierceFly22
You called execute_remotely a bit too soon. If you have any manual configuration, they have to be called before, so they are stored in the Task. This includes task.connect and task.connct_configuration.

  
  
Posted 3 years ago

ok, I will try again. Thank you very much!

  
  
Posted 3 years ago

Hi AgitatedDove14 ,
I found that when I use with open('modelsummary.txt', 'w') as f: with redirect_stdout(f): model.summary()trains server upload modelsummary.txt to be a artifact. So I think, if I use os.system("touch abcd") ,what will trains server do? Doesn’t It just upload specific files(model,image)?
2.3,4 Thank you very much, I will try again 😊

  
  
Posted 3 years ago

FierceFly22 wow that is a cool hack! Trains will capture any torch.save , so I think the actual driver here is the 'model.summary' . You can also upload any artifact with task.upload_artifact('name', 'modelsummary.txt')
Touching a file will not trigger Trains, as it does not monitor the files themselves. Make sense?
BTW, how will you get the file when running with the agent? If you are using the connect_configuration it will be downloaded from the trains-server for you. Otherwise you can always get artifacts from any task on the system with Task.get_task('taskidhere'). artifacts ['name'].get()

  
  
Posted 3 years ago

AgitatedDove14 , I got it! Thank you very mush πŸ˜€

  
  
Posted 3 years ago
416 Views
8 Answers
3 years ago
one year ago
Tags
Similar posts