Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
. Is There A Tab Showing The Resource Usage(Cpu/Gpu Per Experiment? I Would Like To Use It To Better Profiling My Code)

None .

Is there a tab showing the resource usage(CPU/GPU per experiment? I would like to use it to better profiling my code)

  
  
Posted 3 years ago
Votes Newest

Answers 8


Is it possible to set the frequency of sampling?

  
  
Posted 3 years ago

Hi @<1523713917677735936:profile|EnviousStarfish54> ,
Trains samples the usage every 30 seconds, and reports on every iteration (if no iterations are reported, it should be reported every 3min approximately). The error you see indicates that Trains failed to detect stats for a GPU on your machine, in which case it should still report all the other stats (CPU etc.)

  
  
Posted 3 years ago

Hi @<1523713917677735936:profile|EnviousStarfish54> ,
In your experiment's Results/Scalars sub-panel you should see a graph titled :monitor:machine which contains a series per monitored value (e.g. CPU usage etc.)

  
  
Posted 3 years ago

Currently, no - is that a real use-case in your opinion?

  
  
Posted 3 years ago

it's ok, I don''t think this is very important. thx

  
  
Posted 3 years ago

@<1523701087100473344:profile|SuccessfulKoala55> Thx! I swear I saw it before, but somehow I just overlooked it.

  
  
Posted 3 years ago

I also get this from the logging

TRAINS Monitor: GPU monitoring failed getting GPU reading, switching off GPU monitoring
  
  
Posted 3 years ago

How often does Trains record the usage? I tried running a dummy programming which sleeps for 20 seconds and I don't see any plot.

Is this configurable?
image

  
  
Posted 3 years ago