Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Folks, Good Morning

Hi folks, good morning 🙂
In our setup we have a set of queues that do not use any GPU resources. Yet, when I run an experiment in such queues, we see a Warning:
ClearML Monitor: GPU monitoring failed getting GPU reading, switching off GPU monitoring
Is there a way to disable GPU monitoring for an agent?

  
  
Posted one year ago
Votes Newest

Answers 6


Thanks CostlyOstrich36 I was thinking more to a setting of the environment, for example the documentation mentions the "--cpu-only" flag (which I am not sure I can use as I am using the helm charts from AllegroAI, I don't think I can override the command), or to set the env var NVIDIA_VISIBLE_DEVICES to an empty string (which I did, but I can still see the message)

  
  
Posted one year ago

Hi folks, I think I found the issue, the documentation mention to set NVIDIA_VISIBLE_DEVICES to "", when in reality it should be "none" according to the code:

if Session.get_nvidia_visible_env() == 'none': # NVIDIA_VISIBLE_DEVICES set to none, marks cpu_only flag # active_gpus == False means no GPU reporting self._active_gpus = False

  
  
Posted one year ago

Hi CostlyOstrich36 , the document mention ResourceMonitor class, but when I check the code at https://github.com/allegroai/clearml/blob/master/clearml/utilities/resource_monitor.py
I don't see gpu monitoring can be turned off from the class interface

  
  
Posted one year ago

Hi SarcasticSquirrel56 ,

In Task.init() you have the parameter auto_resource_monitoring
https://clear.ml/docs/latest/docs/references/sdk/task#taskinit
You can specify there what to turn off

  
  
Posted one year ago

As much as possible, I'd like removing the burden off the shoulders of people writing their models

  
  
Posted one year ago

that disabled gpu for me

  
  
Posted one year ago