Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! I’Ve Run A Task In A Docker Container With Memory Constraint 16Gb (Clearml-Task ….. --Docker_Args “--Memory=16G”), So I Expected To See The Max Memory Available Equal 16Gb In Web Ui (Scalars/Monitor:Machine), But It Shows Memory Available In The Whole

Hi! I’ve run a task in a docker container with memory constraint 16gb (clearml-task ….. --docker_args “--memory=16g”), so I expected to see the max memory available equal 16gb in web ui (scalars/monitor:machine), but it shows memory available in the whole machine (much more than 16gb). How can I change that?

  
  
Posted one year ago
Votes Newest

Answers 10


EnviousPanda91 notice that when passing these arguments to clearml-agent you are actually passing default args, if you want an additional argument to Always be used, set the extra_docker_arguments here:
https://github.com/allegroai/clearml-agent/blob/9eee213683252cd0bd19aae3f9b2c65939d75ac3/docs/clearml.conf#L170

  
  
Posted one year ago

Wait, I noticed you need another set of quotes:
Sample in the docs is: --memory="300m"

https://docs.docker.com/config/containers/resource_constraints/

  
  
Posted one year ago

Wait... Will it limit memory usage but not show it in overview or just plain stop the container if it happens to use more memory?

  
  
Posted one year ago

Can you try again with "-m 16g" ?

  
  
Posted one year ago

Basically internally we use psutil to get those stats ...
https://github.com/giampaolo/psutil/issues/1011
See psutil version that fixed that, what do you see on the Task "installed packages" ?
https://github.com/giampaolo/psutil/blob/master/HISTORY.rst#591

  
  
Posted one year ago

One additional thing to notice, docker will Not actually limit the "vioew of the memory" it will just kill the container if you pass the memory limit, this is a limitation of docker runtime

  
  
Posted one year ago

it will only if oom killer is enabled

true, but you will still get OOM (I believe). I think the main issue is the even from inside the container, when you query the memory, you see the entire machine's memory... I'm not sure what we can do about that

  
  
Posted one year ago

docker will Not actually limit the “vioew of the memory” it will just kill the container if you pass the memory limit, this is a limitation of docker runtime

it will only if oom killer is enabled

  
  
Posted one year ago

Also have a look at --memory-swap
It seems you might not anticipated this usage:

If --memory-swap is unset, and --memory is set, the container can use as much swap as the --memory setting, if the host container has swap memory configured. For instance, if --memory="300m" and --memory-swap is not set, the container can use 600m in total of memory and swap.

  
  
Posted one year ago
611 Views
10 Answers
one year ago
one year ago
Tags