Re: memory reported in /proc/pid/status is wrongly scaled

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

On Aug 17 19:14, Corinna Vinschen wrote:
> On Aug 17 16:05, Livio Bertacco wrote:
> >  Hi,
> > While playing with reading process memory usage in Linux and cygwin, I
> > found that cygwin reports too large values in /proc/self/status (in 2.10.0
> > and earlier).
> > Whenever I was allocating a few kB in my test program, the VmSize line in
> > /proc/self/status was growing several times faster.
> > 
> > Small bash script to show the issue:
> > #!/bin/bash
> > pid=$$
> > vmsizesplit=($(grep VmSize /proc/$pid/status))
> > vmsize1="${vmsizesplit[1]}"
> > echo Initial memory reported in status: $vmsize1 kB
> > echo Allocating a 1000 kB string (bash can use more memory)
> > eat=$(printf '%1024000s')
> > vmsizesplit=($(grep VmSize /proc/$pid/status))
> > vmsize2="${vmsizesplit[1]}"
> > echo Current memory reported in status: $vmsize2 kB
> > echo Difference is $[$vmsize2-$vmsize1] kB
> > 
> > Running this in cygwin on my laptop I get:
> > Initial memory reported in status: 84928 kB
> > Allocating a 1000 kB string (bash can use more memory)
> > Current memory reported in status: 106880 kB
> > Difference is 21952 kB
> > 
> > While bash may use quite more than 1000 kb in this case, 22x times larger
> > doesn't seem right.
> > 
> > Checking source file fhandler_process.cc, the
> > function format_process_status which writes the "status" proc file
> > retrieves memory usage via get_mem_values. Get_mem_values obtains that info
> > from NtQueryInformationProcess/PagefileUsage which is in bytes, then it
> > scales it to pages dividing by wincap.page_size:
> > 1515: *vmsize = vmc.PagefileUsage / wincap.page_size ();
> > 
> > Then format_process_status scales it back, in theory to bytes, and shifts
> > it by 10 bits in order to print it out in kB:
> > 1219:  unsigned page_size = wincap.allocation_granularity ();
> 
> Looks like this is the bug.  get_mem_values returns all values
> in multiple of OS page_size (4K), but format_process_status multiplies
> with allocation_granularity (64K), leading to 16 times overallocation
> value.  The other caller of get_mem_values, format_process_statm,
> returns number of pages.  This must be expressed in multiples of
> allocation_granularity since that's the virtual page_size in Cygwin.
> But in case of format_process_status we're looking at KB values, so
> patch 8a32c24a7bdb0, replaceing page_size with allocation_granularity,
> was incorrect.
> 
> Good catch!
> 
> I'll revert patch 8a32c24a7bdb0 for 2.11.0.

On second thought there's more wrong than just that.  Just dividing
by page_size or allocation_granularity results in too small values.
I applied patches to return more correct values and I made sure
the values in status and statm are consistently rounded up to
Cygwin's page size of 64K.

I uploaded new developer snapshots to https://cygwin.com/snapshots
for testing.  Please give them a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Attachment: signature.asc
Description: PGP signature