OCT
26
2005

On measuring memory usage #2

Hmm ... I should wish things more often. I complained about the lack of any usable memory reporting tool on Linux, and only a couple of days later I discovered exmap. Unlike e.g. top with its number of useless memory usage values, exmap has only three of them.

  • First of them is virtual memory, i.e. top's VIRT. Not really that useful in practice except for some specific cases.
  • Second is mapped memory, i.e. memory that the process actually really uses. This value still includes memory that's shared with other processes.
  • Last and definitely not the least but rather the other way around, there's the effective size. It is mapped size, but for shared memory the shared portion are divived into all processes that use it. To give a simplified example, if there are 10 KDE apps running, then for each of them the effective size is its unshared memory plus 1/10 of memory used by KDE libraries.

The effective memory size is so far the best approximation of the one-number-for-memory-usage-of-an-app value I've seen that even Joe User could use without getting it horribly wrong. Not that it'd be impossible to get this value wrong too, you still need to understand what this is about, but if Joe User says "app XYZ uses too much memory according to top" you can ignore this as being completely clueless and wrong; with exmap, Joe User would have a number that really kind of means what they think it means.
Too bad exmap needs to insmod a kernel module and run under root. I wish it replaces top and top remains only as a (very) experienced user's tool.

Comments

BTW, Linux 2.6.14 will feature a new /proc/$PID/smaps file, which can help to track that kind of things too, check http://wiki.kernelnewbies.org/LinuxChanges


By diegocg at Wed, 10/26/2005 - 13:08

thx


By paris hilton movie at Sun, 07/06/2008 - 19:49

The memory usage reported on XP isn't very accurate either. The other day I was playing Quake4 and I had set the texture option to 'ultra' which probably wasn't the best idea. It then filled up my ram and filled my swap file to 1.66 gigs. The interesting thing is that Task Manager reported the memory usage of Quake4 to be only ~200 megs which led me to conclude that task Manager's reporting isn't accurate in the slightest either since Windows is always kicking stuff into the swap file (in windows I generally have always at least 266 megs of swap used, in linux my swap is almost never being used unless i run a game)

So when people compare KDE programs like Konqueror to Windows programs like Explorer, those comparisons aren't even accurate on the Windows side of things! I don't know about OS X, but hopefully it reports things better...


By sirtalon at Wed, 10/26/2005 - 16:20

Hi.

Firstly thanks for the positive comments about exmap. I'm afraid a kernel module is necessary to get the information out. Unless I'm mistaken, even the /proc//smaps in newer kernels doesn't give enough information to calculate the 'effective size'. I believe it will tell you how much memory is shared, but not how many processes share the memory.

As regards running as root, it is mostly relevant for examining files which are also mapped by some root-owned procs. If you solely interested tracing KDE mem usage, I suspect you'll get fairly similar numbers running as the desktop user.

There is an updated version of exmap at:

http://homepage.ntlworld.com/john.berthels/exmap/exmap-0.5.tgz

which corrects some inaccuracies in 0.4 and will give byte-level granularity of ELF sections and symbols (in unstripped binaries).

If you have any observations, suggestions or problems, please let me know (email address is in the README in the tarball linked above). Also, I'm always interested to hear how people are using the tool and any particular successes they have.

John Berthels


By jbert at Fri, 11/04/2005 - 14:11

Hey, finally something that makes me believe there will be a day when my first reaction to a user reporting "big" memory usage won't be "sigh". You certainly deserve positive comments. And I'm mostly aware of the root-related problems, I wrote those parts mostly to make it obvious to readers. Hopefully exmap will become a standard tool and this problem will more or less go away.

As for suggestions, it'd be nice if could add these to your TODO :
- Most KDE processes are shown as /opt/kde3/bin/kdeinit here, because of the kdeinit hack. However with recent kernel versions top shows them as e.g. kwrite instead of kdeinit. I'm not quite sure how that's done though.
- The UI tool could either default to downwards sorting by effective memory or even better save the sorting. I always have to double-click the column in all views to have them sorted in the IMHO best way.

Oh, and successes ... does finding and fixing a bug in aspell resulting in 60MiB unshared memory per process count ;) ?


By Lubos Lunak at Mon, 11/07/2005 - 15:05

I've put up a lightly-tweaked 0.5 here:

http://homepage.ntlworld.com/john.berthels/exmap/exmap-0.5.1.tgz

which hopefully includes your two wishlist items (and some other (fairly unproven) fixes, sorry if that breaks anything).

I'm *very* pleased to hear that you've managed to get some benefits - may I ask the details of the aspell issue?


By jbert at Mon, 11/07/2005 - 18:36

Seems to work fine so far, and yes, kdeinit apps now show fine, although that now sometimes causes a little problem - some apps have very long command-line arguments and that makes the memory usage numbers columns not visible without adjustments :( .

As for aspell, that wasn't actually that interesting. Since I finally had a tool that worked :), I just wanted to confirm that KDE apps are actually quite lightweight and went onto comparing them to non-KDE counterparts. Konqueror managed to beat Firefox hands down, but I was rather suprised to see KOffice need much more than OpenOffice. With exmap it was however rather simple to guess that the huge anon block came from huge single malloc, finding out it was an incorrect mmap offset in aspell making the mmap fail still took some time though.

However, as I'm now working with my home machine, which unlike the work machine has swap, I think I've just found out something that rather worries me :(. I can noticeably alter reported memory usage for apps by swapping them out. That of course makes sense technically, just like the columns in top actually make sense technically, but it suddenly makes exmap less useful for this-number-is-the-memory-usage-of-the-app purpose that even Joe User could use.

I'm aware that read-only pages such as binaries are simply discared on memory shortage, so even not using swap helps here, but it seems to me this affects even things like heap that have to be swapped out. Is there any way exmap could take this into account? Maybe as new columns, something like "total memory of the app that can't be discarded" and "this column is my best guess at how much memory this app really needs". I'm afraid I don't know how to better explain but I hope you understand what I mean. And the documentation should probably stress this fact when describing the various types of memory usage. Maybe I'm too paranoid and this shouldn't usually happen, but on the other hand I think this could hide things like an app with a memory leak if the leaked memory gets continually swapped out. What do you think about this?


By Lubos Lunak at Mon, 11/07/2005 - 20:21

Oops. OK, cmdline will be trimmed to the first one or two tokens (second token included if it begins with a "[", so kdeinit is flagged).

More importantly, regarding the lower effective size when the app is swapped out. Yes, you're right this is by design. The effective size shows what memory is in use at the time of the snapshot. If things aren't too dynamic, this is the resident set. On low-mem machines which swapping is actively occuring, more work is probably needed to get more accurate numbers.

For file-backed maps, as you say, this is probably the most useful thing. For anon-maps, well, thats interesting. On a per-process basis, you can see the total amount of [heap] and [anon] via the 'vm size' column. This would show up memleaks for example.

I guess a "Lubos Size" for a process might be "effective size of all file-backed maps + vm size of non-file-backed maps"? Is that a number which makes any sense? I'll give this some more thought.

[NB: Linux does support shared anon-maps too. I think they are rarely used.]

[NB2: GNU malloc allocates large chunks as seperate [anon] maps, so heap usage is practically speaking [heap]+[anon] for most apps. Does anyone know a good heuristic exists for determining if an [anon] map was made by GNU malloc?]

[NB3: I get a mailed copy of the comment, but only a few days afterwards, so sorry for the slow reply]


By jbert at Thu, 11/10/2005 - 15:52