Cannot correctly display memory usage under Linux

I got VPS and have ubuntu on top of it. I have installed apache in worker role (not prefork ) also installed ruby EE 1.8.7 & mysql 5.x

Now when I check for memory I see 18.6 % (of 924 MB ) is used by the rack ( must be rails app that I am running)

and When I do free -m I get the following results :

             total       used       free     shared    buffers     cached
Mem:           924        602        322          0          0          0
-/+ buffers/cache:        602        322
Swap:            0          0          0

where is my memory used up ? free -m shows all the memory used is my the cache and nothing else how is that possible ? I don't have any users for the site as of now.

AUX doesn't show anything interesting except 18.6% by rack and 2.6% by mysql .

As soon as I shut down my apache instance the memory comes down to 150 mb.

List of apache modules loaded:

 core_module (static)
 log_config_module (static)
 logio_module (static)
 mpm_worker_module (static)
 http_module (static)
 so_module (static)
 alias_module (shared)
 auth_basic_module (shared)
 authn_file_module (shared)
 authz_default_module (shared)
 authz_groupfile_module (shared)
 authz_host_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 cgid_module (shared)
 deflate_module (shared)
 dir_module (shared)
 env_module (shared)
 mime_module (shared)
 negotiation_module (shared)
 reqtimeout_module (shared)
 rewrite_module (shared)
 setenvif_module (shared)
 status_module (shared)
 passenger_module (shared)

This is the result of cat /proc/meminfo

MemTotal:       946344 kB
MemFree:        336624 kB
Buffers:             0 kB
Cached:              0 kB
SwapCached:          0 kB
Active:              0 kB
Inactive:            0 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       946344 kB
LowFree:        336624 kB
SwapTotal:           0 kB
SwapFree:            0 kB
Dirty:               0 kB
Writeback:           0 kB
AnonPages:           0 kB
Mapped:              0 kB
Slab:                0 kB
PageTables:          0 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:         0 kB
Committed_AS:        0 kB
VmallocTotal:        0 kB
VmallocUsed:         0 kB
VmallocChunk:        0 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

There are a couple reasons for the behavior you are seeing. First:

• ps aux's % is based on resident:

  %mem       %MEM    ratio of the process's resident set size  to the physical memory on the machine, expressed as a percentage. (alias pmem).
  

• free's report includes things other than Active memory (the resident/%MEM from ps) when displaying values

The gist of this is that when you look at free, you are seeing VSZ and other allocations being added up when displaying the total. When you look at the % in ps, you are only seeing a piece of the puzzle.

To verify this you can look at two things, first run:

$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
www      18442  0.0  0.0 519156 65460 ?        Sl   Sep08   1:47 httpd

You should see that the VSZ will be considerably larger than the RSS and if you do the calculations would probably yield a much bigger % if included.

Secondly, you can check the real amount of memory in use by looking at /proc/meminfo directly:

# cat /proc/meminfo 
MemTotal:       99197880 kB
MemFree:        72697684 kB
Buffers:        13813488 kB
Cached:          4841820 kB
SwapCached:          196 kB
Active:          4662952 kB
...

You'll notice that the 'used' column from free is more than just Active(roughly RSS from ps).

The actual explanation of why this is is actually covered in a collection of several books which are a really great read if you are trying to fall asleep, but you can get some basic understanding from a couple articles:

Meminfo: http://www.redhat.com/advice/tips/meminfo.html

ps/virtual and VSZ explained: http://virtualthreads.blogspot.com/2006/02/understanding-memory-usage-on-linux.html

If shutting down Apache releases 450mb (approx half of your memory), then start looking at Apache. Look that the modules that it loads on startup, any related environmentals, and any memory consumed answering requests. You might might a module that is a bit memory heavy or leak in your app. Also look at number of workers at startup and the worker pool size.