Memory management with free

Basic monitoring should be done on a regular basis. This keeps you informed as to the general condition of your server and may warn of impending problems.

This article concentrates on memory management using the command 'free'.


Memory management is actually very easy to do. What you do about any problem is a different matter and we'll look at monitoring what is actually using the memory and how much it's using in a later article.

Free

To start with, log into your server and give the 'free' command:

free

The output consists of four lines:

Well, for a start, that's all a bit confusing as the answer is in Kb. I don't know about you but dividing the reported figures by 1024 is a bit of a pain.

Free -m

Let's make it more readable by adding the '-m' option which will displays the output in megabytes:

free -m

That's better:

One of the aliases in my .bash_profile is:

alias free="free -m"

So I only have to enter 'free' to get a readable output.

Buffers and Cache

The first line gives the column headers: total, used, free and so on.

A common mistake is to read the second line and have a panic attack as that line reports:

.                total       used       free     shared    buffers     cached
Mem:         254        249          5          0             15             181

However, the figure of 249MB out of 254MB includes cached and buffered memory.

Unless you are particularly interested in the buffers and cached figures, ignore this line.

Actual Usage

Concentrate on the third line:

.                total       used       free     shared    buffers     cached
Mem:         254        249          5          0         15        181
-/+ buffers/cache:    51         202

This tells us what memory is being used and is not available for another process.

So it reports 51MB of 254MB being used by various tasks and applications. I have 202MB to play with and for my application to grow into.

Not bad, but then I don't have anything installed on this test slice - this was just after a base install and update.

Swap

The final line tells me how much swap space I am using:

.                total       used       free     shared    buffers     cached
Mem:         254        249          5          0         15        181
-/+ buffers/cache:    51         202
Swap:        511          0          511

As you can see, I have 511MB of swap available and have used none of it.

If you are using swap space then you need to investigate what is using so much memory that it had to resort to hard drive swapping to work. Often it's a case of your application 'outgrowing' your Slice and you may need to upgrade it to include more RAM.

The odd bit of swapping is fine as it is, after all, there to be used. However, consistent swapping or very high levels of swapping does mean your Slice is struggling and it will make your web application slow down.

Regular monitoring

Actually, that's a good example as to why regular monitoring of your Slice is a good idea - your site may be up and running but it may be struggling and becoming slower and slower.

It's a lot better to catch any issues before they result in downtime and a non-operational site.

That's it - quick and easy. Next time we'll look at system monitoring with 'top' which will pinpoint what processes are actually using the memory and, more importantly, how much each process is using.

PickledOnion.

Article Comments:

Milan Andric commented Wed Oct 03 20:47:05 UTC 2007:

Here's an example monitor script to let you know when swap is in use.

http://parsed.org/tip/388/

Paul commented Wed Feb 13 01:44:22 UTC 2008:

Milan, thanks for the script link. It works perfectly on my ubuntu slice, without mods, once I apt-getted mailx.

Kirill Maximov commented Tue Mar 25 20:00:27 UTC 2008:

I really like monit utility to monitor my server, including site load, process memory consumption, process and service availability control and much more stuff.

Jeff O'Hara commented Thu May 22 17:10:44 UTC 2008:

Don't forget the ever useful top command to watch processes in realtime.

-Jeff

G commented Wed May 28 05:34:23 UTC 2008:

Don't forget the even prettier htop command to watch processes like top but with colour. :D http://htop.sourceforge.net/

Ben Wilson commented Sat Oct 18 00:39:56 UTC 2008:

You mean :aptitude install htop. :) Nice tool. Thanks.

ed commented Sun Oct 19 05:40:05 UTC 2008:

Is there any way to add more swap space ?

Nathan commented Sat Oct 25 23:56:27 UTC 2008:

Ed, you could do something like this to add 512 MB of swap:

dd if=/dev/zero of=/swapfile bs=1M count=512 swapon /swapfile

You will need to add the swapon command to your startup scripts if you intend to use the extra swap space long term. Of course, swapoff will disable it if you need to as well.

Nathan commented Sat Oct 25 23:58:17 UTC 2008:

Apparently my carriage return in the comment box wasn't interpreted as such. In the above comment, the dd and swapon commands are separate. I'll try again:

dd if=/dev/zero of=/swapfile bs=1M count=512

swapon /swapfile

PickledOnion commented Mon Oct 27 13:38:16 UTC 2008:

ed,

The absolute last thing you want to do is add more swap space.

If you are using all your swap, adding more will NOT help. It means that something is using all the RAM and all the resouces and having to resort to using the HDD as RAM.

Please don't add more swap, it will simply make things worse if you have a Slice that needs more resources.

PickledOnion

xian commented Fri Feb 20 04:33:55 UTC 2009:

Has anyone successfully used compcache on a slice? I've seen it do wonders for laptops with 256m ram (like OLPC XO, which is where i originally ran into it...)

it's not rock-solid yet, and it's kernel space, which makes crashes pretty bad, but it's a pretty amazing experimental component.

http://code.google.com/p/compcache/

Nick commented Sat Jul 04 19:34:26 UTC 2009:

This link no longer works. Does anyone have the script still? Thanks

http://parsed.org/tip/388/

Francisco Costa commented Thu Jul 09 06:53:50 UTC 2009:

Hello from Portugal =)

When you eant to clear your swap and you want everything that has been swapped out to be placedback in to physical RAM, then the quickest way is to do:

sudo swapoff -a sudo swapon -a

Turning the swap off will automatically move all swapped pages back in to physical RAM (if you've got enough of it).

Dan commented Tue Oct 06 17:24:47 UTC 2009:

Used the wayback machine, heres the script:

!/bin/bash

Notify me one time if my swap is over MAXSWAP and log the

swap usage as well in SWAPLOG.

Usage: ./monitor.sh &

SWAPLOG=~/logs/monitor/swap MAXSWAP=5 INTERVAL=30

sendsmsmsg () { if [ "$1" ]; then STRING=$1; fi if [ $SMS_SENT ]; then return 0 else echo $STRING | mailx -s 'monitor msg' 5101234567@cingularme.com SMS_SENT=1 fi }

while true; do # parse free output, get current swap value swaptest=free -m |grep Swap|perl -pe 's/Swap:\s+\S+\s+(\S+).*/$1/' if [ $swaptest -ge $MAXSWAP ]; then echo date Swap is: $swaptest >> $SWAPLOG sendsmsmsg "Swap is: $swaptest" fi; sleep $INTERVAL done;

Dan commented Tue Oct 06 17:27:06 UTC 2009:

#!/bin/bash

Notify me one time if my swap is over MAXSWAP and log the

swap usage as well in SWAPLOG.

Usage: ./monitor.sh &

SWAPLOG=~/logs/monitor/swap MAXSWAP=5 INTERVAL=30

sendsmsmsg () { if [ "$1" ]; then STRING=$1; fi if [ $SMS_SENT ]; then return 0 else echo $STRING | mailx -s 'monitor msg' 5101234567@cingularme.com SMS_SENT=1 fi }

while true; do # parse free output, get current swap value swaptest=free -m |grep Swap|perl -pe 's/Swap:\s+\S+\s+(\S+).*/$1/' if [ $swaptest -ge $MAXSWAP ]; then echo date Swap is: $swaptest >> $SWAPLOG sendsmsmsg "Swap is: $swaptest" fi; sleep $INTERVAL done;

Dan commented Tue Oct 06 17:29:13 UTC 2009:

I obviously don't know how to post code on here, so heres the link

http://web.archive.org/web/20080410221917/http://parsed.org/tip/388/

Adrià commented Tue Nov 17 17:15:28 UTC 2009:

I have a new installation, and i have my siteonline.

With free -m command i have this result

         total       used       free     shared    buffers     cached

Mem: 256 253 2 0 0 7 -/+ buffers/cache: 245 10 Swap: 511 370 141

So, ok ,I see there's something wrong with the buffers/cache, but I don't know why and what is causing this.

How can I see what is consuming the machine resources? I run the top command but I can't see any problems.

Any help

BC commented Thu Dec 31 06:11:36 UTC 2009:

Two quick questions and a comment:

  1. How much swap usage is considered "very high levels of swapping"?

Based on the monitor.sh script 5M is too much. Is there a general consensus on this matter? A percentage of your RAM, an absolute value, or is any swap too much swap?

  1. How would one monitor "consistent swapping"?

Comment: vmstat is nice for looking at memory usage as well. try: vmstat -S M 10

Want to comment?


(not made public)

(optional)

(use plain text or Markdown syntax)