Using ServerDensity to monitor a slice

You have a number of options for monitoring your slice. Commercial services like ServerDensity can be easier to set up and maintain than free monitoring applications.


Why ServerDensity?

Fixing problems with a slice can be good, but catching problems before they happen is even better. If you can monitor your slice's resource use over time you can identify some potential problems and compensate for them before they cause downtime.

A commercial monitoring service like ServerDensity can be easier to set up and maintain than a free monitoring package like munin or mrtg. While we encourage you to check all your options, we chose ServerDensity as our subject for an introduction to commercial monitoring. They provide free monitoring for a single slice (useful for many of our customers), and their paid options include access to some spiffy extras like an iPhone app for checking your graphs.

A site like ServerDensity is great for tracking performance over time, but if you simply want to set up alerts to track slice or program uptime it's better to use a service designed for that purpose (like Pingdom).

Prerequisites

To make sure your setup is supported by ServerDensity, check their system requirements page. The agent installation is the only part of this guide that varies according to the distribution running on your slice. The instructions here encompass the distributions we provide images for that are supported by ServerDensity: Ubuntu, Debian, CentOS, Red Hat Enterprise, Fedora, and Gentoo.

You will need to have a Python installation on your slice to run the ServerDensity agent. All of Slicehost's images for the ServerDensity-supported distributions have Python installed by default.

Getting an account

The first step is to get an account with ServerDensity. Direct your web browser to their sign-up page to begin the process.

Create an account name you will use to sign on to their site, and enter your email address. The initial password for your account will be emailed to that address. You will also need to create a host name that will be used to access your reports. If you enter "slicename" for the host, the address you would visit to view your statistics and make changes would be:

https://slicename.serverdensity.com

We recommend making the host address different from your account name for security purposes (no need to make it easy for someone to find out what your account name is, eh?). The host name you enter does not have to match your slice name of course, we're just using "slicename" as a convenient example.

Once the account is created, check your email for the initial password. When you log in you'll be able to add the name of the first slice you want to track, and then you'll get a page with instructions for setting up the agent on your slice. Don't close that page just yet — you'll need some of the information listed there.

Installing the agent

In order to gather data from your slice you need to install ServerDensity's "agent" program. This is a small Python program that runs in the background periodically to send data about the slice's resource use to ServerDensity. The agent consumes very few resources by itself, and sends information about your system and resource use only (it doesn't gather the contents of any of your data files, for instance).

Downoad the agent

To download the agent package from ServerDensity run the following command:

wget http://www.serverdensity.com/downloads/sd-agent.tar.gz

Install the agent files

Once you have the agent package downloaded, run the following commands to unpack the files and then install them to /usr/bin:

tar xvzf sd-agent.tar.gz
sudo mv sd-agent /usr/bin/sd-agent

Editing the configuration

Now that the files are in place you'll need to add information specific to your slice to the agent's configuration file. This is where those instructions come in handy.

If you closed that page despite me telling you not to, you can get back to it via the ServerDensity servers page. For example, if you used "slicename" as your host name, you can go to:

https://slicename.serverdensity.com/servers/

On that page you would click the "Agent" link next to your slice's entry to bring the instructions back up.

With the instructions page in front of you, scroll down until you find the entry for the "agent key". You'll want to copy that.

Next edit the agent's configuration file at:

/usr/bin/sd-agent/config.cfg

By default the contents of that file should look like:

[Main]
sd_url: http://example.serverdensity.com
agent_key: 

apache_status_url: http://www.example.com/server-status/?auto

mysql_server:
mysql_user:
mysql_pass:

nginx_status_url: http://www.example.com/nginx_status

report_anon_stats: yes

The two entries you will definitely want to change are "sd_url" and "agent_key".

For the sd_url entry, change "example" to the host name you entered for your slice. Note that if you prefer that your reported data be encrypted in transit, you can change the "http" at the beginning of the sd_url entry to "https". Using "slicename" as the host name, then, you could change that entry to:

sd_url: https://slicename.serverdensity.com

For the agent_key entry you want to paste in that agent key you found on the instructions page. Once you paste the key in, your entry would look something like:

agent_key: 5514ab444a2b2d61638eaf0904406cdb

The rest of the settings are pretty self-explanatory if you can use them, and unnecessary if you can't. Leave them alone for now (so we can test the agent with the most basic settings first). After a successful test you can check the ServerDensity documentation to make sure each service you run supports status reporting to ServerDensity.

The last line of the configuration file, the "report_anon_stats" setting, is not used yet by the ServerDensity agent. It will control whether or not the agent reports anonymous statistics like your Linux distribution and python version for use in further ServerDensity development. If you do not want the agent to report that information, set this entry to "no".

Adding iptables rules (optional)

If your slice was set up using one of Slicehost's setup articles you won't have outgoing traffic filtered by iptables by default, so you won't need to make any changes to iptables to allow the agent to communicate with ServerDensity.

If you have iptables filtering outgoing traffic you will need to open either port 80 or 443 for the agent's communication, depending on whether you set the sd_url entry to use "http" or "https".

Starting at boot time

To start the ServerDensity agent when your slice reboots you will need to first copy the init script for the agent to /etc/init.d on your system:

sudo cp /usr/bin/sd-agent/sd-agent.init /etc/init.d/sd-agent
sudo chmod 0755 /etc/init.d/sd-agent

The steps you take to enable the init script will depend on the Linux distribution running on your slice.

Ubuntu and Debian

On Ubuntu and Debian systems, run the command:

sudo /usr/sbin/update-rc.d sd-agent defaults

CentOS, Fedora, and RHEL

On RPM-based systems (CentOS, Fedora, and Red Hat Enterprise Linux), run the following commands to enable the init script:

sudo /sbin/chkconfig --add sd-agent
sudo /sbin/chkconfig sd-agent on

Gentoo

For Gentoo systems you enable the sd-agent init script with:

sudo /sbin/rc-update add sd-agent default

Starting the agent

To start the ServerDensity agent, run the command:

sudo /etc/init.d/sd-agent start

Once the agent has started, a process ID file should be created at:

/var/run/sd-agent.pid

If that file doesn't exist the agent did not start properly, and you may need to double-check the permissions on the init script or the settings in your configuration file.

Viewing the reports

Once the agent has started you'll need to wait about five minutes to make sure the agent collects some data and reports to ServerDensity. I know, waiting's not fun, but the agent only collects data periodically, and you'll want at least one collection in order to initialize the reporting screen. Use the time to plot a practical joke on your co-workers or to start a load of laundry, depending on whether you're setting this up from an office or from home. Or if you're in a friend's home, offer to do their laundry, it would be a nice way to pay them back for letting you use their Internet connection.

Now that a little time has passed, go to your dashboard using the address you set up when you created your account. If you used the hostname "slicename", that would be:

https://slicename.serverdensity.com/

You should see your slice on the list of servers, and be able to click it to see more information. That link would be something like:

https://slicename.serverdensity.com/servers/view/1/

There won't be much of a graph there yet, since it takes a while for the agent to report back enough data to make much of a line. If you don't see any reports at all you may need to recheck your agent configuration and make sure the agent is running.

Further information

Your account should be collecting data from the agent running on your slice now, and you'll eventually be able to see how your slice uses system resources like memory and CPU over time. Good work! Where do you go from here?

For starters, to get more details on using the ServerDensity service, as well as information on setting up status reports for apache, nginx, and mySQL, visit the ServerDensity documentation page. But some useful bits to keep in mind...

Trial account

It's important to bear in mind that your ServerDensity account starts as a trial of their paid service. If you want to continue using the service for a single slice, you'll need to downgrade your trial account to a free account before the trial period expires. If you decide you want to continue on the paid service (to get reporting for multiple slices, or to use their iPhone app, for example), you will need to upgrade your account before the end of the trial period.

Adding more slices

If you want to set up reporting for more than one slice you'll need to set up a copy of the ServerDensity agent on each slice. This means going through the installation and configuration in this guide for each slice - there isn't really a practical way to get around that.

Fortunately if you have several slices you can at least make the process of setting the slices up on your dashboard a little easier by using the "Add Cloud" option on ServerDensity's "Add Server" interface. You will need to add your Slicehost API key to your ServerDensity account (you can find it in the Slice Manager's account settings, after enabling API access), and then you'll be able to select from a list of your slices which ones you'll be adding to the monitoring service. Once they are created in the dashboard you can get agent keys for each from their "Agent" pages.

Updating the agent

You can check for agent updates and install them with the following commands:

cd /usr/bin/sd-agent
sudo python agent.py update
sudo python agent.py restart

Summary

Your ServerDensity account should be set up and recording data for your slice. Keep an eye on the graphs it generates and watch for trends like increased traffic or low memory conditions, then adjust your application configurations or slice size accordingly. You can also poke around in the ServerDensity configuration screens to set up email alerts and what data is reported in your dashboard.

  • -- Jered

Article Comments:

cc commented Fri Mar 05 10:33:59 UTC 2010:

What advantage does this have over cacti?

Jered commented Fri Mar 05 17:42:47 UTC 2010:

At a guess, the setup for ServerDensity might be a bit easier, but you'd need to compare the output of both to see if one has a clear advantage over the other (I haven't tried cacti myself). Odds are that if you're using cacti and like it, there's no compelling reason to switch.

David M commented Mon Mar 08 11:43:05 UTC 2010:

Ease of setup is one of the advantages over cacti, but there are a few others worth mentioning:

  • You don't have to mess around with hosting your monitoring elsewhere

  • Alerting by e-mail, SMS and iPhone push notification

  • iPhone app

  • Server snapshot functionality allows you to see a point in time snapshot of what was running on your server. See http://www.serverdensity.com/img/screenshots/serverSnapshot-large.png

  • Statistical analysis picks out anomalies based on past behaviour so you can detect problems e.g. a failed backup job

  • Individual process monitoring

Mir commented Tue Mar 23 05:26:59 UTC 2010:

Looks very interesting, I'll be sure to give it a try when I get a chance.

Want to comment?


(not made public)

(optional)

(use plain text or Markdown syntax)