1.4.all

monitoring the real user experience

The Problem You may know a lot of site monitoring tools like Nagios, BigBrother, Cacti etc. But none of them can measure a near real user experience. We often have...

The Problem

You may know a lot of site monitoring tools like Nagios, BigBrother, Cacti etc. But none of them can measure a near real user experience. We often have the case that someone is complaining about slow page loading. But all of your monitorings and measurement tools show normal performance of the site. Most of the tools are capable of fetching urls. But none of the tools I know behaves like a real browser. Thus doing the real loading of the page, all it’s elements and includes and rendering the page in fact.

But with a little scripting and a Linux system you can do just that.

The Solution Idea

So we like to have a site monitored as realistic as possible from a users point of view. Why not use a real browser to monitor the site performance? The trick is, to use Firefox to render a page and simply measure the time it needs to complete.

Why Firefox you ask? Because Firefox is free. You can control it via the command line (scriptable) and it got some usefull add-ons we’ll need. And – it’s very popular.

The solution itself

To run firefox you need a graphical user interface e.q. the X11/X86org environment. But this is not what we want for automatic continous measures. So we need a way to run Firefox in headless mode. Unfortunatly Firefox itself has no option to run it without a valid DISPLAY. This is where Xvfb comes in handy. This is a virtual frame buffer daemon for Linux. It simulates a X-Window server but doesn’t need a real/physical graphical output system. So you can run this on your server without a video card and tell Firefox this is your X-Server.

You can even run your normal X-Server and the Xvfb in parallel on your system. Just start Xvfb on a different Port:

/usr/bin/Xvfb :2

Then tell Firefox where the DISPLAY is:

firefox –display=127.0.0.1:2.0

But first we have to pimp Firefox a little bit. When you open Firefox via the command line and give it a URL this Page will be opened and rendered. But to measure the time for that in a script we need Firefox to close itself after successfull rendering. Then we can measure the whole time with the systems “time” command. OK, this will also count the startup and shutdown time of Firefox. But these times should be almost constant on your system. So if you like, you can measure these times in more detail and substract them from the page render times.

To have Firefox close after rendering the page and to have a proof that the page was rendered correct we use the PageSaver Add-On to Firefox. This add-on enhances Firefox to take screenshots of the whole page and save it to a jpeg file. Even more, you can control this add-on from the command line as well.

Put all this together and you have a browser which can be startet on a headless system (with Xvfb), given an url to render, renders the page, saves the rendered page as an image and closes. Perfect!

Now put a little shell scripting around and put the measured times into an RRD file, generate graphics with rrdool and you get a nice near real site performance monitoring.

This is my dirty script: make_screenshot_new.sh
The script to create the RRD files: create_screenshot_rrd.sh
The script to generate the graphs: make_graphs.sh

After creating the RRD file your can call the make_screenshot_new.sh periodically via Cron to measure the render times. But make sure the interval correspond with the interval you set in the RRD file

• December 3rd, 2008
• Tagged with: Cacti • digital • Firefox • Klugscheissen • Monitoring • Nagios • RRDTool
  
• 1 Comment