LOAD - Load Generator - Implementation

Base Class

LOAD is branched from the WEB5 base class.

Class Volumes

The volume configuration for LOAD is determined by the base class image from which it is created.

Volumes for release 1.0 of LOAD include:

• boot volume, 160 MB size, mounted as "/", writeable, instantiable.
• usr volume, 430 MB size, mounted as "/usr", read-only, common.
• config volume, mounted as "/mnt/config", writeable, placeholder.

The boot volume is unmodified from the WEB5 appliance. The usr volume is enlarged to 420MB. The content volume is removed and /etc/fstab is modified to additionally mount the config volume. All software specific to LOAD is installed in /usr/LOAD and requires apache, PHP, and bash in order to function.

Packages

• Main: 3tapp-load-1.0.0-1.noarch.rpm. This contains the scripts and web interface which perform the basic operations of LOAD. All files are installed in /usr/LOAD.

Contents of the release directory:

Exported appliance image:

• LOAD-1.0.0exp.tar.bz2

"Generator" RPMs: install this on the usr volume of LOAD to re-create the appliance:

• 3tapp-load-1.0.0-1.noarch.rpm

Source RPMs:

• None. All sources are PHP or Javascript and are available in the "Generator" RPM.

Appliance related customization files:

• 3tapp-load_custom-1.0.0.tar.bz2

Tests (see the test plan CatMiscLoadTp and the release notes CatMiscLoadRelNotes):

• load-test-app.tar.bz2 the test application

Re-creating the Appliance

To recreate the LOAD appliance:

• Branch a WEB5 appliance
• Set its interfaces, volumes, resources and properties according to the LOAD Data Sheet
• Temporarily mark the usr volume as rw, not shared
• Re-size the usr volume to 430 MB
• Set mandatory properties, connect mandatory terminals, and start in debug mode
• Install the rpm 3tapp-load-1.0.0-1.noarch.rpm on LOAD
• Install the appliance related customization files
• Set the usr volume to ro, shared and move the singleton into the misc catalog

Installation of Appliance Related Customization Files

These appliance related scripts have been added to, or modified from, those on the original WEB5 base class:

• /appliance/appl_dyn.pl modified to set Apache max clients to 32
• /appliance/appliance.sh modified to call load_setup.sh
• /appliance/httpd_prerun.sh modified to link /mnt/content to /usr/LOAD/html and to use the ctl terminal
• /appliance/load_setup.sh created to handle directories and mounts
• /etc/ccad.conf custom counter definitions
• /etc/fstab modified to mount the config volume and not any content volume
• /etc/sysctl.conf modified to increase available file handles and set fast tcp recovery mode

To install these scripts, copy the file 3tapp-load_custom-1.0.0.tar.bz2 to / on the appliance, and:

           cd /
           tar -jxf 3tapp-load_custom-1.0.0.tar.bz2
           rm 3tapp-load_custom-1.0.0.tar.bz2

Third Party Software

LOAD includes http_load in binary form. http_load is used to generate HTTP requests through the out terminal. The version used is http_load-12mar2006. http_load can be downloaded at http://www.acme.com/software/http_load/http_load-12mar2006.tar.gz and is released under these conditions:

Copyright © 1998,1999,2001 by Jef Poskanzer <jef@mail.NOSPAM.acme.com>.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.

The above notice is included in the appliance as /usr/LOAD/http_load.license.

Theory of Operation

Basic operations performed by LOAD are:

• Generate HTTP requests on the out terminal
• Expose a GUI allowing users to configure load generation
• Display an historical graph of load generation results

http_load accepts these parameters:

• rate the number of requests per second to make
• seconds the number of seconds to continue generating these requests
• file a file listing the URL's to be requested. If more than one URL is present in the file, URL's are selected randomly.

load_daemon

• initialize results to "nodata"
• Read the load rate from a file on the config volume
• Read a list of URL's from a file on the config volume and re-write these in a form usable by http_load
• If either the rate file or the URL file is missing or cannot be parsed, sleep 5 seconds. Otherwise:
  • If necessary, break the rate into smaller values which target the stepping points of the http_load timer.
  • Execute as many http_load processes as required to generate the selected rate for 5 seconds
  • Parse the output to form results
• write results to a file on the config volume
• add results to an array of historical values
• fork a child to generate .png images for the GUI. This graph covers the previous 72.5 minutes of load generated where each pixel equates to 5 seconds. The graph includes:
  • a graph of requests per second selected by the user
  • a graph of fetches per second performed
  • a graph of bytes per second fetched
• The graph also provides some indication of bad HTTP response codes. For example, if the results related to a given pixel include HTTP code 503, then some of the fetches failed. This is indicated by using a red pixel in the fetches/sec graph.
• feed custom counters to ccad

On-start Script

• start load_daemon
• create the root crontab
• delete the rate file from the config volume, effectively setting the rate to 0

Cron

• Checks that load_daemon is running. If not, then it is re-started

Logs

GUI

• Authentication upon initial access
• Notify the user if the URL file is not yet created
• Provide a slider control for selecting the rate. If the slider is moved, AJAX is used to update the rate file on the server.
• Periodically use AJAX fetch the results of the most recent interval of load generation and write them to the DOM. These results include:
  • fetches/second the number of pages per second fetched.
  • bytes/second fetched average number of bytes per second fetched
  • mean bytes/connection mean number of bytes fetched per connection
  • max parallel maximum number of parallel requests running
  • msec/connection mean, maximum and minimum number of milliseconds to complete a fetch once the first-response has been received
  • msec/first-response mean, maximum and minimum number of milliseconds before the first response to a request
  • HTTP response codes a list of HTTP response codes and the number of fetches per second returning each.
• Periodically write the graph to the DOM

Configuration

Configuration is performed by scripts in /appliance as well as by the start-up script /var/www/html/.rc.local.

Log Files

There are no log files generated by LOAD, aside from those generated by the WEB5 base class from which it is branched.

-- StephenQ - 09 Jan 2008