URLSW - Url Port Switch - Test Plan

Preparation

Host platform

These tests must be launched from a remote host, that has an SSH access to the AppLogic controller.

Tools

All necessary tools are supplied within the test suite. You need SSH installed and ssh keyfile for your AppLogic grid to perform the tests.

ab

source: http://mirror.centos.org/centos/5.2/os/SRPMS/httpd-2.2.3-11.el5_1.centos.3.src.rpm

RPM: http://mirror.centos.org/centos/5.2/os/i386/CentOS/httpd-2.2.3-11.el5_1.centos.3.i386.rpm

Tests Summary

Configuration Tests

• Configuration propagation test: verify the nginx config files are updated according to URLSW configuration
• Client connection failure test: verifies that URLSW can be accessed only through the in terminal

Logging Tests

• Log file tests: verifies log path, log lifecycle

Functional Tests

• ctl terminal test: verifies ability to changing configuration via ctl terminal
• mon terminal test: Verifies that URLSW reports correct statistics counters to mon appliance.

Resource Tests

• CPU/Memory resource tests: tests URLSW performance under various CPU/Memory configurations.

Endurance Tests

• Endurance test: verifies that appliance can work long time under high load
• Start-Stop test: verifies that appliance can be started and shut down multiple times

Benchmarks

• Benchmark: verifies the maximum URLSW performance

Running the Tests

To run the tests on AppLogic2.AppLogic grid, copy and un-compress the tests archive file to a directory of your choice. Launch the test suite running the urlsw-test.sh file. Test suite imports the test application, or prompt you to use an existing, created previously, one.

Example test aplication:

If you choose to create a new test application, the test suite will import a new application and configure it, assuming that the URLSW instance that you want to test is located in the /proto/ catalog and in named URLSW. If you want to test another instance of URLSW, you can either copy it to the /proto/ catalog or open the test suite application in AppLogic GUI and manually drag the necessary URLSW instance over an existing one.

Design

Structure

• LINUX5 appliance used as client for tests
• backend1, backend2 and backend3 (all based on WEB5 appliance) used as backend web servers
• PS8 appliance used to connecting to in and ctl terminal of URLSW and aux terminal of mon appliance
• NAS appliance used for log storage (via cifs terminal)
• MON appliance used for monitoring statistics

The test script is fully automated and includes the ability to install the necessary test software on the LINUX5 instances. The user does not need to install any software on any of the appliances; the test script takes care of everything.

Test Details

• For successful operation, test suite must be the only application running on the grid, or some tests, mainly performance ones, may fail.

• Some tests, especially resource-intensive ones, may fail if you do not have enough resources available. This is not a URLSW fault.

• Most of the tests should not be interrupted by a user. If it happened that you had to interrupt the test, stop the test application with "application stop" before proceeding to the next test.

Configuration propagation test

Reconfigures URLSW with different settings and verifies that all settings in nginx configuration files are updated accordingly

Test changes multiple properties of URLSW, launches the application, verifies successful changes in nginx's config files and verifies that URLSW is operational with these settings. Every possible value of every property is tested.

Client connection failure test

Verifies that URLSW is accessible only through the "in" terminal

This test tries to connect to URLSW through other than "in" terminals. All attempts must fail.

Log file test

Verifies that URLSW can write access logs for debugging or statistical purposes

Monitoring subsystem functionality test

Verifies that URLSW reports correct statistics counters to mon appliance.

CPU/Memory resource tests

Tests URLSW performance under various memory, cpu and bandwidth resources available

This test detects the amount of memory, cpu and bandwidth resources available on the grid, and tests the URLSW performance with all possible combinations to the limits imposed by resource availability. URLSW is tested with the following settings:

• Memory: 64Mb and 128Mb
• CPU: 0.10, 0.20, 0.40, 0.50, 1, 1.5
• Bandwidth: 1Mbit, 10Mbits, 100MBits, 200MBits, 500MBits, 1Gbit, 1.5GBits, 2Gbits
• 100 clients, 1000 transactions

Client LINUX5 appliance is configured with 512Mb of memory, 1 cpu and 500Mbit of bandwidth for this test. This test may fail if there are not enough free resources available on the grid to run all required appliances.

Benchmark

Verifies maximum URLSW performance

For this test URLSW is configured with 2 cpus, 500Mbits of bandwidth. Then the ab test is used to verify URLSW performance with 1,10,100 clients.

This test may fail at some point when there are not enough free resources available. This is not a URLSW fault.

Start-Stop test of the URLSW appliance

Verifies that URLSW could be safely started and stopped multiple times.

This test verifies that the URLSW can be successfully started and stopped multiple times. It is an important test that verifies that all log rotation is performed properly. Amount of start/stop cycles may be specified and be unlimited (in this case the test can be interrupted using control-c).

Endurance test of the URLSW appliance

Verifies that URLSW could work on heavy load during long time periods

Perform ALL Tests

Run all important tests in automated mode without any details except success/failure messages.

-- AndriyMayevskyy - 15 Jan 2009