NET - Subnet Output Gateway Appliance - Test Plan

Preparation

Host platform

The automated tests described here for the NET appliance are designed to run on a Linux system and are verified to work on CentOS 5. They don't need anything CentOS-specific and should work on other Linux distros.

These tests must be launched from a remote host, that has an SSH access to the AppLogic controller.

Tools

All necessary tools are supplied within the test suite. You need SSH installed and ssh-key file for your AppLogic grid to perform the tests.

Tests Summary

Configuration Tests

• Configuration propagation test: verify that configuration files are updated according to NET configuration
• Memory management test: test with different memory configurations

Functional Tests

• Net/host access test: verifies ability to access to allowed hosts only

Resource Tests

• CPU/Memory resource tests: tests NET performance under various CPU/Memory configurations.

Endurance Tests

• Start-Stop test: verifies that appliance can be started and shut down multiple times

Running the Tests

To run the tests on AppLogic2.AppLogic grid, copy and un-compress the tests archive file to a directory of your choice. Launch the test suite running the in-test.sh file. Test suite imports the test application, or prompt you to use an existing, created previously, one.

Example test aplication:

Design

Structure

• client appliance (based on LINUX5) used for testing NET functionality

The test script is fully automated and includes the ability to install the necessary test software on the LINUX5 instances. The user does not need to install any software on any of the appliances; the test script takes care of everything.

Test Details

• For successful operation, test suite must be the only application running on the grid, or some tests, mainly performance ones, may fail.

• Some tests, especially resource-intensive ones, may fail if you do not have enough resources available. This is not a NET fault.

• Most of the tests should not be interrupted by a user. If it happened that you had to interrupt the test, stop the test application with "application stop" before proceeding to the next test.

Configuration propagation test

Reconfigures NET with different settings and verifies that all settings in its configuration files are updated accordingly

Test changes multiple properties of NET, launches the application, verifies successful changes in configuration and verifies that NET is operational with these settings. Every possible value of every property is tested.

Memory management test

Tests NET configuration under various memory resources available

Test detects the amount of free memory available on the grid where the test application is running, and verifies proper updates to the Java memory settings values. NET is tested with 64Mb, 128Mb, 256Mb, 384Mb, 512Mb, 1Gb, 1.5Gb, 2Gb, 3Gb, 4Gb, 8Gb, 16Gb, 32Gb of memory, subject to available free memory on the grid. Any mismatches between reported property values and reference values are reported.

This test may fail at some point when there are not enough free resources available. This is not a NET fault.

Net/host access test

Verifies NET firewall operation. Test NET to serve requests to allowed host (or subnet) only. Also, verifies that requests to hosts listed under denied_hosts are prohibited.

Start-Stop test

Verifies that NET could be safely started and stopped multiple times.

This test verifies that the NET can be successfully started and stopped multiple times. It is an important test that verifies that all log rotation is performed properly. Amount of start/stop cycles may be specified and be unlimited (in this case the test can be interrupted using control-c).

Perform ALL Tests

Run all important tests in automated mode without any details except success/failure messages.

-- OlegSmolov - 13 May 2009