r17 - 20 Dec 2007 - 22:02:59 - BeckyHYou are here: Wiki >

December 24 - AppLogic 2.4.7 is now available and is the latest production release!

Session-aware HTTP Load Balancer

Latest version: 2.2.2 - Release Notes

	At a Glance
	Catalog	System
	Category	Switches
	User volumes	no
	Min. memory	48M
	OS	Linux
	Constraints	no
	Questions/Comments	Ask Forum

HLB is a switch for distributing workload to multiple Web servers of the same type. Its operation is transparent both to the client and to the servers - each client sees it as one server and each server sees incoming requests as if they came directly from a client.

Incoming HTTP requests are checked for association with a multi-request session and routed to one of the servers, as follows:

Requests not associated with any session (not having the session cookie present) are distributed using simple round-robin selection.
Requests that carry a session cookie are directed to the same server as the one that issued that cookie in the first place, if that server is alive; otherwise the next server from the round-robin schedule is chosen (note that taking down a server at runtime will break the client's session and require a re-login; the same will happen if the server dies on its own). The cookies are expired if replaced/deleted upon server request or upon a configurable timeout.

For the cases where two different servers connected to HLB can issue the same exact session cookie, HLB can be programmed to insert a terminal ID into the cookie data before sending the response to the client, and strip the terminal ID from client requests, to restore the cookie to its original state before sending the request on to a server.

In addition to monitoring server-defined session cookies, HLB can issue a cookie of its own to support associating each client with a single back-end server, even if the server does not use session cookies or does not have any notion of a 'session' at all.

HLB monitors closely the response from the servers connected to its outputs and stops directing traffic to an output as soon as it detects that there is no response from that output. The incoming traffic is re-distributed to the outputs that are alive, just as if the dead output was not connected at all. HLB continues to probe the disabled outputs and re-instates traffic as soon as it gets a response.

Boundary

Resources

Resource	Minimum	Maximum	Default
CPU	0.05	4	0.10
Memory	48 MB	256 MB	48 MB
Bandwidth	1 Mbps	100 Mbps	1 Mbps

Note: the amount of memory given to HLB does not affect its throughput. Use more memory only to support more concurrent requests that have been forwarded to a back end server, pending a request, if the back-end servers hold up requests for excessively long times.

Terminals

name	dir	prot.	description
`in`	in	HTTP	common input. HTTP requests sent to 'in' are directed to one of the outputs, either using round-robin selection or session information embedded in the request as a cookie (see the Properties)
`out1-out8`	out	HTTP	balanced outputs. Any and all of these outputs can be left unconnected; traffic will be distributed to the connected ones only.
`mon`	out	cce	Sends performance and resource usage statistics.

Properties

name	type	description
`cookie_name`	string	the name of the cookie used to identify a session. For the passive modes ( `passive` and `sync` - see the `mode` property below), this is the name of the cookie used by the back-end servers connected to `out1..out8` to identify client sessions. For the active ( `insert` ) mode, this is the name of the cookie that HLB should insert into HTTP responses in order to make each client 'stick' with a single server. If this property is set to the empty value, no session tracking is done and all requests are distributed in a simple round-robin fashion. Default: (empty).
`mode`	string	Specifies how to use the named session cookie for session identification. Valid values are: `passive` - the cookie is not modified, it is expected that a given cookie value is always unique and the same value will never be used by another server. `synch` - the cookie string in the HTTP response from a server is modified to make it unique to the terminal from which the response came (by adding a 1-character terminal ID into the cookie value). The 'terminal ID' is stripped before a request is forwarded to a server on one of the `outX` terminals. Other than inserting the terminal ID, the mapping between cookie values and output terminals is the same as for `passive` - i.e., the entire cookie value is compared. `insert` - the load balancer itself inserts a cookie into responses sent back to the client on `in` , so when the client sends subsequent requests with that cookie, they are directed to the same server as the first request from that client. The cookie inserted by HLB is without an expiration date, meaning that it is not to be permanently saved by the client software.
`timeout`	integer	Optional timeout (in seconds) to expire inactive sessions. If this is set to a non-zero value, a session that has been inactive for more than the value of this property in seconds is 'forgotten'. Further requests bearing the 'forgotten' cookie are treated as if they have no cookie at all and are directed to to a random server, using the usual round-robin method. Default: 0
`max_connections`	integer	If set, this specifies the maximum number of concurrently active connections that the load balancer will handle. When this number is reached, new connections are still accepted, but their processing is delayed until another connection is closed. the maximum allowed value for this property is 500. If set to 0 (the default), HLB automatically determines the maximum based on the available system memory. Note that neither the available memory nor an explicit setting of this property have a direct effect on the balancer's throughput or its maximum request rate - setting a low number (or having little memory) will affect response only if the back-end servers are performing lenghty operations for each request (e.g., database searches), causing many requests to remain open at the same time.
`cookie_path`	string	The URL path to use in the HLB-generated session cookies (when the `mode` property is set to `insert` ). If HLB is sitting behind a URL switch and serving only a subset of the namespace, this property may be set to avoid having the client send the same cookie for pages not served through HLB, e.g., if HLB is used to load-balance a bank of servers doing nothing bug CGI scripts, in the `/cgi-bin/` directory, cookie_path should be set to `/cgi_bin` . The value of this property should normally have a leading "/" - if it is missing, it is added automatically. Default: "/"
`cookie_domain`	string	This optional property defines the domain name to use in the HLB-generated session cookies (when the `mode` property is set to `insert` ). By default, if this property is not set and HLB does not send a 'domain=..' property with the cookie at all. This is the recommended behavior in most cases, as the client will remember the cookie as associated with whatever domain it used to contact the service provided through HLB. Note that setting a domain name that does not match what a client's browser would be using to contact the server can have negative consequences, including: (a) the cookie being sent to a different server, but not to HLB; (b) the cookie being rejected altogether, if the browser has a restrictive policy in place. Default: (empty)

Performance

Request Rate

HLB routes no less than 2000 transactions (request/response pairs) per second, when operating on a single 2GHz CPU.

Data Throughput

HLB routes no less than 10 mbytes / second, on a 3K average packet size (request and response).

Concurrent Connections

When given its maximum memory requirement, HLB should be able to handle no less than 500 concurrently pending requests. (A "pending request" being an open TCP connection from the client, on which there is one or more un-completedHTTP request in progress).

Typical Usage

Load-balancing Slow Dynamic-content Applications

If your application requires a lot of computation to produce each page, it will naturally be able to serve a limited amount of clients concurrently. However, if the data source is fast enough or can be replicated and served from multiple servers, HLB can be used to increase the number of connections that can be handled simultaneously. A typical example may be serving maps from a carthography database - the database is fast to read, but each pageload requires the map to be rendered as an image file.

The diagram below shows how HLB can be used to employ multiple CPUs to serve the same content, providing faster response under heavy load.

Typical Usage Diagram

Configuration for a Server Keeping Sessions - Bugzilla

Note: the configuration examples list only properties set to non-default values.

Property	Value	Notes
`cookie_name`	Bugzilla_logincookie	This is the name used by Bugzilla for its login cookie
`mode`	passive	Bugzilla issues unique cookies for each username/IP address combination, therefore two bugzilla instances that are connected to the same database will never return the same cookie. It is thus safe to use the simplest `passive` mode. For other server types, the `sync` setting may be more appropriate.
`timeout`	0	Bugzilla does not expire its cookies, so we follow the tradition and set no timeout on the sessions.

"Sticky" Load-balancing for Session-less Servers

For servers that do not have sessions, but keep some cached data and could benefit from a client being consistently directed to the same server, passive cookie monitoring will not work as there's no cookie to monitor. The insert mode is used in this case, as shown in the table:

Property	Value	Notes
`cookie_name`	session_id	Arbitrary name, but make sure it does not match a name used by the servers for something else.
`mode`	insert	HLB will insert a Cookie-set: header
`timeout`	300	Don't keep the client stuck with the same server for too long, if it is inactive. This should typically be set to match the expiration time of whatever cache the servers have that we're trying to take advantage of by the "sticky" load-balancing.
`cookie_path`	/movies	Make the client "stick" to a specific server only for the `/movies` sub-directory

Comments on this appliance can be found below and edited here? .

Notes and Links

Notes

HLB does not support HTTP 0.9 clients. Only HTTP 1.0 and HTTP 1.1 are supported. Note that there is little benefit in load-balancing at the HTTP level for 0.9 clients, as the 0.9 version of the protocol does not support cookies.
Although the load-balancing software used in HLB (Pound) can be configured to log each incoming request in the system log, this functionality is disabled. HLB is intended for very high load applications and logging thousands of requests per second will quickly overrun the limited log file space available on the appliance.