r14 - 01 May 2008 - 12:03:00 - BeckyHYou are here: Wiki >  AppLogic23 Web > CatWebserversWebx8
ALERT! AppLogic 2.3 Beta Documentation The latest production release is AppLogic 2.4.7

WEBx8: Scalable Web Server

Latest version: 1.1.0

webx8.PNG At a Glance
Catalog System
Category Web Servers
User volumes yes
Min. memory 1.046 GB
OS Linux
Constraints no
Questions/Comments Ask Forum

Functional Overview

WEBx8 is a scalable web server that consists of a single load balancer (HLB) and eight web servers (WEB5).

WEBx8 serves static web content and executes scripts from a user-configurable read-only content volume. The paths to the documents and scripts are configurable, so that the same volume can be shared between multiple web servers and/or other appliances serving different content.

WEBx8 has three generic output terminals intended for accessing external services from scripts on the content volume. The db terminal is for accessing a MySQL database; the fs terminal is for accessing shared file storage (using NFS); and the aux terminal is for sending e-mail messages to an SMTP server.

The log terminal can be used to connect WEBx8 to a shared file system on which WEBx8 can store its logs.

The configuration of the web server is provided through properties. The properties are designed to cover the most uses in an easy-to-configure way and in most cases only a few need to be set to non-default values. Advanced configurations can be achieved through include files in the Apache configuration format, coming from the content volume. Access and options to content and script files can be further specialized through .htaccess files in the content directories.

If the features provided by the WEBx8 appliance do not meet your needs, please contact our Technical Support to discuss the possible options. We may be able to extend the catalog by providing appliances that cover your needs.

Boundary

Resources

Resource Minimum Maximum Default
CPU 0.85 139 0.9
Memory 1.046 GB 128.25 GB 1.046 GB
Bandwidth 9 Mbps 900 Mbps 81 Mbps

Terminals

Name Dir Protocol Description
in in http Serves HTTP requests coming from web clients. Serves the HTTP/1.1 and 1.0 protocols.
db out mysql Access to a MySQL server. Usage is defined by whatever scripts reside on the content volume (if any). This terminal may be left unconnected if it is not used.
fs out nfs Access to a network file system for shared file storage, providing read-write file access over NFS. Usage is defined by whatever scripts reside on the content volume (if any). Software on the content volume 'sees' the mounted remote file system as /mnt/fs in the appliance's filesystem space. This terminal may be left unconnected if it is not used. The connected server must have a read-write share named /mnt/data.
log out cifs Access to a CIFS-based network file system for storing access and error logs. This terminal may be left unconnected if it is not used. The connected server must allow anonymous logins and have a read-write share named share.
aux out any Access to an SMTP server for sending outgoing e-mail. Usage is defined by whatever scripts reside on the content volume (if any). Note that 'sendmail' is not configured on the appliance, any application that needs to send mail, should simply use SMTP directly, using 'aux' as the target hostname. For Perl CGI scripts the Mail::Mailer module can be used to do that.
This terminal may be left unconnected if it is not used.
net out any Gateway output for subnet access.
This terminal may be left unconnected if it is not used.
mon out cce Sends performance and resource usage statistics.
This terminal may be left unconnected if it is not used.

User Volumes

Volume Description
content Read-only, shareable volume providing the web content (documents, media and scripts) to be served by WEBx8.

The volume may provide static and/or dynamic content with an optional configurable directory dedicated for scripts. Script execution from any directory can also be configured using a file name pattern.

In addition to content, the document directory (configured through the docs_dir property) on the content volume may also hold the following optional configuration files, all in the standard apache configuration format (see http://httpd.apache.org/docs/2.0):

  • .htconf - configuration settings for the document directory and its sub-directories. This file can contain any Apache configuration options that are allowed in a <Directory> section, and has the same effect as a .htaccess file, except that it is read only once during startup.
  • .apache_conf/*.conf - all files matching this pattern are included in the configuration. This should be used for advanced configuration only. See Advanced Configuration below.
  • .htaccess - Per-directory configuration. Each directory may have its own .htaccess file, defining options and access rules for the files in that directory and its sub-directories. The use of .htaccess files is disabled by default. Since the content volume is read-only and is not normally modified at runtime, it may be better to configure per-directory access using <directory> or <location> sections in the custom config files in the .apache_conf directory (see above). Enable the use of .htaccess with the use_htaccess property, if the use of config files is not feasible or if some of the content is sym-linked to data modifiable at runtime (not recommended).

Properties

Property name Type Description
lb_cookie_name string the name of the cookie used to identify a session. For the passive modes ( passive and sync - see the lb_mode property below), this is the name of the cookie used by WEBx8 to identify client sessions. For the active ( insert ) mode, this is the name of the cookie that WEBx8 should insert into HTTP responses in order to make each client 'stick' with a single server.
If this property is set to the empty value, no session tracking is done and all requests are distributed in a simple round-robin fashion.
Default: (empty).
lb_mode string Specifies how to use the named session cookie for session identification. Valid values are:
passive - the cookie is not modified, it is expected that a given cookie value is always unique and the same value will never be used by another server.
synch - the cookie string in the HTTP response from a server is modified to make it unique to the terminal from which the response came. The mapping between cookie values and output terminals is the same as for passive - i.e., the entire cookie value is compared.
insert - the load balancer itself inserts a cookie into responses sent back to the client on in , so when the client sends subsequent requests with that cookie, they are directed to the same server as the first request from that client. The cookie inserted by WEBx8 is without an expiration date, meaning that it is not to be permanently saved by the client software.
Default: passive
lb_timeout integer Optional timeout (in seconds) to expire inactive sessions. If this is set to a non-zero value, a session that has been inactive for more than the value of this property in seconds is 'forgotten'. Further requests bearing the 'forgotten' cookie are treated as if they have no cookie at all and are directed to to a random server, using the usual round-robin method.
Default: 0
lb_max_connections integer If set, this specifies the maximum number of concurrently active connections that WEBx8 will handle. When this number is reached, new connections are still accepted, but their processing is delayed until another connection is closed. The maximum allowed value for this property is 500. If set to 0 (the default), WEBx8 automatically determines the maximum based on the available system memory. Note that neither the available memory nor an explicit setting of this property have a direct effect on WEBx8's throughput or its maximum request rate - setting a low number (or having little memory) will affect response only if the back-end WEBx8 is performing lenghty operations for each request (e.g., database searches), causing many requests to remain open at the same time.
Default: 0
lb_cookie_path string The URL path to use in the WEBx8-generated session cookies (when the mode property is set to insert ). If WEBx8 is sitting behind a URL switch and serving only a subset of the namespace, this property may be set to avoid having the client send the same cookie for pages not served through WEBx8, e.g., if WEBx8 is used to load-balance a bank of servers doing nothing bug CGI scripts, in the /cgi-bin/ directory, cookie_path should be set to /cgi_bin . The value of this property should normally have a leading "/" - if it is missing, it is added automatically.
Default: "/"
lb_cookie_domain string This optional property defines the domain name to use in the WEBx8-generated session cookies (when the mode property is set to insert ).
By default, if this property is not set and WEBx8 does not send a 'domain=..' property with the cookie at all. This is the recommended behavior in most cases, as the client will remember the cookie as associated with whatever domain it used to contact the service provided through WEBx8. Note that setting a domain name that does not match what a client's browser would be using to contact the server can have negative consequences, including: (a) the cookie being sent to a different server, but not to WEBx8; (b) the cookie being rejected altogether, if the browser has a restrictive policy in place.
Default: (empty)
host_name String Host name of the web site. WEBx8 uses this name to display in automatically generated pages, as well as for constructing absolute URLs, e.g., in server-generated redirects (e.g., as may be set up in a custom config file installed on the content volume). Note that although explicit redirects are rarely configured, a request for a directory without the trailing '/' causes a re-direct to be issued to the client, with the '/' appended; this redirect will use the value of host_name if it is set.
Setting this property correctly is recommended if WEBx8 is expected to be accessed by older clients that do not send a Host: header with their HTTP requests.
If this property is set to an empty string, the host name used in the client's request is used.
Default: (empty)
admin_email String E-mail address for the server administrator. WEBx8 uses this name to display it in automatically generated pages.
Default: root@localhost
content_on_fs String Specifies whether the content is relative to the file system at the fs terminal or is on the content volume. Allowed values are on and off. If set to on, all content is relative to the file system at the fs terminal. If set to off, the content is on the content volume.
Default: off
docs_dir String Root directory on the content volume where the documents to be served are located. For example, it may be /mydocs. If docs_dir is set to the empty string, the root directory of the content volume is used.
The directory must be pre-existing on the content volume.
Default: (empty)
docs_loc String An optional location within the client-visible namespace where the content should appear ("client-visible namespace" is the path portion of a URL, or what appears to the right of the host name in a URL, including the leading /). If set to a non-empty value, this becomes the namespace root, as seen by the client, where the document root directory appears. For example, if docs_loc is set to subspace/samples, an HTTP request for /subspace/samples/file1.html will serve file1.html from the document root directory. Note that a request for a location outside /subspace/samples will return an error, except if it is a request for a script in the scripts directory (see scripts_loc).
This property is useful when WEBx8 serves a subspace of a Web site. Do not append a trailing slash to this value.
Default: (empty)
scripts_dir String Root directory on the content volume where CGI scripts are located. For example, it may be /scripts. This should be set to a non-empty value, referring to a directory on the content volume that contains only executable scripts (don't set it to "/").
The directory must be pre-existing on the content volume.
Default: (empty)
scripts_loc String The location in the web space where the scripts directory specified by scripts_dir should be visible. This must be set to a non-empty string to enable the use of a dedicated scripts directory. A typical value may be /cgi-bin. Do not append a trailing slash to this value.
Default: (empty)
logs_enabled String Controls whether WEBx8 will send its logs out the log terminal. Allowed values are on and off. If set to on, the log terminal must be connected. If this is set to off, no access log is used at all and the error log is written to a file on the root filesystem of the WEB servers, rotated weekly, with 4 weeks of back logs kept (/var/log/httpd/error_log*). Default: off
logs_base_dir String Directory where WEBx8's logs are stored. This propery has no effect if logs_enabled is set to 'off'. Default: /
error_log_level String Severity level of messages to be written to the error log. Allowed values are debug, info, notice, warn, error, crit, alert and emerg. debug writes most messages, emerg writes only emergency messages. Default: warn
srv2_standby Integer Specifies whether the second WEB server is in standby mode. If non-zero, the second WEB server is in standby mode, otherwise it is not in standby mode.
Default: 1
srv3_standby Integer Specifies whether the third WEB server is in standby mode. If non-zero, the third WEB server is in standby mode, otherwise it is not in standby mode.
Default: 1
srv4_standby Integer Specifies whether the fourth WEB server is in standby mode. If non-zero, the fourth WEB server is in standby mode, otherwise it is not in standby mode.
Default: 1
srv5_standby Integer Specifies whether the fifth WEB server is in standby mode. If non-zero, the fifth WEB server is in standby mode, otherwise it is not in standby mode.
Default: 1
srv6_standby Integer Specifies whether the sixth WEB server is in standby mode. If non-zero, the sixth WEB server is in standby mode, otherwise it is not in standby mode.
Default: 1
srv7_standby Integer Specifies whether the seventh WEB server is in standby mode. If non-zero, the seventh WEB server is in standby mode, otherwise it is not in standby mode.
Default: 1
srv8_standby Integer Specifies whether the eighth WEB server is in standby mode. If non-zero, the eighth WEB server is in standby mode, otherwise it is not in standby mode.
Default: 1

ALERT! All enumerated string properties are not case sensitive (lowercase). All other string properties are case sensitive.

Advanced Properties

These are additional properties that should normally not need to be configured. They can be used to tune up WEBx8 in non-standard circumstances.

Property name Type Description
cgi_ext String Defines the file name suffixes for CGI scripts as a space-separated list. This is used to specify which files to treat as executable CGI scripts (in any directory) and can be used as an alternative to specifying a common scripts directory - see the scripts_dir and scripts_loc properties. Default: .cgi
shared_perl_ext String Defines the file name suffixes for in-process Perl scripts as a space-separated list. This is used to specify which files to treat as Perl scripts to be run with mod_perl. Any extension that matches one already specified in cgi_ext overrides the use of CGI for that extension and configures it to be run by mod_perl instead. The in-process Perl module is configured to run scripts with the specified extensions in CGI compatibility mode - most Perl scripts designed to be run as CGI processes should work if set up as in-process scripts as well. Note: it is important to check your scripts for proper initialization of all file-scope variables, e.g., with use strict; use warnings; , since all scripts run by mod_perl execute without re-starting the Perl interpreter. If unsure whether your scripts will work with mod_perl, check the mod_perl documentation for more information: http://perl.apache.org/docs/index.html.
Default: (empty) (i.e., don't use mod_perl)
index_files String Ordered, space-seperated list of files to use as a directory index page in case the client requests a URL that refers to a directory name.
Default: index.html index.html.var
use_htaccess String Specifies whether to enable the use of .htaccess files for the entire document tree. This is disabled by default and should be used only if you have a content directory structure that requires fine-grain control of server options, but is too complicated to configure using a set of sections in a common config file, or if you have a pre-configured content volume that already uses .htaccess and want to avoid the hassle of re-arranging it. If config options are needed only for the document root itself, put your settings in the document root directory in the .htconf file instead - they will be applied exactly the same way as for .htacces, except the file is read once at boot time. Default: off
max_connections Integer Maximum number of concurrent active connections that each WEB server can handle. If this limit is reached, additional connection requests are queued (up to a limit of 511) and processed as currently active connections get closed. This value should be raised only if WEBx8 is expected to serve a large number of requests that need access to a slow external source of data (e.g., lengthy database queries). The actual number of concurrent connections may be limited below the value set for this property, if there is not enough system memory. See the Memory Usage section below. The max_connections value cannot be set to more than 256.
Default: 128
persistent_connections String Controls whether WEBx8 allows clients to request more than one document on the same connection. Allowed values are on, off. You may need to disable the persistent connections only if you have problems with clients that don't support HTTP/1.1 properly but don't fall back to HTTP/1.0. Default: on
idle_timeout_sec Integer Timeout, in seconds, for keeping a client connection open if there is no request or response going through. This property is used only if persistent_connections is on. Keeping this timeout short helps drop forgotten connections quickly. Default: 15
data_timeout_sec Integer Timeout, in seconds, for receiving or sending more data if a data transfer has started but is not completed. Having this timeout allows WEBx8 to drop connections that got forgotten, while still allowing delays during the transfer. Default: 300
default_charset String Default character set to report for returned documents of type text/html or text/plain. The property should be set either to the name of a character set or to "off" (to disable sending of a charset header in HTTP responses). Note that for some browsers setting this property can override the character set value provided by the author of the served HTML documents and is therefore recommended only if there are documents that do not contain a valid character set specification and it is known in advance that all documents served by WEBx8 are in the same character set. Typical example values may be iso-8859-1 or utf-8. Default: Off
Note: to cover for special cases, including old CGI scripts that don't generate a proper 'content-type' meta tag, per-directory setting of the default charset is possible using custom configuration files stored on the content volume.
info_level String Controls how much information WEBx8 discloses about itself to web clients (in HTTP responses and in automatically generated pages). Allowed values are full (most info), os, minor, major, prod (least info). As a security measure, it is recommended that you disclose the minimum information. Default prod
server_signature String Controls whether WEBx8 appends a server signature line at the bottom of all generated pages (e.g., on error pages). Allowed values are on, off. The amount of information WEBx8 includes in the signature is controlled by info_level. Default off (no signatures).
env String Used to pass settings to WEBx8 as name=value pairs. Default: (empty)

ALERT! All enumerated string properties are not case sensitive (lowercase). All other string properties are case sensitive.

Memory Usage

Each WEB server in WEBx8 configures itself automatically to run with a wide range of available memory, to fit applications of different size and load. Despite this, please note that the configuration calculation cannot predict the memory usage of dynamic-content scripts that may be installed on the content volume and a misbehaving script can cause the server to malfunction by overcomitting memory and causing the OS to kill processes.

By default, each WEB configures the maximum number of active connections assuming:

  • 16MB for system use (OS kernel, system daemons, Apache shared code)
  • 2MB per active connection, with 1MB used by Apache and 1MB for an external script

For example, in the 'sandbox' configuration with 32M, the max number of active connections will be set to 8.

The maximum number of connections can be limited below the automatically computed value using the max_connections property. Remember that if max_connections is above the limit imposed by the available system memory, it is trimmed without warning.

In addition, the PHP pre-processor's allocation limit is set to 1/2 the memory available for scripts, as computed according to the above rules, i.e., the PHP is configured to limit the memory for a running script to

(system_memory - 16MB - max_connections * 1MB) / 2

Where the max_connections value is the smaller of the max_connections property and the limit imposed by the available memory (computed assuming 2MB per connection, as described above). If the max_connections property is left at its default value, this will yield the following value for the PHP memory limit:

(system_memory - 16MB) / 4

Note that the "1/2 of available memory per script" rule used is somewhat optimistic and is based on the assumption that either not all active connections will use a hungry PHP script that actually hits the limit, or that the scripts don't actually use all the memory that they allocate.

TIP: If a memory-heavy application starts misbehaving under load (drops connections), and increasing the available memory is not an option, try setting the max_connections property to a lower value. This will reduce the possible number of script instances that run at the same time, giving each instance more memory to run in.

Setting Up The Content, Shared File Storage

This section provides useful information for configuring scripts that serve dynamic content.

Fixed Directory Names

On starting the HTTP server, the following directories and files are available within the filesystem space of each WEB instance. Note that using absolute directory names outside of these locations in any script or configuration file is not recommended.

/var/www/html - web root, visible as '/' to the client. This refers to a location on the content volume and is read-only. Note that if the 'docs_loc' property is set, then /var/www/html itself will have no data files in it. Appending the value of docs_loc will produce the name of a valid symbolic link that refers to the document root. This directory name and the /var/www/cgi-bin name (see below) and any of their sub-directories can be used in Apache configuration files to set up additional per-directory settings. This avoids the need for such configuration files to be aware of the particular setting for the docs_dir and scripts_dir properties.

/var/www/cgi-bin - symbolic link to the scripts root, if set up via the scripts_dir property.

/mnt/fs - root of the shared read/write file storage provided by a NFS server connected to the 'fs' terminal. If one is not connected, /mnt/fs will be empty.

Fixed Web Space Locations

The /icons/ path is aliased to a directory containing stock icons for server-generated directory listings. Therefore, a directory named icons in the content volume will not be visible.

Access Control

The HTTP server runs as user 48, group 48. CGI scripts run in the same context.

Files on the content volume should have 'read' permission for everyone, to be eligible for serving through HTTP. Executable scripts should have 'read' and 'execute' permissions for everyone.

Although the Apache server has other means to control access to files, one may also remove the 'read' permission for 'everyone' from files or directories that should not be made accessible through HTTP. Do leave the 'x' bit set on for directories for which a listing should not be accessible, but contain files or sub-directories that are accessible.

Sending E-mail

The 'sendmail' system is not configured on WEBx8. Do not use it for sending e-mail from this appliance. If using Perl-based scripts, the Mail::Mailer module can be used for sending e-mail, if it is configured to use the 'smtp' mailer. In all cases, your SMTP mailer should be set up to use 'aux' as the hostname of the SMTP server, e.g., if using Mail::Mailer :

$mailer = new Mail::Mailer 'smtp', (Server => 'aux');
$mailer->open(\%headers); #... etc.

Interior

The following is the interior of WEBx8.


WEBx8_interior.PNG

Typical Usage

Scalable 2-tier application


Scalable_2-tier_application_webx8.PNG

Notes and Links

Known Limitations

  • ALERT! If using WEBx8 to serve pages to clients outside of the application, the web servers never get the real IP addresses of the clients. Instead, it sees the IP address of the output of the appliance that is connected to WEBx8's in terminal. This creates problems in (a) the access log and (b) getting the IP address of the client to CGI scripts. We are working on finding appropriate solutions.
    • For the log, a solution may be to log the incoming IPs on an advanced IN gateway, using the iptables log capability. A current workaround is to use a small JavaScript on each page and record the stats on a regular web site -- similar to the access tracking done with Urchin (now Google Analytics).
    • For the IP addresses, if what you want to do is control what clients can connect, use the IP filtering in the gateway.
    • A new gateway, INSSL is available. It provides the real IP address in the X-Forwarded-For http header field and is OK to use even though the SSL support is not used..
    • For problems and suggestions, please contact Technical Support.

Related Documents

Questions and Comments

IDEA! To post a question or comment on this appliance, visit our forum.

-- MinhQ - 04 May 2007

 
Copyright © 2005-2008 3tera, Inc. All Rights Reserved.
%