ADL - Application Descriptor Language

Overview

ADL is based on UDL - a generic language for describing hierarchically structured data in plain ASCII text form. You can find the description of UDL in a separate document, however understanding UDL in general is not necessary - this document contains a complete description of the ADL syntax.

Descriptor Types

Syntax Rules That Apply to All Descriptor Types

Lexemes

Punctuation Details

Here is the meaning of other punctuation characters in ADL:

Descriptor Structure

entity-type entity-name
   {
    attributes-and-subentities
   }

Where:

entity-type is one of component, assembly or package and identifies the type of descriptor that is contained in the file.

entity-name is the name of the entity being described in this file (SIDENT).

attributes-and-subentities is an unordered set which contains any number of attributes and sub-entities. Each attribute is on a single line and has the form: 'name=value'., where name is an identifier (IDENT) and value is a string value (STR). Each sub-entity has one of the following forms:

name : attributes

typename :attributes

name
   {
    attributes-and-subentities
   }

typename
   {
    attributes-and-subentities
   }

type and name are the type and the name of the sub-entity, respectively.

attributes is a comma-separated list of name=value pairs. Note that attributes can also be specified in the { } block following the entity heading line. When attributes are specified on the same line as the entity definition (after the colon), binary attributes can be specified without a value (just the attribute name, meaning 'set to 1'), e.g.

   volume boot: dev=/dev/hda1, ro

is the same as

   volume boot: dev=/dev/hda1, ro=1

Each subentity type defines a namespace, and within that namespace only one subentity of a given name can exist. That applies to the sub-entities with no type at all (one can think of the subentities with no type as having the empty string as the type).

Specifying attributes both in-line and in the { } block is allowed, it should be avoided except in the cases where one particular attribute should stand out (e.g., the .class attribute in a subordinate component's specificaion); otherwise for simpler sub-entities with few attributes the inline syntax is preferred; for more complex entities that have many attributes and/or sub-entities, use the { } block.

Component Descriptor Syntax

Below are two examples of the component descriptor structure; the first example is the current standard form, the second example shows the old format of the boot information specification. The old format is fully supported for compatibility with existing catalog appliances.

component name
   {
   volume sname : dev= pathname [, mount= pathname][, ro] [, high_bw]

   resource cpu : min= val , max= val
   resource mem : min= val , max= val , abs= val
   resource bw  : min= val , max= val

   (input|output)  sname : protocol= name [, mandatory ][, alias = dnsname ]
   interface  sname

   property  sname  : type=(string|integer|ip_addr) [, filter= regexp ] \
                    [,min= val ,max= val ] [, (mandatory| dflt= val) ]
   property  sname  : type=(string|integer) [, values= v1|v2|v3... ] \
                    [, (mandatory| dflt= val ) ]
   .config_mode=(dhcp|volfix)
   virtualization: mode=(paravirt|hvm)
      {
      path = filename
      initrd = filename
      options = " string "
      console = " string "
      device_schema = " string "
      }

   visual
      {
      ...
      }
   }

component name
   {
   volume sname : dev= pathname, [, ro] [, high_bw]

   resource cpu : min= val , max= val
   resource mem : min= val , max= val , abs= val
   resource bw  : min= val , max= val

   (input|output)  sname : protocol= name = [, mandatory][, alias = dnsname =]
   interface  sname

   property  sname  : type=(string|integer|ip_addr) [, filter= regexp ] \
                    [,min= val ,max= val ] [, (mandatory| dflt= val) ]
   property  sname  : type=(string|integer) [, values= v1|v2|v3... ] \
                    [, (mandatory| dflt= val ) ]
   cfgfiles
      [
      vol= sname , path= filename
      vol= sname , path= filename
      ...
      ]

   kernel: path= filename  [, initrd = filename ] [, options = " string" ]

   visual
      {
      ...
      }
   }

The component entity has the following attributes defined:

All component attributes are optional and need not be present in the descriptor.

All of the attributes are also valid properties of the component, which can be overridden in an assembly that contains the component. Note that the attribute names are prefixed with a dot, to avoid name conflicts with regular properties (see the 'property' entity below).

The table below is a summary of the valid sub-entities in a component, followed by sections that explain each sub-entity in detail.

volume

A 'volume' entity that has no 'class' attribute also defines a configurable property on the boundary of the component, which can be set the same way as other properties of the component - see the property entity below. The mandatory attribute for such volumes works the same way as the mandatory attribute for properties. A 'volume' property should be set to the logical name of one of the application's volumes (as found in the application's package descriptor). Note that this means volumes and properties share namespace and one cannot define a volume and a property of the same name.

resource

The 'resource' entities are mandatory , all must be specified in a component's description and all must have the 'min' and the 'max' value specified. The 'abs' value may be omitted and is assumed to be equal to 'min' by default.

input, output

interface

    interface external
    interface default

The 'interface external' entry enables the device named 'eth0' to be used as the external network interface of the component (accessible from outside the application). If enabled, eth0 is not used for terminals and its IP configuration is not set up automatically. Instead, it is expected that properties are defined to configure the network adapter.

The 'interface default' entry enables configuring an unused network interface for unrestricted use, with an automatically assigned IP address on the same subnet as the ones used for terminal connections. The assigned IP address is made available to the AppLogic controller as the IP address of the component; this can be used for maintenance logins.

property

• all properties are available for access by shell scripts and executables as environment variables defined in a shell file that is stored automatically on the component's boot volume and can be included by any script;
• also, any files named in a 'cfgfiles' entity (see below) are searched for special markup identifying a portion of the file as corresponding to a property value - all matching instances are automatically updated to reflect the property value as configured in the component's descriptor or as overridden by any assembly in which the component participates. There are several variants of "markup", which allow making it appear as comments to the software that uses the file, so it is completely transparent to the component's code. See the Property Markup Syntax section later in this document.

Note that since a volume can appear as a configurable property on the boundary, volumes and properties share namespace and one cannot define both a volume and a property of the same name.

The property entity has the following attributes:

cfgfiles

vol - names the volume on which the file is located (this should match one of the component's volumes, as defined by the volume entities). A value for this attribute must be specified for each cfgfiles array element.

path - this is the file name of the configuration file, relative to the volume on which it is located. Note that this may be different from the path where the application "sees" that file - depending on how the particular volume is mounted by the component. A value for this attribute must be specified for each cfgfiles array element.

quoting - defines the method for quoting meta-characters in the configuration file. A "meta-character" is any character that has a special meaning in the config file and must be quoted (or "escaped") in some manner in order to appear as a data character and not in its special-function role. The 'quoting' attribute can be set to one of the following strings:

NOTE: the cfgfiles sub-entity is useful only when .config_mode is set to volfix. In the dhcp mode, the component's boot volume will not be modified and it is expected to configure itself dynamically over the network.

virtualization

This entity contains the boot information needed to start the component in the virtual environment for which it is designed. The virtualization entity supersedes the kernel/os_info definitions and should be used instead of them. The presence of this entity also indicates a 'new-style' component descriptor and turns off the compatibility mode, which includes forced .config_mode=volfix.

The following attributes are defined for the virtualization entity:

kernel, os_info

obsolete, use virtualization instead. The presence of a kernel or os_info sub-entity in the descriptor indicates an old descriptor and the volfix configuration mode is forced automatically.

This entity contains OS-specific boot information, its contents depend on the value of the .os_type attribute of the component. The keywords kernel and os_info are equivalent, kernel is retained for backward compatibility.

When .os_type=linux , the following attributes are defined for the os_info entity:

path= filename - the name of the kernel image file, relative to the boot volume's root directory.

initrd= filename - the name of the ramdisk filesystem image to use during boot. The kernel image file and the ramdisk image file are typically produced as a result of building the Linux kernel.

options= "string" - other arguments to pass to the bootloader's 'kernel' command line.

When .os_type=legacy, the following attributes are defined:

options=" name=value name=value ... "

The legacy OS type is used for all hardware-assisted virtual machines. The name=value pairs are not interpreted, they are passed on to the HVM emulator directly. Examples of parameters supported by the qemu emulator (used in XEN) include: acpi=(0!1), apic=(0!1), pae=(0!1), etc.

Assembly Descriptor Syntax

The assembly descriptor has the following structure:

assembly sname
   {
   .category = text
   .description = " text "
   .console = " subord-name "

   input  sname
   output  sname
   ...

   property  sname &nbsp[ : dflt = value ]
   property  sname &nbsp[ : mandatory ]
   ...
   volume sname

   subordinate  sname
      {
       .class = clsname
        attr = val
       ...
      }

   connections
      [
       sub-name . trm-name => sub-name . trm-name
      ....
      ]
   visual
      {
      ...
      }
   }

The following attributes are defined for assemblies only, they have no meaning in simple components:

The following attributes are defined for an assembly; they have the same meaning as their counterparts for a simple component:

The order of entities in the assembly is not important and all of the sub-entities are optional, except that an assembly must have at least one subordinate entity.

Here is a summary of the sub-entities of the 'assembly' entity, followed by sub-sections defining each one in detail:

input, output

The terminal entities support a single attribute: mandatory . Specifying this attribute means that the terminal must be connected; mandatory terminals will trigger a compilation error in an assembly that includes a component with such a terminal left unconnected. Note that specifying the mandatory attribute for an assembly is not necessary if the subordinate component's terminal to which it is connected is already mandatory.

An assembly can have any number of terminals, except for the top-level assembly of an application, which should have no terminals.

property

volume

subordinate

All attribute names other than the pre-defined ones are considered to be property names of the subordinate component that need to be set to the specified value - this includes the predefined attributes of the subordinate component ( .boot_tout, .migrateable, .server, .standby ), as well as the component-specific properties defined in it by the 'property' or the 'volume' entities (see the component descriptor syntax).

A special type of value is defined indicating that the property is connected to the assembly's boundary: $.name, where name is the name of one of the 'property' or 'volume' entities in the assembly. If a property has to be set to a literal value that begins with the "$." characters, the value must be quoted to ensure that it is not interpreted as a property connection. More than one subordinate property can be connected to the same boundary property and all such properties get the default value from the boundary property definition if none is provided in an outer-scope assembly. The $.name can be used for the pre-defined attributes of the subordinate as well, making them regular properties on the assembly boundary.

The 'subordinate' entity in an assembly also accepts the resource sub-entities mem , cpu and bw , with the same attributes as defined in the Component Descriptor Syntax section.When applied to a subordinate that is a simple component, they override the resource settings in that component. The "override" must remain within the limits of the range defined in the component (i.e., the new range must no wider than the old one and must fit entirely into the old one). When applied to a subordinate that is an assembly, the specified resources are distributed pro-rata according to the relative weight of the resource requirements in each of that assembly's subordinates. If a resource setting for a subordinate assembly causes a component to receive a resource setting that is outside of the min-max range defined for it, an error is reported by the ADL linker.

connections

$. atrm-name => sub-name . strm-name, or

sub-name . strm-name => $. atrm-name.

Both syntax variants are equivalent and mean that the terminal strm-name of subordinate sub-name is to be visible as atrm-name on the assembly (atrm-name must correspond to an input or output entity defined in the assembly). Since an input terminal is a 'network server' and an output terminal is a client, the following rules apply:

• an output can be connected to at most one input
• an input can be connected to any number of outputs (subject only to limitations on the number of clients that the component supports for the specific service)
• an input terminal on the assembly boundary must be redirected to exactly one input of a subordinate compoent
• any number of subordinate component's outputs can be redirected to a single output on the assembly boundary

Package Descriptor Syntax

Following are the different types of package descriptors:

The package descriptor contains one entity of type 'package'.

An application package descriptor may also contain an entity of type 'assembly', with the same structure as the one found in an assembly descriptor, except that it cannot have terminals and properties on the boundary. It is used as the topmost component of the application containing the property settings for the application itself, with a single subordinate that is the application's main assembly.

The 'package' entity has the following attributes:

type=	one of: application, catalog, volcache (obsolete), recycle or clipboard.
description=	A human-readable description of the package's contents.
uid=	An ID assigned to the application at the time it was installed. This ID is an integer value in the range 1..254 and is unique among the applications installed on the same cluseter of servers.
template=	0/1 designating whether the application is a template. Supported in AppLogic 1.2.5
user1=	Free-form user-defined text intended for specifying billing code.
user2=	Free-form user-defined text intended for specifying billing code.

The 'package' entity has the following sub-entities, described in detail further below:

The 'package' Sub-entity

The following attributes are defined for the 'package' sub-entity:

The 'class' Sub-entity

The 'volume' Sub-entity

Note that the 'volume' entities in a package descriptor are the only explicit reference to volumes that belong to the application. However, each component class defined in the application scope or in an application-specific catalog also implicitly refers to one or more volumes that belong to the same application - each volume listed in such components' descriptors that has the class attribute has a corresponding 'template' volume, considered an integral part of the application.

The 'protocol' sub-entity

If the same protocol filter is defined in more than one package descriptor, all definitions must match exactly, otherwise an error is reported by the ADL compiler. The well-known protocols' filter definitions are defined in the global catalog that is part of any AppLogic installation; these include: http, ftp, smtp, ssh, etc.

The name of each 'protocol' entity is a name that can be used in the ' protocol= xxx' attribute of a terminal (see the component descriptor syntax).

The 'protocol' subentity has a single attribute: 'filter', with a string value that defines the protocol constraints, e.g.:

   protocol http: filter="tcp_in:80"

Note: the syntax of the protocol filter string is not part of this specification - it is not interpreted by the ADL compiler.

The 'resources' Sub-entity

Property Markup Syntax

Note that the property markup is only supported for appliances that use the volfix configuration mode.

General Requirements

• it is a plain-text file - that is, a file that can be opened and edited by a text editor.
• if the text file contains multi-byte characters, it uses the UTF-8 encoding, or any similar encoding that, if processed as a stream of bytes, allows interpreting any value in the range of 0-127 as the corresponding latin ASCII character, regardless of context.
• the file format allows for inserting comments that are transparent to the component's code that uses the configuration file; the comment syntax is such that comments can be placed sufficiently close to the property values that are being instrumented.
• within a comment, the characters [ " $ : , -\ ] do not serve as terminators of the comment block and are not interpreted in any other special way.
• no properties need to be configured to have the newline character as part of their value.
• if newline is not the terminator for a comment block, then in the normal text the file format must allow for encodig all characters that are part of a comment terminator in a way that they stop looking like a comment terminator.
• if the file is being read and re-written by anything other than the AppLogic volume fixup utility, the markup comments are preserved and additional newlines are not inserted in the middle of a property value.
• if the file is being read and re-written by anything other than the AppLogic volume fixup utility, there is a way to ensure that during a re-write, no new text is inserted between a markup comment and the property values to which it refers.

The following file formats are known to comply with the above requirements, and are therefore "instrumentable":

• Linux line-oriented configuration files (commonly residing in the /etc/ directory and having the .conf suffix), in which lines beginning with the # character are treated as comments.
• sh/bash and Perl scripts
• source files in C++ (including header files).
• source files in C, when compiled by GCC or another compiler that allows the C++ single-line comments (//).
• HTML and similar SGML files

To handle non-instrumentable files, the recommended approach is to write a bash (or Perl) script that updates the config file on boot and instrument the script itself.

Markup Styles

Inline markup:

The presence of a $$prop: string on a line of text in the configuration file means that this line contains one or more property values. Usually, the "$$prop:" string and the rest of the line of text are made invisible to the application that uses this configuration file by making it appear as a comment, e.g., if the configuration file is a Perl script this might look like:

$port = 3306 # $$prop: 3306:ip_port

The val:name pairs following the markup identifier string are interpreted as follows:

val1:name1 indicates that the first occurrence of the string val1 on this line is to be treated as the value of the property name1 and replaced whenever that property needs to be changed.

val2:name2 indicates that the first occurrence of the string val2 following the first occurrence of val1 is the value of the property name2.

etc.
If the name in a val:name pair is the string '-', the string 'val' is simply ignored. This is used to skip parts of the configuration data that might otherwise match a property value, e.g., a markup like this:

x1 = 1 # $$prop: 1:val

will cause the '1' in x1 to be considered the value of the property 'val'. To make the string '1' that follows the = sign be the property value, the markup has to be:

x1 = 1 # $$prop: 1:-, 1:val

The special '-' property name is also used in case the property value is the empty string, e.g., in the following markup the property is the empty string that follows the "x1 = " string:

x1 = #$$prop: "x1 = ":- , "":val

If a value contains punctuation characters that are part of the markup syntax (colon, comma, space), the value must be quoted, using the double-quote syntax that is defined for the ADL descriptor files.

Next line markup:

This markup is used for configuration files that do not allow comment text to appear on the same line as the value that needs to be exposed as a modifiable property. It is similar to the inline syntax, but it indicates that the text on the line that follows the one on which the markup appears is to be searched for matching strings, not the current line.

Markup for files that are not line-oriented:

This markup is used for configuration files in which the newline character is not considered different from other whitespace and updates of the file may cause newlines to be added or removed at any place where un-quoted whitespace occurs. For this type of markup, a closing $$ sequence is required to indicate the end of the list of val:name pairs. Newlines are allowed between the val:name pairs. All text following the closing $$ mark, regardless of newlines is searched for strings matching the values, until all values are found or until 1K of text is read for each val:name pair (if the latter occurs without finding all values, an error is reported). Note that each property value is still expected to reside on a single line.

Meta-character Quoting in Configuration Files

Whenever a property value contains such "escaped" characters, they must appear in the same exact way both in the markup (which is inside a comment section in the file) and in the actual text, even if these characters need to be "escaped" only in the normal config file text (or only in the comment). For example, the & character doesn't have a special meaning in HTML comments, but it has to be "escaped" in HTML data, i.e.:

<!-- $$propF "&":my_prop $$ --><sometag someattr="text &amp; more text">

is not valid, even though "&" is OK to appear in the comment, and must be re-done this way:

<!-- $$propF "&amp;":my_prop $$ --><sometag someattr="text &amp; more text">

The quoting of the data values may be in different formats, depending on the file type, as specified by the 'quoting=' attribute (see the component descriptor syntax). This quoting is independent of the C-style double quotes used by the markup syntax itself to enclose a property value - the latter is always done with the C-style double quotes and is superimposed on top of the former, e.g., the string 'abc"def', which may be quoted as abc\"def for a C file, will appear as follows in the markup:

p = "abc\"def"; // $$prop "abc\\\"def":p_val

Appendix

Future Directions

Per-property quoting style override

To provide for cases like this, a per-property override can be defined as an extension of the markup syntax, e.g.:
$$prop value:prop-name(quote-style), ...

More quoting styles

• user-defined styles (e.g., specified as perl-style replacement commands like: s/&/&/)

• pre-defined styles:
  makefile : \<LF> is encoded as \#<LF>, trailing whitespace (<SP><LF>) is encoded as <SP>#<LF>, leading whitespace is not encodeable and is forbidden.
  xml : TBD, should be similar to HTML.

Unresolved

"Fixing" the device name for the "external" interface.

Global ".failover" group IDs

Included topics, for easier editing: AdvADLDescriptorTypes AdvADLSyntaxRulesGeneral AdvADLComponentDescriptor AdvADLAssemblyDescriptor AdvADLPackageDescriptor AdvADLPropertyMarkup AdvADLAppendix

-- VladM - 25 May 2006