Filer_Linux, Filer_Solaris, Filer_Windows03: Filer Appliances

In order to use Windows-based appliances in AppLogic you must have the 3tera Windows enhancement hotfix installed on your grid.

	At a Glance
	Catalog	Filer
	Category	Filers
	User volumes	yes
	Min. memory	See table below
	OS	Linux/Solaris/Windows
	Constraints	no

Functional overview

Filer_Linux is an appliance that provides filesystem-level access to a volume with a Linux file system. Filer_Linux supports the following Linux file systems: ext2, ext3, fat16, fat32, reiserfs, swap, and ext3-snapshot (ext3 over LVM with snapshots) and supports the following modes of operation:

• format: format the volume to the specified filesystem (e.g., execute mkfs)
• fscopy: perform a filesystem-level copy from one volume to another, the destination volume is formatted prior to the copy
• fsck: check the file system on the volume
• fsrepair: check and repair the file system on the volume
• manual: provide user-level access to the volume through both a Web GUI and root shell (through SSH)

In manual mode, Filer_Linux provides GUI access and root shell to the volume through its default interface. In this mode, Filer_Linux also optionally provides external network access in order for the user to copy files to and from the volume.

Filer_Solaris is a filer appliance that provides filesystem-level access to a volume with a Solaris file system. Filer_Solaris supports the following file systems: ufssol and zfs. It supports the same modes of operation as the Filer_Linux appliance.

Filer_Windows03 is a filer appliance that provides filesystem-level access to a volume with a Windows file system. Filer_Windows03 supports the following file systems: ntfs and ntfs03. It supports the same modes of operation as the Filer_Linux appliance.

The Filer appliances are used internally by AppLogic and should not be used in regular AppLogic applications.

Filer_Windows03 is distributed without the windows placeholder volume which is required for it to boot and perform its operations. Please see the Windows Installation Reference for instructions on how to create this volume and include it in the Sys_Filer_Windows03 reference application.

AppLogic 2.4.7+ supports manual mode operations over two volumes.

Boundary

Resources

Filer_Linux resources

Filer_Solaris resources

Filer_Windows03 resources

Terminals

The external interface is enabled. It is used for incoming and outgoing traffic and its network settings are configured through properties. It is only used in the manual mode and is not configured in all other modes.

The default interface is enabled. It is used for maintenance. Also, in the manual mode, it is used for accessing the Web GUI.

User Volumes

Properties

Operation modes

The following table lists the supported mode for each of the supported file systems:

In manual mode:

• for all file systems, but swap, the volume is mounted on /mnt/vol.
• for a swap volume, the block device is accessible on /dev/hda4.

Filesystem options

This section lists the file system options (as specified on fs_options) for each file system supported by Filer_Linux.

• ext2
  • None

• ext3
  • None

• fat16
  • None

• fat32
  • None

• reiserfs
  • None

• swap
  • None

• ext3-snapshot
  • vol_group_name: string, name of the LVM volume group to create on the dst volume. If empty, randomly generate a volume group name. Default: (empty).
  • data_percentage: integer, percentage of the volume that is used to store data, remaining portion of the volume is for snapshots. Default: 80 (80% of the volume is for data).

• ufssol
  • None

• zfs
  • pool_name: name of the zpool to create on the dst volume. If omitted, the vol_name property value is used instead.
  • mountpoint: mountpoint of the root dataset of the created zpool. Valid values are: an absolute path, e.g. /mnt/mypool, legacy and none. Datasets with legacy mounts are not automatically managed by zfs but require entries in /etc/vfstab or manual mounting. Datasets with mountpoint of none are not mountable. Default: /pool_name.
  • autoreplace: controls automatic device replacement. If set to off device replacement must be manually initiated using zpool replace; if set to on any new device found in the same physical location is automatically formatted and replaced. Default: off.
  • delegation controls whether a non-privileged user is granted access based on permissions defined on datasets. Valid values are off and on. Default: on.
  • failmode: controls behavior in the event of failure. Valid values are wait, continue and panic. Default: wait.
  • version: zpool version. Valid values are 1-10. Default: 10 (current).

• ntfs and ntfs03
  • volume_label: the volume label for the dst volume. If empty, the vol_name property value is used instead.
  • active: create the new partition as active (bootable). Valid values are yes and no. If omitted, the default value of no is used during format while the value defaults to the src volume type during fscopy

Interface

The Filer appliances provide an HTTP interface on their default interface in order to collect status on non-manual volume operations and to access the Web GUI when in manual mode. The following functions are available by URL:

• /: interactive access to the dst volume through the Web GUI, only available in manual mode
• /api/status: returns the status for the current volume operation, only available in non-manual mode
  • The format of the output is the following: [progress=W, ]poll=X, status=Y, errortxt=Z
    • progress: integer, 0..100, progress of the current operation. If progress cannot be reported, then the progress field is not returned. Progress is not reported for the following modes:
      • format for all file systems
      • fsck and fsrepair for all file systems, but ext2, ext3, ext3-snapshot, and ufssol
    • poll: integer, recommended status poll interval, in seconds.
    • status: integer, status of the volume operation. See below for the list of statuses that can be returned by Filer_Linux.
    • errortxt: string, error message, if an error occurred (e.g., non-zero status)
  • The following is the list of statuses that Filer_Linux can return in the status field for a specific volume operation:
    • 0 - success
    • 100 - operation failed
    • 101 - operation not supported
    • 102 - operation not implemented
    • 103 - operation canceled
    • 104 - I/O error
    • 200 - no space left on volume
    • 201 - file system errors detected
    • 300 - out of memory
    • 400 - pending

Web GUI

The Filer appliances use a web-based file manager named eXtplorer to provide the Web GUI access to a volume (accessible only in manual mode). eXtplorer is released under the GNU GENERAL PUBLIC LICENSE Version 2. The version of eXtplorer used in the filers have been modified. The following are the changes to eXtplorer:

1. Removed the login.
2. Updated eXtplorer not to display its own files.
3. Changed the file list to show the target for all links under the "Type" column.
4. Changed the tooltip generated when the mouse is over a directory in the directory list to show the symlink target if the directory is a symlink.
5. Changed symlink creation through the GUI to support orphaned links.
6. Changed delete file through the GUI to support deletion of symlinks.
7. Added an interface for editing the volume base path for any available volume.
8. Changed the generation of file & directory lists to support links.
9. Resolve relative & absolute links which include '..'.
10. Add UI for chgrp/chown, allowing numeric entries only.
11. Add owner/group to the file display.

The reference for the eXtplorer Volume Browser GUI can be found here.

The eXtplorer licenses and the source to the original un-modified eXtplorer can be found on the Filer appliances in /mnt/monitor/.volume_browser/LICENSES/.

ZFS Implementation Specifics

Filer_Solaris supports zfs pools containing a single virtual device to allow users access to zfs volumes in the same manner as volumes using other file systems such as ufssol. More complex pools using multiple devices can be created manually using raw volumes within an AppLogic appliance, but such volumes cannot be used with Filer_Solaris. ZFS filer operations are constrained to the following behaviors.

• Pools are created using the altroot property. As a result the mountpoint of the root dataset must be explicitly set, rather than defaulting to the pool name. This is due to a bug in the current zpool command which sets the default mountpoint to /altroot rather than /altroot/pool_name.

• fsrepair executes zpool scrub and returns a single line of status on completion; either success or failure. However, zpool scrub can be executed live on any pool within a running appliance and displays much more information in the event of a problem.

• fscopy supports only file system datasets (volume, snapshot and clone datasets are not copied). Administrative permissions are not preserved by fscopy.

• While the zpool version can be set with fs_options on create, the zfs version of the root dataset is 2, which is not backwards compatible with version 1. Solaris 10 appliances use zfs version 1. To use zfs pools with Solaris 10 appliances, create the pools manually from raw volumes rather than using Filer_Solaris.

• The Solaris filer does not support root zpools (zfs boot volumes). There is a bug in OpenSolaris 2008.05 which renders a root zpool un-bootable once it has been imported into another Solaris OS.

NTFS Implementation Specifics

Please carefully note the following implementation specifics and behaviors of Filer_Windows03:

• Filer_Windows03 operations are limited to:
  • basic disks only (not dynamic disks)
  • MBR based partions only (not GUID based partitions)
  • exactly one primary partition per disk

• Short names are enabled.

• fscopy operations are performed with robocopy. As a result, compressed source files are un-compressed on the destination volume. When performing a filesystem-level copy of a volume, be certain to leave enough room on the destination volume to account for this. If you copy the boot volume of a Windows appliance and the destination volume does not have sufficient free space, Windows will fail to boot (even though the copy operation proceeded without error). The recommended minimum free space for a Windows 2003 Server Standard Edition appliance is 1.25 GB, although in practice the OS will boot with less. robocopy does not support circular junction point references. If such references exist, the fscopy operation will fail.

• If, during vol manage, you want access to the console but do not know the Administrator password, ssh to the appliance from a 3T shell and net user Administrator new-password to set a new Administrator password.

• Local user account SID's on a volume mounted on Filer_Windows03 are not recognized by Filer_Windows03. In manual mode, files owned by local user accounts will display without a recognized user name.

• Filer_Windows03 boots from a small ext3 volume with GNU GRUB installed. Grub is used to hide the first primary partition of the src volume during fscopy and of the src and dst volumes during manual operation. The Windows bootloader ntldr is not equipped to discriminate among mulitple active (bootable) partitions, and so grub is used to prevent Filer_Windows03 from possibly booting from the wrong active partition. Upon successful start, the hidden partition, if any, is un-hidden so that filer operations may proceed.
  • If Filer_Windows03 abnormally terminates after grub has hidden a partition but before it has been un-hidden, the partition will be left in a hidden state. While it is very unlikely that this should occur, if it does, simply vol manage the affected volume and exit. This will restore the partition to its normal state. To avoid this potential problem, do not ctrl-c to terminate a filer operation while the filer is early in the boot process.
  • Unlike other ext3 volumes under AppLogic, the small ext3 boot volume of Filer_Windows03 has an MBR. As such, it cannot be managed with Filer_Linux, and filesystem level copy operations over it using Filer_Linux will also fail. If you would like to use this volume as the basis for your own custom grub-based appliance, you can access this volume as follows:
    • Create a new application including a branched LUX5 with a placeholder volume.
    • Copy the Filer_Windows03 grub volume to this application: vol copy Sys_Filer_Windows03:mon.grub application-name:volume-name
    • Start the new application and open a shell on the branched LUX5: mkdir /mnt/tmp ; mount -text3 -oloop=/dev/loop0,offset=32256 /dev/hda3 /mnt/tmp. You can now access the filesystem of the copy of the grub volume at /mnt/tmp.

• Please see the Windows Appliance Reference for significant details on the operation of Windows appliances under AppLogic.

Typical Usage

The following sections describe the configuration of Filer_Linux in several typical use cases:

formatting a volume

Example:

Filer_Linux executes mkfs over the dst volume, specifying a filesystem type of reiserfs.

formatting an ext3-snapshot volume

Example:

Filer_Linux creates a volume group named VGVOL on the dst volume. It then creates a partition named data in the volume group. This partition uses 75% of the space on the dst volume and is formatted with ext3. The remaining partition is for snapshots and is named snap.

filesystem-level volume copy

Example:

Filer_Linux formats the dst volume to ext3 with mkfs. It then mounts the src volume read-only and mounts the dst volume read-write. Finally, Filer_Linux copies the contents of the src volume to the dst volume using cp and unmounts both volumes.

file system check

Example:

Filer_Linux executes fsck on the dst volume.

file system check with repair

Example:

Filer_Linux executes fsck with the repair option on the dst volume.

user-level access to volume

Example:

Filer_Linux mounts the dst volume read-write at /mnt/vol. It then starts the eXtplorer GUI and starts sshd, which gives the user root access to the volume. The GUI is accessible through the default interface and any file transfers to/from the volume is through the external interface.

Notes

• the Linux Filer is based on Ubuntu 7.04
• the Solaris Filer is based on OpenSolaris Build 2008.05
• the Windows Filer is based on Windows Server 2003 Standard Edition R2 32-bit
• The Windows Filer by default uses a random Administrator password for security purposes. If this is not desired, change the password as described in the Windows Appliance Notes.
• in non-manual mode, there is no SSH or GUI access

Related Documents

• Filer_Linux
  • Test Plan
  • Implementation Design

• Filer_Solaris
  • Test Plan
  • Implementation Design

• Filer_Windows03
  • Test Plan
  • Implementation Design

-- MinhQ - 24 Jan 2008