Filer_Windows - Windows Filer - Implementation Design

Base Class

The Windows Filer application windows volume is based on WIN03S.

Volumes

Filer_Windows has a single class volume:

• boot volume, named grub, with GNU GRUB installed, 8 MB

Filer_Windows includes a placeholder volume for the actual Window OS (the application windows volume)

• placeholder volume, named windows, 3 GB, mounted as drive "c", read-write

Packages

The Windows Filer windows volume contains the following 3rd party open source Cygwin packages:

• alternatives-1.3.30c-2.tar.bz2 (GNU GPL v2)
• ash-20040127-3.tar.bz2 (BSD)
• base-files-3.7-1.tar.bz2 (Not Applicable)
• base-passwd-2.2-1.tar.bz2 (Not Applicable)
• bash-3.2.39-19.tar.bz2 (GNU GPL v2)
• bzip2-1.0.5-2.tar.bz2 (BSD)
• libbz2_1-1.0.5-2.tar.bz2 (BSD)
• coreutils-6.10-1.tar.bz2 (GNU GPL v2)
• cpio-2.9-1.tar.bz2 (GNU GPL v2)
• crypt-1.1-1.tar.bz2 (GNU GPL v2)
• curl-7.16.3-1.tar.bz2 (MIT)
• libcurl4-7.16.3-1.tar.bz2 (MIT)
• cygrunsrv-1.34-1.tar.bz2 (GNU GPL v2)
• cygutils-1.3.2-1.tar.bz2 (GNU GPL v2)
• cygwin-1.5.25-15.tar.bz2 (GNU GPL v2)
• cygwin-doc-1.4-4.tar.bz2 (NA)
• db4.5 (BSD)
• diffutils-2.8.7-1.tar.bz2 (GNU GPL v2)
• editrights-1.01-2.tar.bz2 (BSD)
• expat-2.0.1-1.tar.bz2 (Thai)
• libexpat1-2.0.1-1.tar.bz2 (Thai)
• libexpat1-devel-2.0.1-1.tar.bz2 (Thai)
• file-4.21-1.tar.bz2 (BSD)
• findutils-4.4.0-1.tar.bz2 (GNU GPL v3)
• gawk-3.1.6-1.tar.bz2 (GNU GPL v3)
• libgdbm4-1.8.3-8.tar.bz2 (GNU GPL v2)
• libintl2-0.12.1-3.tar.bz2 (GNU GPL v2)
• libintl3-0.14.5-1.tar.bz2 (GNU GPL v2)
• grep-2.5.1a-4.tar.bz2 (GNU GPL v2)
• groff-1.18.1-2.tar.bz2 (GNU GPL v2)
• gzip-1.3.12-2.tar.bz2 (GNU GPL v2)
• ipcalc-0.41-1.tar.bz2 (GNU GPL v2)
• less-382-1.tar.bz2 (GNU GPL v2)
• libiconv2-1.11-1.tar.bz2 (GNU GPL v2)
• libssh21-0.15-1.tar.bz2 (BSD)
• links-1.00pre20-1.tar.bz2 (GNU GPL v2)
• login-1.9-7.tar.bz2 (BSD)
• man-1.6e-1.tar.bz2 (GNU GPL v2)
• minires-1.01-1.tar.bz2 (GNU GPL v2)
• libncurses8-5.5-3.tar.bz2 (Thai)
• openssh-5.0p1-1.tar.bz2 (BSD)
• openssl-0.9.8h-1.tar.bz2 (BSD)
• patch-2.5.8-9.tar.bz2 (GNU GPL v2)
• libpcre0-7.6-2.tar.bz2 (BSD)
• perl-5.8.8-4.tar.bz2 ("GNU GPL v1+ / Artistic,BSD,MIT")
• perl-Error-0.17010-1.tar.bz2 (GNU GPL v1+ / Artistic)
• libpopt0-1.6.4-4.tar.bz2 (Thai)
• libreadline6-5.2.12-10.tar.bz2 (GNU GPL v2)
• rebase-2.4.4-1.tar.bz2 (GNU GPL v2)
• rsync-2.6.9-2.tar.bz2 (GNU GPL v2)
• run-1.1.10-1.tar.bz2 (GNU GPL v2)
• sed-4.1.5-2.tar.bz2 (GNU GPL v2)
• tar-1.20-1.tar.bz2 (GNU GPL v3)
• libwrap0-7.6-4.tar.bz2 (BSD)
• termcap-20050421-1.tar.bz2 (BSD)
• terminfo-5.5_20061104-1.tar.bz2 (MIT)
• texinfo-4.8a-1.tar.bz2 (GNU GPL v2)
• tzcode-2008a-1.tar.bz2 (BSD)
• unzip-5.52-2.tar.bz2 (BSD-Info-Zip)
• vim-7.1-1.tar.bz2 (Charityware)
• wget-1.11.3-1.tar.bz2 (GNU GPL v3)
• which-2.19-1.tar.bz2 (GNU GPL v3)
• zip-2.32-2.tar.bz2 (BSD-Info-Zip)
• zlib-1.2.3-2.tar.bz2 (ZLIB)
• setup-2.573.2.3.exe (GNU GPL v2)

The Windows Filer windows volume contains the following additional 3rd party open source packages:

• apache-2.2.9 (Apache License v2.0)
• php-5.2.6 (PHP License v3.01)
• eXtplorer-2.0.0_RC1-15 (GNU General Public License)

Scripts

The Windows Filer windows volume contains the following scripts, which implement the functionality of the filer. In the appliance, these scripts are stored at c:/appliance/scripts/.

• format.sh - setup the filer to operate in format mode (i.e., execute mkfs on the dst volume)
• fscopy.sh - setup the filer to operate in fscopy mode (i.e., mount both src and dst volumes and cp contents from src to dst)
• fsck.sh - setup the filer to operate in fsck mode (i.e., execute fsck on the dst volume)
• fsrepair.sh - setup the filer to operate in fsrepair mode (i.e., execute fsck with repair option on the dst volume)
• manual.sh - setup the filer to operate in manual mode (i.e., mount the dst volume, start ssh, setup external interface)
• get_cp_progress.sh - estimates and returns the current progress for cp, in percentage. Called by the fscopy.sh script
• setenv.sh - sets various environment variables used by other scripts

The scripts corresponding to the mode property are invoked by c:/appliance/appliance.sh

Contents of the release directory

Exported appliance image:

• filer_windows-1.0.0-1.tar.bz2

Self-extracting executable

• Filer_Windows-1.0.0-1.msi self-extracting archive which transforms a clean Windows Server 2003 install into a the windows volume of Filer_Windows

msi Build Environment

• filer_windows_msi_build-1.0.0-1.zip: zip of all the files used to create the self-extracting executable. This is the build environment which should be unzipped on a Windows Server 2003 guest in order to create the msi file. It includes:
  • the files which are installed by Filer_Windows-1.0.0-1.msi
  • Filer_Windows.aip the Advanced Installer project file used to create Filer_Windows-1.0.0-1.msi

Install Environment

• filer_windows_install-1.0.0-1.tar.bz2: tar of all the files used to assist in creating Filer_Windows.

Tests

• filer_windows_test.tar.bz2: Windows Filer test scripts

Install Procedure

Create the grub Boot Volume

The boot volume of Filer_Windows is an 8MB ext3 volume with GNU GRUB installed in the MBR and on the filesystem. The following procedure is used to create this volume (This requires maintainer access to the grid):

• Copy and uncompress filer_windows-1.0.0-1-install.tar.bz2 on the impex volume of the grid controller. The archive extracts its contents into the directory work_Filer_Windows.
• From this directory, scp the archive grub_boot_volume_dom0.tar.gz to /var/applogic/ on dom0 of srv1 and unpack it.
• Install a vnc viewer on your local machine (for example TightVNC viewer available at http://www.tightvnc.com/)
• In dom0 of your grid, /etc/init.d/iptables stop.
• Download the Fedora Live CD to the _impex volume of your grid; scp it to the directory /var/applogic/grub_boot_volume/ in dom0.
• In the dom0 working directory dd if=/dev/zero of=grub-image bs=1k count=8192
• losetup /dev/loop0 grub-image
• Edit the file run-fedora-livecd to include the IP address of srv1 for vnclisten, e.g. vnclisten='192.168.100.100'. Also verify that the correct name is used for the livecd iso image.
• xm create run-fedora-livecd
• Access the VM using a vnc client and create the grub boot volume:
  • /sbin/fdisk /dev/sda
  • Use the commands o, n, a and w to create an MBR and a new active partition. Execute /sbin/fdisk -l -u /dev/sda. The result should be:

              device      boot    start   end      blocks   id    system
              /dev/sda1   *       63      16064    8001     83    Linux

• /sbin/mkfs.ext3 /dev/sda1 to create a new ext3 filesystem on the first primary partition of /dev/sda.
• mkdir /mnt/tmp ; mount /dev/sda1 /mnt/tmp ; /sbin/grub-install --root-directory=/mnt/tmp /dev/sda to install grub.
• In /mnt/tmp/boot/grub/ create the desired menu.lst file and ln -s menu.lst grub.conf. For reference, the menu.lst file used with the Filer_Windows boot volume is included in the grub_boot_volume_dom0.tar.gz archive.
• umount /mnt/tmp ; shutdown -y -i0 -g0
• In dom0 of your grid, /etc/init.d/iptables start.
• If you need to access the filesystem of the created ext3 grub based boot volume, it is not necessary to use the livecd again. Instead, mount the filesystem with the correct offset:
  • mkdir tmp ; mount -text3 -oloop=/dev/loop1,offset=32256 grub-image tmp

Create a Windows Filer Appliance including the grub Boot Volume

Use scp to pull the prepared image grub-image from dom0 to /vol/_impex/work_Filer_Windows/ on the controller.

In /vol/_impex/work_Filer_Windows/, edit the file setenv.sh to use current version settings. Execute the script appliance-create.sh, which performs the following:

• Rename and gzip the prepared image
• Generate appropriate descriptor files
• Create the release tar filer_windows-1.0.0-1.tar.bz2 under /vol/_impex

Check the log file appliance-create.log in case of any failure.

Creating the windows Placeholder Volume

Please see the Windows Installation Reference for instructions on how to create this volume and incorporate it into the Sys_Filer_Windows application.

Preparing the Self-Extracting Executable

The self-extracting executable is prepared on a Windows Server 2003 VM which includes a licensed copy of Advanced Installer. Copy the archive filer_windows_msi_build-1.0.0-1.zip to the Windows VM and extract it in C:\. Use Advanced Installer to create the msi.

Cygwin Packages

The msi contains a local repository of Cygwin packages. The automated install is only capable of installing the base packages. To ensure other required packages are installed, the repository is created as follows:

• Point IE to http://cygwin.com and download setup.exe to the desktop.
• Execute setup.exe and Download Without Installing. Install the packages in a temporary directory. In addition to the base install, include these packages:
  • Archive -> zip
  • Archive -> unzip
  • Editors -> vim
  • Net -> curl
  • Net -> openssh
  • Net -> rsync
  • Perl -> ipcalc
  • Perl -> perl
  • Perl -> perl-Error
  • Utils -> bzip2
  • Utils -> cpio
  • Utils -> diffutils
  • Utils -> file
  • Utils -> patch
  • Web -> links
  • Web -> wget
• Make a directroy c:/appliance/cygwin-package. Move the cygwin setup.exe under this directory. Move the file setup.ini and the folder release from the temporary cygwin install sub-directory under this directory as well. While reviewing the additional packages and their dependencies within the Cygwin install GUI, edit setup.ini so as to move these packages into the base install. At this point, everything needed to perform a cygwin install is under the c:/appliance/cygwin-package directory.

Theory of Operation

Filer_Windows is a standalone appliance that provides filesystem-level access to a volume with a NTFS file system.

Configuration

External network setup

Filer_Windows has an external interface which allows the appliance to communicate with the outside world. When the external interface needs to be configured, Filer_Windows invokes the /appliance/appliance_netsetup.sh script to configure the external interface (found in all AppLogic gateways).

The /appliance/appliance_netsetup.sh script assigns ip_addr (if one is specified) to the external interface. This script also sets the netmask and gateway, as well as setting the DNS servers.

Log Files

Filer_Windows does not create any additional logs.

Important Notes

None

-- StephenQ - 26 Jan 2010