r25 - 10 Jun 2010 - 09:55:50 - BeckyHYou are here: Wiki >  AppLogic29 Web > CatDatabaseAppliancesMySQLR
ALERT! AppLogic 2.9 Documentation The latest production release is AppLogic 3.0.30

MYSQLR, MYSQLR64: MySQL Database Appliances suitable for replication.

mysqlr64.png At a Glance
Catalog System
Category Database Appliances
User volumes yes
Min. memory 128 MB
OS Linux
Constraints no
Questions/Comments Ask Forum

Functional Overview

MYSQLR64 is database appliance based on the MySQL database engine (http://www.mysql.org). It provides an easy way to add a database to any application. The appliances can also be used in complex MYSQL replication scenarios. The appliances are based on MYSQL5 (CentOS 5/MySQL 5) but also handle database replication.

Database replication enables data from one MySQL database server (known as "master") to be replicated to one or more MySQL database servers (known as "slaves"). The MYSQLR64 appliances can be setup for master-slave as well as master-master replication and replication with more than two masters.

The replication setup, management and monitoring is done via a web interface. The web interface provides an easy way to start replication with close to zero downtime on the master. It can also be used to repair a replication in case of problems. The web interface can be used for copying databases from older database appliances like MYSQL and MYSQL5. MYSQLR64 also provide an easy way for managing and browsing your database (based on phpMyAdmin).

Replication is useful in several cases:

  • Scale-out solutions - spreading the load among multiple nodes to improve performance ("master-master" configuration).
  • Failover solutions - Using multiple nodes in master-master replication so each appliance can be used alone to serve the application in case the other(s) fail. (ALERT! MYSQLR64 does not automatically take care of failures, if a MYSQL server fails, your application should detect it and stop using it. )
  • Backup solutions - two (or more) nodes, one running as master, the rest running as slaves. The application uses only the master, the slaves are used for doing backup (in which case the database can be stopped to achieve consistency of the backup) or just for doing heavy analysis of the data (reports, summaries or anything that does not change the data) without loading the master, thus not influencing the performance of the application. (ALERT! MYSQLR64 does not automatically backup your database, you need to manage the backup yourself. )

In its default configuration MYSQLR64 acts exactly as a MYSQL5 appliance with a web interface for management. In order to use it for replication scenarios, you need at least two MYSQLR64 appliances with an appropriate configuration (see Typical Usage).

MYSQLR64 stores the database on an application-defined volume that can be configured on each MYSQLR64 instance. MYSQLR64 automatically creates an empty database when it starts on an empty volume.

ALERT! See notes on adding database users for the MYSQLR64 appliance.

Name Latest Version OS !MySQL Notes
MYSQLR 1.6.13-1 CentOS 5 5.0.45  
MYSQLR64 1.6.13-1 CentOS 5 (64 bit) 5.0.77  

ALERT! The 32-bit and 64-bit versions are NOT COMPATIBLE. You should not mix 32-bit and 64-bit MYSQLR appliances when using replication as the database files are copied "as is" from the master to the slave. Also, data volumes from the 32-bit version of the appliance should not be used with a 64-bit version of the same appliance (and vice versa). In order to migrate a database between the 32-bit and 64-bit MYSQLR versions, dump the databases on one host and import them on the other as described here.

Boundary

Resources

Resource Minimum Maximum Default
CPU 0.10 16 0.40
Memory 128 MB 32G 512 MB
Bandwidth 1 Mbps 2 Gbps 250 Mbps

Terminals

Name Dir Protocol Description
in in mysql Receives MySQL database requests.
rin in any Slave MYSQLR64 appliances that use the appliance as a master connect to this terminal.
ui in http Provides access to the web interface of MYSQLR64.
log out cifs Connect to a NAS appliance for storing error logs. This terminal may be left unconnected if not used.
rout out any Connects to a master MYSQLR64 server. This terminal may be left unconnected and should be used only in replication scenarios.
mon out cce Sends performance and resource usage statistics. This terminal may be left unconnected.

The default interface is enabled. It is intended for diagnostics and troubleshooting (over ssh). Future versions of this appliance may disable ssh access.
ALERT! When connected, the rin and rout terminals are used for both ssh (tcp 22) and mysql (tcp 3306) data. When gateways / VPN is used to connect those terminals, the firewalls should be configured to allow both ports.

User Volumes

Volume Description
data Volume used for the database data storage. This volume is mandatory.
binlogs Volume used for binary logs when running in replication mode (either as master or slave). This volume is not mandatory, but if you use the appliance in replication (set rpl_mode to be something other than none) and you don't provide a binlogs volume, the appliance will fail to start.
old_data Volume used for migrating data volumes used in old appliances (MYSQL, MYSQL5, MYSQL64). This volume is not mandatory.

The data volume can optionally contain a my.cnf file in its top directory, which includes MYSQL configuration options. See the 'Custom Configuration' section for more details. NEW This feature is available in MYSQLR64 1.6.1 or later.

ALERT! The data volume must be exclusively dedicated to the MYSQL instance (cannot be shared with other appliances).

ALERT! The data volume should not be formatted with any file-system as it will be used to build an LVM volume. If you use a volume that is formatted with any file-system (possibly with files from an old MYSQL appliance), the appliance will fail to start to prevent you from deleting your database/files.

ALERT! The old_data volume can be used only when the data volume is blank. In this case all data from old_data is transferred to the data volume by using the MYSQLR64 web interface. If you have supplied both data and old_data volumes and the data volume is not blank, the appliance will fail to start, preventing you from overwriting any data on the data volume. If you still want to transfer data from the old_data volume without using a blank data volume, start your application in maintenance mode, login to the appliance and copy the data from 'old_data' to 'data'.

Properties

Property name Type Description
auto_create Integer Whether to create the database if it doesn't exist. Possible values are 1 to create it and 0 to prevent auto creation (to avoid accidental overwrite in case of corrupted volumes). If set to 0 and a database doesn't exist on the data volume, the appliance starts in maintenance mode (appliance will start properly, but the MySQL deamon will not be started so that the user can check the problem). Default is 1.
error_log_filename String Name for the error log file, relative to the log file system (e.g., /mysql_logs/my.log). Directories in the path are auto created. If empty, error log is written to the data volume (/mnt/data/error.log). Default: (empty).
error_log_level String Error logging level. Possible values are: error logs only errors, warn logs both warnings and errors. This property is not case sensitive. Default: error
timezone String Specifies the time zone used in the appliance. If this property is empty, the timezone is not modified and left as-is. A list of supported time zones is available here. Default: empty

ALERT! As of MYSQLR64 1.6.8, the use_old_passwords property was removed. If you need to enable old_passwords, create a custom configuration as described below.

ALERT! The MYSQLR appliance will fail to start if the error_log_filename is specified and the log terminal is not connected or the file system cannot be mounted.

Advanced Properties (used in replication scenarios)

Property name Type Description
server_id Integer Server id. Possible values are from 1 to 10. This specifies the id of the server when doing replication. ALERT! Make sure you setup unique ids for all your servers that are part of the replication. Default: 1
rpl_mode String Replication mode. Possible values are none (no replication), master, slave, and master_and_slave (for multi-master replication scenarios where a server is a master and a slave at the same time). Default: none
web_pwd String Password for authentication to the web interface. This property is optional. If set, the appliance's http server is started and the web interface is exposed on both the ui terminal and the default interface where it is accessible through the Login (web) option in the AppLogic editor. Default: (empty).

Custom Configuration

NEW This feature is available in MYSQLR64 1.6.1 or later.

MYSQLR64 allows the use of a custom MYSQL configuration file which can provide additional configuration options or overwrite existing configuration specified in /etc/my.cnf.

To use a custom configuration, create a file named my.cnf and place it in the top directory of the data volume. The format of the file should follow the MYSQL options file syntax as described here.

For example, the following can be used to tune MYSQLR64 for better performance when using InnoDB? (the default MYSQLR64 configuration is optimized for MyISAM). The example is based on using 512M of memory (default for MYSQLR64).

[mysqld]
# Shrink down MyISAM buffers
key_buffer = 512K
myisam_sort_buffer_size = 512K

# Make InnoDB the default storage engine (optional)
default-storage-engine = INNODB

# Set InnoDB buffer size
innodb_buffer_pool_size=350M
innodb_log_file_size=128M
innodb_log_buffer_size=4M
innodb_thread_concurrency=8

# If you do not have too many tables use this option, so you will not have uncontrolled innodb main tablespace growth which you can’t reclaim.
innodb_file_per_table=1

ALERT! When used in replication mode, MYSQLR64 will also synchronize the my.cnf file on the data volume whenever you fix/initiate the replication, so your slave will have the same configuration as the master.

Web Interface

MYSQLR64 provides a web interface which can be accessed on both its ui terminal and its default interface through the Login (web) option in the AppLogic editor. Using the web interface requires http authentication. Leave the username blank and use the value of web_pwd as password. The interface has the following features:

tab_1_700px.png

  • view server status
    • start/stop/restart mysql service
    • mysql server status and uptime
    • mysql replication status (for this appliance only)
    • latest mysql log entries
    • appliance uptime
    • appliance memory usage
    • appliance disk usage

tab_2_700px.png

  • Migrate Content (visible only when old_data volume is present )
    • Migrate Content - copies the content of old_data volume to data volume. This is used to provide an easy upgrade from MYSQL and MYSQL5 appliances. All data on the data volume will be wiped.

tab_3_700px.png

  • view/manage replication
    • mysql replication status (for this appliance only)
    • initiate/fix replication. In this case the database on the appliance is wiped and the replication is started from scratch from the master. This does not affect the database on the master (no data is deleted on the master) and it does not introduce any downtime of the master MYSQLR64 appliance. During the repair, the appliance will be offline and should not be used.
    • delete snapshot - deletes a database snapshot that was created on this appliance in order to initiate/fix replication on its slave. In normal operation no snapshots will be left on the appliance, if you see the 'delete snapshot' button, either a slave is currently initiating replication from this appliance or a previous attempt to initiate replication failed. Make sure no slave is using the snapshot before deleting it. This option will not be visible if there is no snapshot on the appliance.
    • reset master log position - when adding a new server in a multi-node master-master replication, after running replication initiate/fix on the current appliance you need to login to the web interface of the appliance which connects through its 'rout' terminal to this appliance and click on 'reset master log position'.

tab_4_700px.png

  • Manage Databse
    • browse and edit your databases using PhPMyAdmin

Replication setup/maintenance!

To add a master-slave replication to an existing MYSQLR64 appliance without losing any data:

  • Set rpl_mode to master on the existing appliance
  • Add a new MYSQLR64 appliance to your application with an empty data volume. Connect its rout terminal to the rin terminal of the current appliance.
  • If you are going to use it in master-master replication, connect the rout terminal on the current appliance to the rin terminal of the new appliance
  • Restart the application so that the changes take effect.
  • Login to the web interface of the new appliance and select "initiate/fix replication" from the 'Manage Replication' tab. This will take some time depending on the size of the data on the master. If you use INSSL for accessing MYSQLR64, make sure the timeout property is set to a high value (36000) to avoid timeouts. Also make sure there are no timeouts in any proxies on the client side (better not use proxy at all).
  • After replication is initiated, login to the web interface on both MYSQLR64 appliances and check the replication status, in 5 minutes or less the replication should be running on both appliances.

To add a new MYSQLR64 appliance to an existing master-master replication without losing any data:

  • Add a new MYSQLR64 appliance to your application with an empty data volume. For example dbN, presuming that you already have N-1 MYSQLR64 appliances ( 3 <= N <= 10 ) and each appliance has its rout terminal connected to the rin terminal of the next appliance in a circular setup (db1 rout is connected to db2 rin and so on).
  • Set rpl_mode to master_and_slave of dbN .
  • Connect the rout terminal of the dbN-1 appliance to the rin terminal of the dbN appliance
  • Connect the rout terminal of the dbN appliance to the rin terminal of the db1 appliance.
  • Restart the application so that the changes take effect. After the restart, the replication will not be in sync until the end of the procedure, data written to one appliance may not be replicated on all other appliances for the time of the procedure.
  • Login to the web interface of the dbN appliance and select "initiate/fix replication" from the 'ManageReplication'. This will take some time depending on the size of the data on the master. If you use INSSL for accessing MYSQLR64, make sure the timeout property is set to a high value (36000) to avoid timeouts. Also make sure there are no timeouts in any proxies on the client side (better not use proxy at all).
  • After replication is initiated, login to the web interface of dbN-1 and select "reset master log position". This will make it read dbN-1 read dbN`s binary logs from the beginning.
  • Login on to web interface of all MYSQLR64 appliances and check the replication status, in 5 minutes or less the replication should be running on all appliances.

To fix a broken replication in a master-slave setup without losing any data:

  • Login to the web interface of the slave and select "initiate/fix replication". This will take some time depending on the size of the data on the master. During the operation the mysql service on the slave will be stopped. No downtime will be introduced on the master. This operation is equivalent to adding a new appliance to an existing MYSQLR64 appliance. If you use INSSL for accessing MYSQLR64, make sure the timeout property is set to a high value (36000) to avoid timeouts. Also make sure there are no timeouts in any proxies on the client side (better not use proxy at all).
  • After replication is initiated, login to the web interface on the slave MYSQLR64 appliances and check the replication status, in 5 minutes or less the replication should be running.

To fix a broken replication in a master-master setup without losing any data:

  • Login to the web interface of the appliance that reports the replication as failed and select "initiate/fix replication". This will take some time depending on the size of the database on the master. During the operation the mysql service on the appliance will be stopped. No downtime will be introduced on the master. The database will not be in sync among all masters until the end of the repair, database updates may not be replicated on all other MYSQLR64 appliances for the duration of the repair.
ALERT! All data on this appliance will be initialized from the master, so if there were any update to the database on the current appliance since the replication is broken, they will be lost. If this is the case, try to resolve the conflicts manually.
  • Login to the web interface of the appliance that has its rout terminal connected to the appliance that we run "fix replication" on. Select "reset master log position". This will make it read the binary logs of the "fixed" appliance from the begining. If you use INSSL for accessing MYSQLR64, make sure the timeout property is set to a high value (36000) to avoid timeouts. Also make sure there are no timeouts in any proxies on the client side (better not use proxy at all).
  • Login on to web interface of all MYSQLR64 appliances and check the replication status, in 5 minutes or less the replication should be running on all appliances.

Replication monitoring

There is a cron job that monitors the replication between MYSQLR64 appliances. In case the replication is enabled, the cron job runs every two minutes and sends alerts to the grid dashboard in the following cases:
  • Replication is not running (the following appears on the grid dashboard Replication of master server is not running, please check!)
  • Replication is running but it is too much behind the master (the following appears on the grid dashboard Replication is too much behind master, please check!)
  • Disk space on the data volume is low (< 20% free). This is checked regardless of whether the replication is enabled or not (the following appears on the grid dashboard Free space on the data volume is running low, please check!)
  • Disk space on the binlogs volume is low (< 20% free). This is checked regardless of whether the replication is enabled or not (the following appears on the grid dashboard Free space on the binlogs volume is running low, please check!)
In such cases, the user should solve the problem manually.

In case of failed replication, you can use the web interface to fix it as described in the section above.

Custom Counters

The MYSQLR64 appliance reports the following custom counters through the mon terminal.

The following counters belong to the MySql counter group:

Counter name Description
Aborted Clients Number of clients aborted by the server
Aborted Connections Number of connects aborted by the server
Bytes Received Number of received bytes
Bytes Sent Number of sent bytes
Total Connections Number of connections
Questions Total number of questions
Slow Queries Number of slow queries
Threads Created Number of threads created
Threads Connected Number of threads connected
Threads Running Number of threads running
Max Used Connections Number of max connections used
Open Files Number of open files
Admin Commands Number of admin commands
Alter Table Commands Number of alter table commands
Analyze Commands Number of analyze commands
Backup Table Commands Number of backup table commands
Change DB Commands Number of change DB commands
Change Master Commands Number of change master commands
Check Commands Number of check commands
Commit Commands Number of commit commands
Create DB Commands Number of create DB commands
Create Function Commands Number of create function commands
Create Index Commands Number of create index commands
Create Table Commands Number of create table commands
Delete Commands Number of delete commands
Drop DB Commands Number of drop DB commands
Drop Function Commands Number of drop function commands
Drop Index Commands Number of drop index commands
Drop Table Commands Number of drop table commands
Flush Commands Number of flush commands
Grant Commands Number of grant commands
Insert Commands Number of insert commands
Insert Select Commands Number of insert select commands
Kill Commands Number of kill commands
Load Commands Number of load commands
Load Master Table Commands Number of load master table commands
Lock Tables Commands Number of lock tables commands
Optimize Commands Number of optimize commands
Purge Commands Number of purge commands
Rename Table Commands Number of rename table commands
Repair Commands Number of repair commands
Replace Commands Number of replace commands
Replace Select Commands Number of replace select commands
Reset Commands Number of reset commands
Restore Table Commands Number of restore table commands
Revoke Commands Number of revoke commands
Rollback Commands Number of rollback commands
Select Commands Number of select commands
Set Option Commands Number of set option commands
Truncate Commands Number of truncate commands
Unlock Tables Commands Number of unlock tables commands
Update Commands Number of update commands

Error Messages

The following messages may appear in either the appliance log file or the system log of the grid controller when the appliance fails to start:

Error message Description
Failed to set timezone! Failed to set the appliance timezone as configured by the timezone property.
Appliance is running in [$rpl_mode] replication mode but binlogs volume is missing The appliance is configured to run either as master, slave or master_and_slave but no binlogs volume has been given.
Appliance is running in [$rpl_mode] replication mode but the 'rout' terminal is not connected The appliance is configured to run either as slave or master_and_slave but the rout terminal is not connected.
The 'rout' terminal is connected, but the [rpl_mode] property is set to 'none'. Either configure replication via the [rpl_mode] property or disconnect the 'rout' terminal The rout terminal is connected, but the [rpl_mode] property is set to none. Either configure replication via the [rpl_mode] property or disconnect the 'rout' terminal.
Failed to start mysql due to error_log_filename set and log terminal not connected! The error_log_filename property is configured but the log terminal is not connected.
Failed to mount share through log terminal! The appliance was configured to write logs on the log terminal, but it failed to mount the share on the log terminal.
The share through the log terminal is not writeable! The share on the log terminal is not writable.
Failed to create logdir [$logdir] on the log terminal! Failed to create logdir [$logdir] on the log terminal!
The logdir [$logdir] is not writeable! The logdir [$logdir] on the log terminal is not writeable!
The logfile [$error_log] is not writable! The logfile [$error_log] on the log terminal is not writable!
Failed to create database! Appliance was started with no database and it failed to install mysql databases.
Failed to setup replication! Appliance failed to configure replication.
Failed to start mysql! The MySQL daemon could not be started.
Insufficient permissions in the mysql database! Either the permissions for the 'root'@'%' are insufficient or if used in replication mode, the 'replication_user'@'%' does not have enough permissions to run MySQL replication.
Web interface failed to start! Web interface failed to start!

Additionally, the following errors may appear on the grid dashboard while the appliance is running:

Error message Description

Free space on the data volume is running low, please check! Free space on the data volume is below 20%.
Replication of master server is not running, please check! Replication of master server is not running.
Replication of slave is too much behind master, please check! Replication of slave is too much behind master.
Free space on the binlogs volume is running low, please check! Free space on the binlogs volume is below 20%.

Typical Usage

Simple two-tier application (no replication)

The following diagram shows a typical usage of the MYSQLR64 appliance in a two-tier web application:

use-case-1.png

Appliances in use:

  • user - input gateway
  • web - web server
  • db - MYSQLR64 database appliance
  • logs - NAS appliance for storing log files
  • content - NAS appliance for storing web server content
  • admin - input gateway for viewing log files
  • mon - MON appliance

Client request arrives on the user gateway. The gateway forwards the requests to the web server, which serves the request. When scripts (e.g., Perl or PHP) on web need to access persistent data, they use the db appliance through the out terminal of the web server. The db appliance is configured to store its log files within the root directory of the share exposed by logs.

Using a browser, administrators connect to the admin gateway in order to view the mysql or web server log files. The admin gateway forwards the requests to the logs NAS appliance.

Example property configuration (properties that are not listed should be left to their default values):

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level

ALERT! Note that the data volume must also be configured on the db appliance as well as the logs, content and mon appliances. To create application volumes that can be used here, see the Editor Manage Volumes help topic.

Scalable two-tier application (no replication)

The following diagram shows a typical usage of the MYSQLR64 appliance in a two-tier web application in which the database is used to share state and data between multiple, load-balanced web servers. In addition, this example has a separate input for maintenance, through which an administrator can log in and access the database for maintenance as well as an input through which an administrator can log in and view the mysql error log.

use-case-2.png

Appliances in use:

  • user - input gateway for user requests
  • web_lb - web load-balancer for user requests
  • web1, web2 - web servers with active content (e.g., CGI scripts)
  • db - database
  • logs - storage for database and web servers error log files and access to them
  • content - storage for web servers content
  • maint - input gateway for maintenance access
  • admin - a generic Linux5 server used by the administrator
  • gway - a network gateway for administrator's access to the Internet
  • mon - MON appliance

Client request arrives on the user gateway. The gateway forwards the requests to the web_lb load balancer, which directs the request to one of the web servers web1 and web2. The web servers access the db database.

The db database and web1 and web2 servers write their log files to the logs appliance through the log terminals. In addition, an administrator can log in through the maint gateway to the logs appliance and view the log files.

Additionally, an administrator can log in over ssh through the maint gateway to the admin server (public-private keys need to be setup). From the admin server, the administrator can access the db database for statistics or changing the database schema. The admin server can access the Internet through the gway gateway, for example, to download a newer version of libraries or the database schema.

Example property configuration (properties that are not listed should be left to their default values):

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level

ALERT! Note that the data volume must also be configured on the db, logs, content and mon appliances.

IDEA! The maint, admin, gway, mon and log appliances are not required for the operation of the two-tier application. If present, the admin server may have cron jobs for database scrubbing, sending e-mail, etc.

N-tier application with master-slave replication (suitable for doing backups)

The following diagram shows a typical usage of the MYSQL appliance in a web application in which the database is replicated to a slave server. The slave server can be used to make consistent backups of the data without stopping the master server, thus introducing zero downtime of the application.

master-slave.png

Appliances in use:

  • user - input gateway for user requests
  • admin - input gateway for log files access and web access to MYSQLR64 appliance
  • sw - split http traffic to each MYSQLR64 appliance
  • web_lb - web load-balancer for user requests
  • web1, web2 - web servers with active content (e.g., CGI scripts)
  • master - MYSQLR64 configured as master
  • slave - MYSQLR64 configured as slave
  • content - storage for web servers content
  • logs - storage for databases and web servers log files
  • mon - MON appliance

Client request arrives on the user gateway. The gateway forwards the requests to the web_lb load balancer, which directs the request to one of the web servers web1 and web2. The web servers access the master database.

The slave appliance is connected to the master appliance and replicates its data. slave can be stopped at any time for doing consistent backups of the SQL data or heavy analytics without interfering with the performance of the master appliance and the rest of the application.

Web access to master and slave is available via admin gateway on port 8080 and 8081.

The master, slave, web1 and web2 appliances are configured to store their log files within the root directory of the share exposed by logs. In addition, an administrator can view log files through the admin gateway.

Example property configuration (properties that are not listed should be left to their default values):

master

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename master-db.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 1 Master server (not mandatory to be 1, should be different from server_id on the slave)
rpl_mode master Write binary logs in order to have replication

slave

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename slave-db.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 2 Slave server (not mandatory to be 2, should be different from server_id on the master)
rpl_mode slave Connect to master

ALERT! Note that the data volume must be configured on logs, content and mon appliances.

ALERT! Note that the data and binlogs volumes must be configured on both the master and the slave. They should have enough space to handle both the data and the logs.

ALERT! Each appliance should use its own volume (it cannot be shared between the appliances).

IDEA! The admin, mon and log appliances are not required for the operation of the replication.

N-tier application with master-master replication (suitable for doing load balancing)

The following diagram shows a typical usage of the MYSQLR64 appliance in a web application in which the database is replicated to two servers in a master-master replication scenario. In this use case, the application uses both WEB and MYSQLR64 servers during operation for load-balancing. Also, in the case where one of the WEB/MYSQLR64 instances fails, the other WEB/MYSQLR64 instance can be used to prevent application downtime.

master-master.png

Appliances in use:

  • user - input gateway for user requests
  • admin - input gateway for log files access
  • sw - split http traffic to each MYSQLR64 appliance
  • web_lb - web load-balancer for user requests
  • web1, web2 - web servers with active content (e.g., CGI scripts)
  • db1 and db2 - MYSQLR64 appliances configured to be both master and slave at the same time
  • content - storage for web servers content
  • logs - storage for databases and web servers log files
  • mon - MON appliance

Client request arrives on the user gateway. The gateway forwards the requests to the web_lb load balancer, which directs the request to one of the web servers web1 and web2. web1 uses the db1 database appliance, web2 uses the db2 database appliance. db1 and db2 are connected in order to replicate the updates that the web servers do to the database. Each MYSQLR64 appliance uses an offset (equal to its server_id) for its auto_increment columns so that no duplicate entries occur.

Web access to db1 and db2 is available via admin gateway on port 8080 and 8081.

The db1, db2, web1 and web2 appliances are configured to store their log files within the root directory of the share exposed by logs. In addition, an administrator can view log files through the admin gateway.

Example property configuration (properties that are not listed should be left to their default values):

db1

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db1.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 1 Master server (not mandatory to be 1, should be different from server_id on the slave)
rpl_mode master_and_slave master and slave

db2

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db2.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 2 Master server (not mandatory to be 1, should be different from server_id on the slave)
rpl_mode master_and_slave master and slave

ALERT! Note that the data volume must be configured on logs, content and mon appliances.

ALERT! Note that the data and binlogs volumes must be configured on both db1 and db2. Each appliance should use its own volume (it cannot be shared between the appliances).

ALERT! When doing master-master replication you need to start with empty data volumes.

IDEA! The admin, mon and log appliances are not required for the operation of the replication.

N-tier application with multi-node master-master replication (suitable for doing load balancing)

The following diagram shows a typical usage of the MYSQLR64 appliance in a web application in which the database is replicated to four servers in a master-master replication scenario. In this use case, the application uses all of the WEB and MYSQLR64 servers during operation for load-balancing. Also, in the case where one of the WEB/MYSQLR64 instances fails, the other WEB/MYSQLR64 instances can be used to prevent application downtime(MYSQLR64 does not take care of failures).

multi-master.png

Appliances in use:

  • user - input gateway for user requests
  • admin - input gateway for log files access
  • sw - split http traffic to each MYSQLR64 appliance
  • web_lb - web load-balancer for user requests
  • web1, web2, web3, web4 - web servers with active content (e.g., CGI scripts)
  • db1, db2, db3, db4 - MYSQLR64 configured to be both master and slave at the same time
  • content - storage for database error log files, web content and web logs
  • logs - storage for databases and web servers log files
  • mon - MON appliance

Client request arrives on the user gateway. The gateway forwards the requests to the web_lb load balancer, which directs the request to one of the web servers web1, web2, web3 and web4. Each web server uses its own database appliance. All database appliances are connected in a circular way in order to replicate the updates that the web servers do to the database. Thus an update to db1 for example is replicated to db2, db3 and db4. Each MYSQLR64 appliance uses an offset (equal to its server_id) for its auto_increment columns so that no duplicate entries occur.

Web access to db1, db2, db3, db4 is available via admin gateway on port 8080, 8081, 8082 and 8083.

The db1, db2, web1 and web2 appliances are configured to store their log files within the root directory of the share exposed by logs. In addition, an administrator can view log files through the admin gateway.

Example property configuration (properties that are not listed should be left to their default values):

db1

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db1.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 1 Master server 1
rpl_mode master_and_slave master and slave

db2

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db2.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 2 Master server 2
rpl_mode master_and_slave master and slave

db3

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db3.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 3 Master server 3
rpl_mode master_and_slave master and slave

db4

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename db4.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 4 Master server 4
rpl_mode master_and_slave master and slave

ALERT! Note that the data volume must be configured on logs, content and mon appliances.

ALERT! Note that the data and binlogs volume must also be configured on all db appliances. Each appliance should use its own volume (it cannot be shared between the appliances).

ALERT! When doing multi-master replication you need to start with empty data volumes.

IDEA! The admin, mon and log appliances are not required for the operation of the replication.

N-tier application running on different facilities (suitable for doing load balancing and failover)

The following diagram shows a typical usage of the MYSQLR64 appliance in a web application running in more than one facility. With this setup you can have two or more identical applications running in different facilities with the database being replicated to all applications in master-master setup. This is useful in two cases:

  • geographical load balancing - setup 2 (or more) such applications so that users can use the application that is closest to their geographical region for better response times (load balancing is done with DNS)
  • failover - setup 2 (or more) such applications so if the whole facility where one application is running goes down (or the application itself goes down), the other applications can be used to prevent downtime.

Master application

remote-master.png

Slave application

remote-slave.png

Appliances in use:

  • user - input gateway for user requests
  • admin - input gateway for log files access
  • sw - redirect port 8080 from admin to ui on master or slave
  • vpn - input for the remote application to connect to the master=/=slave appliance in order to replicate the database
  • web_lb - web load-balancer for user requests
  • web1, web2 - web servers with active content (e.g., CGI scripts)
  • master - MYSQLR64 configured to be a master
  • slave - MYSQLR64 configured to be a slave
  • content - storage for database error log files, web content and web logs
  • logs - storage for database error log files

Client request arrive on the user gateway. The gateway forwards the requests to the web_lb load balancer, which directs the request to one of the web servers web1 and web2. The web servers access the master database. The master appliance connects to the remote (slave) application, the only difference being the server_id of slave and the network setup) in order to replicate the database. The remote application connects to the master appliance via the vpn gateway which is configured to allow connection only from the vpn gateway of the remote application. The master and slave appliances in the two applications are running in master-master setup so they always have identical data.

Example property configuration (properties that are not listed should be left to their default values):

Web access to master and slave is available via admin gateway on port 8080.

master

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename master-db.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 1 Master server (not mandatory to be 1, should be different from server_id on the slave)
rpl_mode master Write binary logs in order to have replication


master vpn

Property name Value Notes
mode server Operate as a server.
tunnel certificates Using ssl certificates.
tcp_ports 3306,22 Allow ports needed by MYSQLR64.
ip_addr master_vpn_ip IP address of the VPN in the master application.
remote_host slave_vpn_ip IP address of the VPN in the slave application.

slave

Property name Value Notes
auto_create 1 Create the database if the volumes are empty.
error_log_filename slave-db.error Name of error log file that is to be stored on the logs data volume.
error_log_level error Error logging level
server_id 2 Slave server (not mandatory to be 2, should be different from server_id on the master)
rpl_mode slave Connect to master


slave vpn

Property name Value Notes
mode client Operate as a client.
tunnel certificates Using ssl certificates.
auth_path "client1" Path to the ssl certificate file.
ip_addr slave_vpn_ip IP address of the VPN in the slave application.
remote_host master_vpn_ip IP address of the VPN in the master application.

The remote application is an exact copy, the only difference is the network setup of the user, admin, and vpn appliances, connection(s) between the vpn appliance and master=/=slave, and the server_id of the master=/=slave appliance (it should be unique).

Migrating from MYSQLR to MYSQLR64 (and vice versa)

If you need to migrate from MYSQLR to MYSQLR64 (and vice versa), you should not just use the volumes from the 32-bit appliance on a 64-bit one (or vice versa) as this could cause data corruption. The recommended way to do this is by dumping the database on the old appliance, transferring the dumped file to the new appliance, and then import the database into the new appliance. Here is the procedure for doing this:

  • (optional) Prevent access to the old database - this is optional and it is only to ensure that the database is not modified while it is being dumped. One way to do this would be to start the appliance with nothing connected on its input terminals.
  • Dump all databases on the old appliance. Make sure you have enough space on the data volume before doing this (you can always resize the data volume if there is not enough space). This is done by starting the appliance, log into the appliance and execute the following command:
    • mysqldump --all-databases --lock-all-tables | gzip > /mnt/data/my_dump.sql.gz
  • Stop the application.
  • Create a new application volume that is going to store the database for the new appliance (ext3-snapshot). Make sure the volume is large enough to hold the database dump file (/mnt/data/my_dump.sql.gz) and the database itself.
  • Transfer the database dump file to the newly created volume (below is only one way of copying the database using 2 filers setup with external internet access; if this is not suitable for your configuration, the database may also be copied using scp from the grid to an external server and then back to the destination volume):
    • Manage each volume seperately using two filers (one for each volume) using the vol manage command. Be sure to pass the external IP information on the vol manage command line so each filer can access each other in order to copy the database between the two volumes. See the vol manage help for more information on how to specify the external network settings.
    • In the filer for the source volume, execute the following: cat /etc/ssh/ssh_host_rsa_key.pub >> /root/.ssh/authorized_keys
    • In the filer for the destination volume, execute the following: scp -pr -i /etc/ssh/ssh_host_rsa_key IP_address_to_source_filer:/mnt/vol/my_dump.sql.gz /mnt/vol/.
    • After the database is copied, you may exit from both filers.
  • Edit the application and add a new instance of MYSQLR/MYSQLR64, parameterize it, connect it and assign the newly created volume to the new instance. At this point the old MYSQLR/MYSQLR64 instance may be deleted from the application (however you may want to keep it around for backup purposes until you can verify that the database was transferred correctly to the new instance).
  • Import the database - Start the new appliance, log into it and execute the following command:
    • zcat /mnt/data/my_dump.sql.gz | mysql
  • The database is now transferred and the appliance is ready for use.

Notes

  • MYSQLR64 performance is optimized for the MyISAM storage engine which is used by default. If you want to enhance the performance for Innodb or just fine tune the MYSQL servie for your application, see the 'Custom Configuration' section for details on how to add configuration parameters to the existing configuration. NEW This feature is available in MYSQLR64 1.6.1 or later.

  • MYSQLR64 uses only asynchronous replication (synchronous replication is not supported by MySQL).

  • MYSQLR64 detects replication errors within 2 minutes and notifies the user by posting a message on the grid's dashboard. The user can then use the MYSQLR64 GUI to fix the issue (no automated recovery is provided by MYSQLR64).

  • When MYSQLR64 is started, there is a message written to the appliance log file stating that chmod failed to modify the owner for the mysql log file. This is to be expected and does not affect the operation of the MYSQLR64 appliance.

  • The theoretical maximum number of MYSQLR64 appliances in a single replication is 10. As data is replicated in a circular way, increasing the number of MYSQLR64 appliances will increase the time it takes for slaves to update their databases. The actual maximum of appliances in a replication may be lower depending on your application, load and needs (instant update of slaves may not be critical in some scenarios like data backup).

  • ALERT! When creating users for the database, make sure all users are created with no restrictions on the host they connect from. For example:

mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'%'
    ->     IDENTIFIED BY 'some_pass' WITH GRANT OPTION;

  • Volume requirements
    • The data volume size should be 100MB or more - currently, the InnoDB engine is enabled and it occupies about 22MB on the data volume. In this version, there is no way to disable this, even if you don't use InnoDB for data storage.
    • The data volume should not be formatted with any file-system as it will be used to build an LVM volume (the volume should be unformatted). If you use a volume that is formatted with any file-system (possibly with files from an old MYSQL appliance), the appliance will fail to start to prevent you from deleting your files.
    • 20% of the space of the data volume is allocated for initiating/fixing replication, this leaves 80% of the volume space for database storage.
    • When using MYSQLR64 in replication scenarios either as master, slave or master_and_slave, you must provide a binlogs volume. Binary logs are rotated on 100MB and kept for 2 days after rotating, after that they are deleted. If the appliance is also running as slave, relay logs are also stored to the binlogs volume .You will need at least 500MB binlogs space in order to run master-master replication. The actual size depends on the intensity of MySQL update/insert queries that are executed, you`ll need a few GB binlogs volume for a normal operation of an application.
  • The appliance connected to MYSQLR64 is responsible for creating its own database and tables if they do not exist. Alternatively, you can manually copy the initial database on the data volume.

  • The appliance or server connected to MYSQLR64's in input should have the MySQL client library in order to access the MySQL server and submit requests. All virtual servers provided in the appliance catalog have the MySQL client library installed, including the generic LINUX server.

3rd party open source software used inside of the appliance

MYSQLR and MYSQLR64 use the following 3rd party open source packages in addition to the 3rd party open source packages used by its base class LUX5 and LUX64 respectively.

Software Version Modified License Notes
aspell 0.60.3-7.1 No LGPLv2.1 N/A
aspell-en 6.0-2.1 No LGPLv2.1 N/A
curl 7.15.5-2 No MIT N/A
device-mapper-event 1.02.32-1 No GPLv2 N/A
freetype 1.02.32-1 No FTL N/A
gmp 4.1.4-10.el5 No LGPLV2.1 N/A
libidn 0.6.5-1.1 No LGPLv2.1 N/A
libjpeg 6b-37 No Distributable N/A
libpng 1.2.10-7.0.2 No zlib/libpng N/A
lvm2 2.6.26-2.1.2.8 No GPLv2.0 N/A
mysql 5.0.77-3.el5 No GPL N/A
mysql-server 5.0.77-3.el5 No GPLv2 N/A
perl-DBD-MySQL 3.0007-2.el5 No Artistic N/A
perl-DBI 1.52-2.el5 No Artistic N/A
php-cli 5.1.6-23.el5 No PHPv3.01 N/A
php-common 5.1.6-23.el5 No PHPv3.01 N/A
php-gd 5.1.6-23.el5 No PHPv3.01 N/A
php-mbstring 5.1.6-23.el5 No PHPv3.01 N/A
php-mysql 5.1.6-23.el5 No PHPv3.01 N/A
php-pdo 5.1.6-23.el5 No PHPv3.01 N/A
rsync 2.6.8-3.1 No GPLv2 N/A
samba-client 3.0.28-1.el5_2.1 No GPLv2 N/A
samba-common 3.0.28-1.el5_2.1 No GPLv2 N/A
sudo 1.6.8p12-10 No ISC N/A
lighttpd 1.4.18-1.el5.rf No BSD N/A
perl-IPC-Run 0.84-1.el5.rf No Artistic N/A
perl-Time-Duration 1.06-1.el5.rf No Artistic N/A
phpMyAdmin 2.11.10-1 No GPLv2 N/A

To see the full list of open source packages used in this appliance, please see its Implementation Design

Related Documents

Questions and Comments

IDEA! To post a question or comment on this appliance, visit our Forum.

-- PavelGeorgiev - 29 Jan 2010

 
Copyright © CA 2005-2011. All Rights Reserved.
%