Disaster Recovery Takes Off at Airport

If disaster struck the Phoenix metropolitan area, continued service at Phoenix Sky Harbor International Airport would be essential. Owned and operated by the City of Phoenix Aviation Department, the airport is one of the 10 busiest in the world. The aviation department’s IT group makes it their business to ensure uptime.

No business continuity plan or disaster planning exercise can succeed without the technology and people to put it into play when needed. The technology division kicked off a multiyear business continuity project in 2005, and the first phase went operational last fall. The result is real-time dual-site storage of all airport data through storage area networks, mirroring and server redundancy.

The plan hinges on hardware and software optimized for high availability and system redundancy. The aviation department started by establishing two separate, physical fiber optic paths to another location and established the foundation for replication from the data center to the secondary site. The data center houses an HP StorageWorks Enterprise Virtual Array 6000 accompanied by a combination of Brocade and Cisco Fibre Channel and iSCSI SAN switches and HP half-height blade servers complete with enclosures. All told, the data center warehouses approximately 26TB of data distributed across a combination of 300GB and 500GB disk drive/arrays and 32 blade servers.

Mission-critical systems and applications reside on VMware virtual servers and synchronously replicate to the secondary site. Phoenix’s aviation department uses virtualization to manage and provision the physical resources and components of the SAN as well as facilitate high availability and system redundancy. Our department’s network services team plans to migrate the majority of our 122 physical servers, services and applications to the established virtual environment. We’re also considering deploying new VMware modules, supplemental SAN technologies and storage arrays, and de-duplication technologies.

Thanks to the steps we’ve taken, systems and data can be recovered in the event of a disaster. The goal is to be able to continue to serve the community following a catastrophic event.

Subsequent phases of the project will continue virtual server clustering and remote location projects to ultimately ensure that mission-critical airport systems are capable of continued operations should a disaster affect the airport data center. All new aviation department systems will have redundant site server clustering, disk mirroring and virtualization built in to their project budget.

Like many other technology groups, we have learned a great deal through trial and error and reactive methodologies. We are at a point where we’re shifting from a reactive mode to a more proactive mode.

This involves a significant investment in equipment, software and especially personnel. All of our personnel will be trained and tested in the processes of recovery from various disasters, whether self-imposed or acts of nature.

Skilled and informed personnel are the key to any effective business continuity and disaster recovery process. Technicians should know every aspect of their systems and the impact on their respective environments in the event of a disaster.

Any amount of investment can be thrown at a risk in an effort to mitigate it, but without the skilled personnel to implement and manage such an investment, the process will ultimately fail.

Invest in your people. It’s the least expensive investment with the largest return.