Backup Burden
The Florida Department of Transportation is about to hit a major milestone -- more than 500 servers virtualized. And while this is definitely cause for celebration, CIO Nelson Hill and his IT team know it's time to revisit the agency's backup and recovery strategy.
"The state has a legislative mandate to consolidate primary data centers by 2019, and that will probably mean a statewide standard for backup as well," Hill says. "We definitely want to provide a leadership role for this effort."
The Florida DOT IT team is examining virtualization-aware tools that make backups in their VMware environment more automated, scalable and reliable, and enable them to take advantage of advanced techniques such as data deduplication to save time and storage space.
"We are actively integrating Symantec's virtual backup components as well as its deduplication capabilities to improve the efficiency of backup and recovery," says Amy Price, manager of the enterprise server administration group. Because virtualization-aware software such as Backup Exec is tightly integrated with the virtual management software, it greatly reduces the backup load of the virtual host servers and introduces LAN-free backups to the virtual infrastructure.
The Florida Department of Transportation's IT team supports more than 9,000 users in its main office in Tallahassee and seven other districts.
It's these advantages that are driving government agencies to look for tools that allow backup management directly through a variety of virtual server environments, including VMware and Microsoft. Many backup and recovery software makers, including Symantec and Vizioncore, are heeding this call and tailoring their offerings to support virtualization.
Migration Patterns
Traditional server backup software requires placing an agent on each virtual machine and carefully monitoring backup windows to ensure they don't drain the host server's performance or get cut off if virtual machines move around. This poses quite an IT management challenge.
Moving to native virtual management for backup windows is imperative, according to Jeff Byrne, senior analyst at the Taneja Group.
"Running agent-based backup is hard on a virtual server environment. It requires CPU, memory and bandwidth," he says. "With hundreds of virtual machines moving data across the network, you wind up significantly degrading the performance of applications or having backup windows collide."
Backup costs for virtual servers can spiral out of control because most software is licensed on a per-agent basis, and often money must be spent on staff to manually monitor and manipulate backup windows. For these reasons, some government agencies have chosen to either back up their virtual systems less often or, in some cases, not at all. "These short-term trade-offs are putting their agencies at risk," Byrne says.
Andi Mann, vice president at Enterprise Management Associates, says legacy backup techniques are limiting because they require a system location, which is counter to the dynamic nature of virtual systems. For example, if VMware's VMotion kicks in during a backup and begins moving virtual machines around, it could corrupt the backup process.
Last, these agents are unable to perform granular backup of migrated, offline or paused images.
It was the granularity that attracted Charles Markakis, IT manager for the Maryland State Highway Administration, to Vizioncore's vRanger Pro Data Protection Platform. Over the past few years, the department has deployed about 150 virtual machines across 15 VMware ESX servers.
Markakis sought a backup tool capable of meeting the agency's needs. Every day at 5:00 p.m., every node on the network, including production and development, must successfully be backed up by the next morning. "That's what we promised our users in the physical environment, and we wanted to do the same in the virtual one," he says.
Originally, the team had used traditional backup software, but found that it didn't allow them to fulfill their promise to users, and the cost to put an agent on each virtual machine would quickly become prohibitive. Instead, Markakis needed a solution that managed the whole environment from the virtual core.
With vRanger Pro, which costs about $460 per CPU and includes a year of support, the agency can schedule incremental, differential or full-image backups through virtual system management software and ESX servers. The product also provides direct backup to various storage targets and instant file-level recovery.
In addition, because it's integrated into the virtual system management, vRanger Pro can monitor backup windows and automatically adjust them if they are overlapping or affecting system performance.
The Power of Dedupe
One of the biggest benefits of moving to a virtual-aware backup system is the ability to perform compression techniques such as data deduplication, which eliminates the transfer and storage of duplicate data.
Scott Norton, division administrator for technology support at the Illinois State Board of Education, says the de-duplication ratios he's realized using EMC's Avamar backup and recovery software have saved him from missing backup windows and building out his infrastructure to support increased storage. "With legacy backup software, it took us anywhere from 12 hours to three days to back up our virtual environment. We'd miss incremental backups just to have the weeklies -- it was an unacceptable situation," he says.
Now, because the data is de-duplicated at the virtual machine before it heads off to storage, most backups for his 70-plus virtual servers, which reside on five physical VMware ESX Server hosts, take less than three hours, many even less than an hour. "We're getting a 92 percent to 99 percent deduplication ratio, which is incredible," he says. Pricing for Avamar is about $17,000 for 1 terabyte of deduped storage.
Back at the Florida DOT, CIO Hill and his IT team are looking forward to gaining these benefits in the near future.
It Takes Teamwork
If there's one thing that server virtualization pioneers know, it's that if you want your virtual machine backups to be a success, get your server and storage teams collaborating early on.
"Storage management is one of the biggest issues that data center automation and virtual systems management teams face," says Andi Mann, vice president at Enterprise Management Associates.
Yet, too often, organizations think that because they are dealing with virtual systems, the physical storage team will not be of use. Mann says this couldn't be further from the truth. "The server team is not going to be as familiar with the backup needs of applications as the storage team," he says.
Therefore, he recommends that organizations build cross-functional teams that include servers, storage, network and applications representatives to plot out the virtual environment and ensure that backup windows have enough resources and are properly scheduled.
VMware user Alex Musicante, network analyst for the city of Pittsburgh, agrees. "There's a lot of policy that goes on with backups, and that doesn't change just because the machines are virtualized. So you want to make sure from the proof of concept onward that you have a good backup strategy. Otherwise, you'll leave a bad taste in everyone's mouth," he says.
