How to Meet High-Availability Archiving Expectations

When the Museum of the City of New York received a grant to digitize and post to the Internet more than 100,000 historical photographs, it was met with an entirely new storage archiving paradigm. Rather than just following retention best practices, the New York City organization needed to meet high-availability expectations by ensuring fast access to lost or damaged files.

Many state and local governments face similar dilemmas, according to Brian Babineau, vice president of research and analyst services for the Enterprise Strategy Group. "The intersection of regulatory mandates, born-digital content and self-service access requirements makes online storage a logical investment," says Babineau. "State and local governments are putting sophisticated retention and access systems into place to automate processes as much as possible."

But "sophisticated" doesn't necessarily equal "expensive." Compact and automated iSCSI storage area networks from Drobo offered the perfect solution for the Museum of the City of New York, says Juan Perez, director of IT for the institution. "The Drobos provide enterprise-class dual-drive redundancy with automated RAID 5," he says. "The drives are hot-swappable, and within an enclosure, drives can be different sizes. This permits us to add faster, high- capacity disks as it becomes cost-effective."

After initially purchasing five Drobo rack-mountable SANs, each with a 16– terabyte capacity, the museum recently invested in three larger Drobo B1200i SANs, each with 36TB capacity. Once a unit is full, it's transported to the agency's secure collocated site. "Eventually, we'll use the Drobo's replication capabilities to copy new files to the collocation, but we don't need that functionality yet," Perez notes.

Regardless, Drobo makes management a snap, Perez states. "The console is so easy to use," he says. "We set up e-mail alerts to notify us when a unit nears capacity; then we know it's ready to move offsite."

Although the use of the Drobo's dual-drive redundancy capability is optional, Perez says it optimizes his organization's resources. "It gives us plenty of time to replace a drive, should one fail."

Satisfaction with the Drobo appliances has led the museum to invest in two B800fs units for backing up 75 desktops. "Just like the SANs, they are very cost-effective per terabyte," Perez says.

Centralized Retention

Like the Museum of the City of New York, Minnesota's Washington County chose to invest in online storage to meet the convergence of retention requirements and access needs.

For this long and narrow jurisdiction, which runs about 40 miles north to south, sheer distance also played a role in the decision. "For our staff, there's nothing more frustrating, for example, than arriving at a constituent meeting only to discover the paper files they need are miles away," explains Mjyke Nelson, the county's director of IT.

To centralize data storage and archiving, Washington County relies on two HP SANs. Selected for ease of integration with other infrastructure components and compatibility with a virtualized server environment, one SAN is housed in the county's main government center and the other at a satellite location. "They're on different power grids to ensure continuity," Nelson says.

For the software layer, Washington County implemented Microsoft SharePoint Server. The county initially deployed SharePoint Server 2007, but has seen many archiving improvements since upgrading to SharePoint Server 2010. "We studied internal needs, existing solutions and the work process improvements required,"10-6 explains Nelson. "SharePoint offered the majority of the features we needed."

In addition to making files accessible anywhere, and thereby simultaneously improving efficiency and reducing travel-related costs, the county's archiving effort will also minimize physical storage requirements.

"We're working on reducing our paper-based storage capacity along with the expense of creating and managing hard copies," Nelson says.

A Needle in an Instant

At more than 100 million records and growing, the Washington State Archives boasts one of the most advanced retention facilities in the country.

"Since we established the facility in 2004, we've had visits from other countries as well as U.S. states and municipalities," says Jerry Handfield, state archivist. "Now, the municipalities in our state send us their records to archive." And thanks to a grant from the Library of Congress, the organization is beginning to provide archiving infrastructure to other states.

Highly sophisticated and automated, Washington's system requires only a 10-person development and administrative team to manage, Handfield says. The team works with a host of technology partners, including Cisco Systems, HP, Isilon Systems and Microsoft. Additionally, to permit the system to meet changing needs, upgrades and refreshes occur flexibly rather than on a rigid cycle.

For funding, the state charges a small document recording fee, which goes straight to the digital archives. "It's vital to have a dedicated, steady income stream," stresses Handfield. "In our case, we'll store your property deed forever in exchange for just one dollar."

Today, 40,000 unique users complete 500,000 archive searches on Washington's system every month. "Our storage needs are expanding exponentially, and I expect that rate to continue," Handfield says. "So our system evolves continuously — if something can be made faster, such as upgrading to new switches, we've already built into our budget the ability to keep up."