Tech Debate

For Sheboygan County, Wis., the demand for electronic storage is like a raging animal with an insatiable appetite. Last fall, the county took action to feed the beast. Instead of purchasing servers with high-capacity hard drives—an approach known as direct-attached storage—the county chose a more flexible strategy of pooling storage resources in a storage area network (SAN).

“We can add more capacity as we need it,” says Joyce Schneider, information systems director for the county. Expansion ease is important for the county. She estimates the new 1.3 terabyte (TB) drives in the SAN may be full within 18 months if growth continues at its current pace.

Sheboygan County had considered SANs for the past three years, but managers needed the time to gather funding and to consider a fundamental choice facing all SAN users: Which networking technology is the best for connecting storage resources to a main local area network (LAN)?

For Sheboygan County, the choice was clear. It opted for Fibre Channel, the long-time SAN standard known for high-throughput speeds and reliability.

However, a growing number of government CIOs are ignoring tradition and choosing a popular alternative known as Internet Protocol (IP) storage, which usually relies on an industry standard known as Internet Small Computer System Interface (iSCSI). This networking alternative uses common Internet protocols and ubiquitous LAN standards in SAN implementations. While slower than Fibre Channel, IP storage offers both lower costs and easier management for public sector IT shops that may be short on Fibre Channel specialists.

“We’ve been running everything on our IP storage unit for the last two years, and we’ve never lost a piece of data,” says John DeNardo, director of innovation and technology for Eagle County, Colo.—home, among other things, to fashionable ski resorts in Vail and Beaver Creek.

With SAN fans on both sides, it’s sometimes difficult to know which networking alternative is the right one to choose. Some experts ask, “Why bother choosing?” For them, mixing and matching Fibre Channel and iSCSI technologies is the sensible alternative.

Scalable and Flexible

SANs are a hot technology among state and local CIOs because they offer a scalable, resource-efficient and cost-effective way to deal with a widespread problem: the almost staggering annual increase in storage capacity requirements. (See “What’s Driving SANs ?” on page 26.)

As the demand threatens to outpace resources, an increasing number of governments turn to SANs because the units are designed for expansion. IT managers can plug additional hard drives into the SAN units as needed, thanks to modular internal racks that offer the physical space for more capacity. When a SAN unit hits its maximum capacity, managers can attach additional units to their networks for almost unlimited scalability.

SANs offer other flexibility features as well. IT managers can dynamically allocate space in the SAN’s hard drives. So, if the demands of one application are growing to capacity, a few clicks in a storage management program can open up additional unused megabytes wherever they’re needed. Direct-attached storage, by contrast, allocates a set amount of space for each application, and the allocations are difficult to reconfigure.

In addition, SANs can help local governments preserve and protect data with dual SAN units that provide for data mirroring—sometimes in storage networks at different locations.

For years, the primary choice for IT managers wanting to obtain these storage benefits was Fibre Channel. It’s fast. Maximum throughput now reaches 2 gigabits per second, and, by year’s end, 4Gbps technology should be common. (A 10Gbps Fibre Channel standard exists, but it’s incompatible with the slower standards, so it’s not widely used in IT shops.)

Unfortunately, Fibre Channel has a reputation for being expensive and difficult to administer.

Enter iSCSI, which deftly capitalizes on the popular Transport Control Protocol/Internet Protocol (TCP/IP) and Ethernet LAN technologies. Because it was a relatively new standard—the Internet Engineering Task Force didn’t adopt it as a formal standard until 2003—iSCSI wasn’t immediately embraced by CIOs, who were reluctant to trust critical data to an immature technology.

“A year ago, hardly anyone was using iSCSI in enterprise environments,” says Stephen Foskett, director of the strategy practice for GlassHouse Technologies, a Framingham, Mass., storage consultant. “But today, everything has changed.”

iSCSI is making inroads into private and public sector data centers, partly because the networking infrastructure to support it is in place at most organizations. iSCSI operates over virtual private networks that cover state and local government campuses. Facilities typically communicate with one other via IP networks, the same protocol that is used by iSCSI. Also, most organizations are familiar with IP networking, which is a help if a problem crops up.

“Organizations have the opportunity to piggyback on the people knowledge already in the institution for support issues,” points out Jack Scott, senior partner for the storage consulting firm Evaluator Group, which is based in Greenwood Village, Colo.

For many, this level of familiarity provides a justification for iSCSI’s slowness compared to Fibre Channel’s speed. Today’s iSCSI throughput tops out at 1Gbps. However, within a year, 10Gbps products should be available, although prices may initially make them out of reach for many public sector budgets, says Foskett of GlassHouse.

IP on the Rise

Eagle County was something of an IP storage pioneer when it installed its SAN in 2003. Still concerned about iSCSI’s maturity, the county installed a SAN that used a proprietary implementation of IP technology. Two years later, DeNardo feels more confident in the official standard and is in the process of converting his SAN to iSCSI.

When Eagle County made its IP storage move, it was running 25 servers with about one-half terabyte of direct-attached storage resources. The catalyst to move to a SAN came when Eagle County decided to revamp its entire IT infrastructure, which serves all of the county government departments, including about 75 custom applications running at various offices. The revamp included hundreds of new desktop PCs and network servers, but DeNardo also wanted the flexibility of a SAN.

DeNardo considered both Fibre Channel and IP when he spec’d out his SAN, but quickly settled on the latter. “We figured we could do Fibre Channel for a-quarter-million-dollars or IP for $80,000,” he recalls. IP’s price differential proved irresistible.

Although IP storage was good for the bottom line, DeNardo knew he was making a performance tradeoff. But it was a sacrifice he was willing to accept. “We are not a performance-intensive shop,” DeNardo reasons.

He adds that the 10Gbps iSCSI storage devices on the horizon will make the performance question moot. “We’ll put in a 10Gbps switch, and then we’ll be faster than Fibre Channel.”

Having in-house IP expertise was another incentive for choosing an IP SAN. “I have six people who know IP; nobody here knows Fibre Channel,” DeNardo says. He estimates the county would have had to spend $20,000 for Fibre Channel training to adequately educate one staff member. “My annual training budget for the whole staff is only $22,000,” he says.

Today, the SAN holds 12TB of storage capacity in two physically separate locations. The county mirrors the data on its SAN, and every night it sends snapshots of the information over an IP network to a nearby satellite building.

“In the event that this building totally goes away for some reason, we could go down to the other office and start bringing up our network,” DeNardo says. “People are spending hundreds of thousands of dollars trying to create that kind of backup ability. We are spending less than $100,000 for it. We believe the technology has already paid for itself.”

Fibre Rules

Despite DeNardo’s enthusiasm, not everyone is smitten by IP SANs. Charles County, Md., with offices in La Plata, installed a 350GB Fibre Channel SAN four years ago when Fibre Channel was the only practical option. When the county recently added another SAN, it didn’t switch to iSCSI.

“We considered iSCSI, but we are a small county government, and there wasn’t a big driving force to install collocated storage systems,” says Ty Fuqua, network administrator. Collocation is easy to implement when SANs hang off IP networks, but Charles County currently has no need for that. “For us, that [collocation] would be the primary reason to go with IP,” Fuqua explains.

The county now runs four SANs. The latest one, which cost $17,000, provides 2TB of capacity across 14 high-capacity hard drives. In addition to more capacity, the new SAN provides flexible resource allocation.

“I have the capability to assign half a terabyte to any application and have another half terabyte free to go to any application in the future,” says Matt Goddard, network specialist. “I can expand into the free space as people need it.”

Sheboygan’s Schneider is another Fibre Channel fan, mainly because of its impressive performance and reliability. The county purchased a SAN unit, with 10 146GB fibre drives and 10 320GB ATA drives, at a cost of about $150,000. Installed last fall, the SAN serves several departments and mirrors data as a foundation for the county’s disaster-recovery efforts.

The performance difference between the former direct-attached drives and the new SAN unit is significant, Schneider says. “When we used to do backups on our financials, it took almost 15 hours,” she recalls. “Now we can do it in a little over two hours.”

Schneider estimates that retrieving and restoring files in the old system would have required a day’s worth of work. Now, thanks to higher throughput speeds and functional features, the same backup procedure falls within the same two-hour-plus window.

“When we bought the system, we expected some of this functionality, but we didn’t realize the extent to which it would benefit us,” Schneider says. “It was a nice surprise.”

Because Sheboygan opted to install redundant power switches, fans and other obvious points of failure in the SAN, she feels the IT operations are less vulnerable to system crashes.

Mix and Match

Although Sheboygan, Eagle and Charles counties all run a single networking technology in their SANs, these IT shops are bucking a trend. GlassHouse’s Foskett says other organizations aren’t choosing Fibre Channel over iSCSI—or vice versa. Instead, they’re installing both, for different applications.

“iSCSI’s growth isn’t necessarily coming at the expense of Fibre Channel,” he notes. “Instead, organizations are choosing iSCSI for applications that had never seen SANs before.”

So how can a government IT shop choose the best technology for its needs? Foskett believes the best approach is to maintain flexibility and an open attitude about SAN technology.

Fibre Channel remains a popular choice for high-performance uses such as storage for multi-terabyte relational database systems and for high-end Unix or Windows servers. iSCSI, in contrast, seems best suited for workgroups and departments that have smaller storage requirements.

“If you’re not as concerned with performance, then by all means go with iSCSI,” Foskett says. “You’ll save money, and everything will work fine.