Staying on Message

Last winter, the judge presiding over a lawsuit brought by a former employee against the city of Watertown, Wis., instructed city officials to produce a series of old e-mail messages as part of the relevant public record. But the municipal government retained messages for just 30 days. In the end, a consultant was hired to retrieve files from a hard drive, and the city attorney’s office combed through them to identify which were part of the public record.

It was then that City Attorney Thomas Levi decided Watertown needed an e-mail archiving system. “Increasingly in Wisconsin, public-records laws are being used to obtain e-mail and other electronic documents,” Levi says. He points out the requirements are more stringent than for e-discovery, the process by which electronic documents are requested and produced to be used as evidence in legal cases. “The city must retain the electronic record or e-mail record for seven years. Even though we were not in compliance with the law up to that time, in this one case, I had to find the old computer containing the e-mail, hire a forensic expert to retrieve the e-mail from the disabled hard drive, and spend hours deciphering which records were public and which were personal.”

Couple public-records requests with the Federal Rules of Civil Procedure (FRCP), and it becomes clear that more government IT departments need to find a way to archive and retrieve electronically stored information as well as create and enforce policies that support compliance efforts.

In 2006, amendments to the FRCP specified that the discovery process applied to electronic documents and provided guidance as to how those documents should be handled. The amendments codified what had been a reality in the legal system for several years, according to David Goldstone, a partner at law firm Goodwin Procter in Boston.

“Courts have been treating electronic documents in the same way they treated paper documents for many years. It’s just that the number of electronic documents is multiplying exponentially. For example, some people send 50, 60, even 100 e-mails a day,” says Goldstone.

The issues raised by the FRCP and e-discovery for the private sector are exacerbated for state and local governments, which gather massive amounts of electronic information and often have limited resources to deal with the data, says Goldstone. Compared with private businesses, government entities with their agency’s and department’s mandates for public transparency must also deal with a wider array of retention requirements, he says.

Meanwhile, in Watertown

Like many other small cities, Watertown, with a population of about 23,000, lacks a formal IT department. Instead, a committee consisting of the city attorney and several employees with IT responsibilities in individual departments took on the job of finding and implementing an appropriate e-mail archiving technology. “It was a challenge, but city officials saw the archiving project as a priority and made funds available,” says Jay Haberkorn, a technician in the engineering department who, among his other duties, manages the server and e-mail system.

Watertown has plenty of company, as state and local governments of all sizes implement e-mail archiving to deal with regulatory and legal pressures, such as those created by the FRCP and e-discovery. The savings in time and money from improved storage management are welcome, but staying on the right side of the law is driving most new archiving efforts.

A key issue for Watertown’s committee was finding an archiving product that could operate across the city government’s three different e-mail platforms — Microsoft Exchange at city hall, Novell GroupWise at the police department and a hosted e-mail system for the water/wastewater department, says Haberkorn. Together the disparate platforms support more than 100 e-mail accounts.

“We thought about consolidating on Exchange, but the police department was reluctant to make the switch, so we found something that could tie all the systems together,” says Haberkorn.

Needing to support multiple platforms narrowed the committee’s options. The city soon settled on the Barracuda Message Archiver 350 appliance, an integrated hardware/software system from Barracuda Networks. The appliance itself cost approximately $8,000, and the city contracted for three years of support services for $2,700. In mid-May, Haberkorn and colleagues performed the rollout.

The first phase of the Message Archiver deployment, the step that sends e-mail messages into the archiving system as they are created, went smoothly.

“I was leery of us doing it ourselves, but city hall was up and running in a day, and there weren’t many problems with the other platforms,” he says. By the end of July, all existing stored e-mail had been imported into Message Archiver, to be stored on mirrored drives.

Measuring return on investment wasn’t a major consideration because compliance and legal protection for the city were the real drivers of the project, says Haberkorn.

Money and Time Savings

For the state of Missouri, the need for compliance added urgency to the decision, but the potential for huge savings in both money and staff time was what initially made e-mail archiving a priority, says Deputy CIO for Infrastructure Chris Wilkerson. With responsibility for approximately 34,000 state government e-mail accounts, the cost of combing through e-mail mounted quickly.

Before Missouri deployed a Symantec Enterprise Vault archiving system, the only way the state could respond to requests for data, including e-mail, was by sifting through backup tapes. Finding one day’s worth of e-mail for one user took about 30 minutes, Wilkerson says.

“There are sunshine requests and legal requests that can involve 20 users and a year or more of their e-mail,” he says. “Now you’re beginning to talk about months of effort and the cost that involves. The IT effort for a single legal review could easily cost us much more than $100,000.”

With the archiving system, one copy of every e-mail message is held in a single storage repository, even if it was sent to multiple users. Previously, multiple copies were retained, which complicated searches and ate up space on storage media, says Wilkerson. Enterprise Vault’s search algorithm identifies and retrieves messages by subject matter as well as name and makes e-mail reviews more focused and more thorough. Archiving e-mail allows individual departments to respond to requests for information, typically without involving the IT department, he says.

Missouri’s initial investment in Enterprise Vault hardware, software and related services was $1.6 million, with another $500,000 budgeted for ongoing services over the next several years. The system will pay for itself within 18 to 24 months, says Wilkerson.

Wilkerson’s Information Technology Services Division (ITSD) tested four leading e-mail archiving platforms: Computer Associates’ Message Manager, EMC’s EmailXtender, IBM’s CommonStore and Symantec’s Enterprise Vault. Key criteria used to evaluate the systems included vendor support, ability to delegate administration rights, available browser interface and ease of use, ability to capture historical e-mail files for the archive, support for e-discovery, and scalability.

“There’s no single factor on which the decision hinged; you look at a lot of things to find the technology best suited to your needs,” Wilkerson says.

The IT team had begun researching archiving technologies in July 2007. By late fall, Wilkerson was concerned about how he could sell the cost of the system to the Missouri Legislature when Gov. Matt Blunt asked ITSD to find a solution to the state’s e-mail management problems. Wilkerson’s proposal arrived on Blunt’s desk Dec. 31.

ITSD had 34,000 state accounts recording new e-mail in Enterprise Vault by June 18. The effort to capture old e-mail and bring any remaining accounts online to the archive should be complete by the end of the year.

The Enterprise Vault system is the first step in a much larger effort, Wilkerson says.

“E-mail archiving is just the tip of the iceberg. I wanted something that would mature with the evolving e-discovery process,” he says. “So far there’s no one solution; we need some combination of the Symantec tools and an enterprise content management system.”

Early Adopter Shifts Focus

“Nobody was doing this when we started,” says Bellevue Network Manager Shayne Williams of the Washington city’s 2004 EMC EmailXtender deployment. “Initially what drove us was managing our e-mail storage more effectively. We had information we weren’t using and wanted to create better access to it. Now requests for public information and rules for e-discovery have become much more important in our thinking.”

Bellevue purchased EmailXtender for $90,000. Maintenance for the system costs $15,000 annually. After multiple upgrades, Bellevue is currently using version 4.8 to archive messages from 1,500 Exchange accounts.

Because the system has been in place for so long and has gone through updates and revisions, Bellevue has not done a formal ROI calculation for the e-mail archiving system, but the benefits are clear, Williams says. Before the city started archiving, it took the IT department an average of three days to find and retrieve e-mail. Even with 15 hours a week of administration time, the archiving system saves hundreds, perhaps thousands, of hours a year, which translates into substantial cost savings.

As regulations covering e-mail retention, public requests for information and the demands of the e-discovery process all multiply at once, archiving is becoming a more complex undertaking, Williams says. To ensure the system meets the needs of various departments, Bellevue is moving toward user-directed archiving. Departments have long done their own e-mail retrieval through the archiving system, and now employees are being trained to set varying retention periods for messages that meet their specific requirements. Individual departments can also create folders in the archive that cross-reference related issues and speed retrieval.

“The legal department leads the way with 16 separate folders, but the capability is there for everyone,” Williams says. “You make sure the archiving system is updated to comply with laws and meet the needs of citizens and users, and then it proves its value all the time.”