From Crisis to Convergence

Human crises present enough challenges without an antiquated phone system that is unreliable and virtually unmanageable. A basic Voice over Internet Protocol (VoIP) implementation eliminated this problem for Michigan’s Department of Human Services while laying the foundation for next-generation converged applications.

With the old Time Division Multiplexing infrastructure largely beyond repair, DHS turned to VoIP, zipped through a 30-day comparison of three leading solutions, and mapped out a five-month rollout of Cisco’s IP telephony across 77 sites. Quickly learning that smaller steps make for a smoother migration, DHS abandoned this site-per-day schedule and made sure users at each location were seated securely on the VoIP bandwagon before moving on to the next site.

This patience paid off in fewer help desk calls and greater user satisfaction as it helped streamline communication with colleagues and service recipients. Next-generation applications will have to wait for migration to DHS’s remaining sites; meanwhile, users are learning new capabilities every day.

Beyond Repair

Like many geographically dispersed organizations in the pre-convergence era, Michigan’s Department of Human Services had collected an incompatible hodgepodge of voice technologies, including private branch exchange platforms, small key systems and Centrex service. Maintaining the aging equipment was increasingly inefficient and expensive; some of it was no longer supported by manufacturers.

With such antiquated voice technology, redundant configurations were either impossible or too expensive to contemplate, so critical services such as 24-hour crisis hotlines had no fall-back options in the event of system failures. The old infrastructure was also hampering the evolution of a mobile and location-independent workforce.

In early 2004, the Michigan IT Executive Council started looking at VoIP as a likely solution. Voice communication is particularly crucial in human services, and in October 2005, DHS — the state’s second-largest agency — was selected as the initial migration target. The total budget for the first stage, which would migrate 77 of the agency’s 148 sites, was $4.3 million.

“Short term, DHS wanted to replace the antiquated systems, centralize and standardize voice services, and save some money,” says Patrick Hale, Michigan’s director for infrastructure services. “Long term, the goal was to create a foundation for next-generation telephony services and applications.”

Picking a Platform

To choose the best VoIP solution, the convergence team decided on a bake-off among three leading IP telephony vendors whose products already figured heavily in Michigan’s legacy voice and data infrastructure. They piloted Avaya, Cisco and Nortel systems separately at three different sites.

One disadvantage of these brief 30-day pilots was their standalone nature: Because each environment was self-contained, the IT team was unable to test wide area network capabilities. Instead, the staff had to evaluate such capabilities on paper or accept vendor- supplied information.

IT polled users and managers at the end of the pilot period. The results showed all three solutions worked well with the underlying IP data network and there was good installation support from all three vendors. However, the tipping point was how the end users experienced them up close and personal.

The Michigan Department of Human Services’ VoIP rollout required lots of customization, says Ruth Goldman, telecommunications manager.

“We paid a lot of attention to the employees participating in the pilots,” says Ruth Goldman, the telecommunications manager for DHS. Cisco got the highest ease-of-use ratings, and users were particularly enthusiastic about the design and tactile characteristics of the physical IP phone sets.

DHS purchased 6,539 of the IP phones, along with four Unified Communications Manager servers, one Contact Center server, five Unity voice-mail servers, and 78 Integrated Service routers.

Timetable Stretches to Nine Months

In an effort to contain project cost overruns, DHS initially planned an aggressive five-month rollout. The schedule called for one site migration per day once the installation of core services was finished.

“Migrating 7,000-plus desktops in five months was way too aggressive,” recalls Hale. “You only get one chance to make a good first impression, and we didn’t want to have people thinking this technology is difficult or arduous to use.”

Adjusting all the local processes to fit one standard was a bigger challenge than the IT team had anticipated. One obstacle that needed to be overcome was incorporating DHS’s unique voice standards into the Cisco platform.

“We couldn’t use a cookie-cutter approach because each office had its own nuances, and all these customizations created a lot of extra work,” Goldman says. The rollout was moved to a nine-month timeframe and morphed quickly from a standard data technology implementation project to a uniquely designed voice management project. Some business change was necessary at most of the sites, and the IT team had to do a lot more hand-holding than they originally anticipated.

For the first phase, project planning started in March 2006 with the central CallManagers installed between May and June 2006. The system rollout began on July 1, 2006, and was completed in March 2007. Migration of DHS’s remaining sites awaits additional budget allocations in 2008 and 2009.

Rollout Results

One unpleasant surprise of the DHS rollout was the need to replace some 20-amp electrical outlets with 30-amp single-phase twist lock outlets. What’s more, two sites had to be rewired because they still had Category 3 wiring.

“We weren’t expecting that, and we had to postpone the installation at that site and rewire,” says Jack Harris, director of telecommunication for Michigan’s IT department. “Doing a physical inventory can get you 90 percent of the way there, but you are still going to miss some things. You need a flexible approach to your project that lets you move things around as you go.”

At every stage of the project, IT worked to build consensus by involving the users. “Executive support is all well and good, but you need the support of people who are responsible for things at the local offices,” Goldman says. “Otherwise you can get stuck on menu choices and other basic decisions forever.”

The Michigan team also took user training seriously and convened formal sessions conducted by professional trainers. The technical people stayed onsite for the first few days to field questions from users and resolve problems. Hale estimates that this strategy reduced the number of help desk calls resulting from the rollout by 60 percent to 70 percent. The team further minimized the potential help desk burden by using a dedicated VoIP help desk with its own phone number.

“Customer service is very important because you don’t want your users to have an initial VoIP experience that is negative,” Hale says. “Your people need to drop everything and fix user problems as soon as possible.”

Similarly, all the new reporting capabilities are a big plus. IT staff can monitor usage statistics on its 24-hour hotline to count the number of incoming calls, the number of abandoned calls, the average queue time, agent status, and other metrics. DHS is using such information from automatic call distributors at three locations to develop formulas for staffing requirements and target the best times to have staff available for handling calls.

Return-on-investment numbers for the initial phase of the VoIP migration are not going to be available for at least a year. However, the team is confident that the Cisco VoIP solution is at worst no more expensive than installing updated Time Division Multiplexing equipment would have been. It is probably generating some savings. More important, it provides a foundation for next-generation converged applications that Time Division Multiplexing platforms do not, no matter how new they are.

“On paper, the installation of [primary rate interface] trunking at most of the DHS sites showed significant savings to the state over the costly Centrex systems or the cost for incoming/outgoing trunks,” says Goldman. “We also reduced our telecommunications repairs budget by more than 50 percent.”