Washington DC Is Grading Customer Service One Data Point at a Time

The District of Columbia is on a mission to make customer service better, but growing at an annual clip of 5.1 percent makes scaling any initiative a challenge.

In July 2012, the city launched Grade.DC.gov, a project that aims to grade local agencies by pulling data from social media sites, such as Twitter, Facebook, Foursquare and Yelp, as well as customer surveys and text messages. The pilot started with five agencies and has grown to 15 over the first year.

It turns out that the growing population is providing just the data the city needs. As District of Columbia communications officer Matt Desjardins says, technology, and social media in particular, is key to gathering feedback and making changes to improve the city’s services.

"There has always been customer feedback loops in the district, but this is a new one with a social media focus. A lot of residents use social media to contact agencies, and it was getting to the point where we needed to do a better job of listening and collecting that information governmentwide."

Harvesting Data for Better Service

Shortly after taking office, Washington, D.C., mayor Vincent C. Gray approached local social-intelligence company newBrandAnalytics (nBA) about the project. nBA works with hotels and restaurants to mine social data in order to discover trends and improve service. Industries such as retail and hospitality are extremely customer-centric, but as Mayor Gray underscored, so is government.

When a resident tweets a local agency, writes a review on Foursquare or posts on an agency’s Facebook page, the information is pulled into the nBA platform for processing. Natural-language-processing software and a few human “taggers” parse the data by category and sentiment. Each day, the participating agency’s director receives a new report, and key agency members always have access to an updated dashboard, where data can be bent to identify opportunities for improvement.

"We're trending the data. We're seeing where a lot of the negative chatter is coming from in regards to customer service. We're making tweaks and adjustments that are helping these agencies better respond to their customers, the residents," Desjardins says.

A Proactive and Reactive Approach to Customer Service

The first year has been extremely successful. Grade.DC.gov has already processed more than 12,000 reviews containing more than 62,000 data points, known as insights. (A review could be a tweet, a post, a text message or a survey. Most tweets contain only a single insight, while surveys contain as many as 25.) The information allows the city to be both proactive and reactive with data that is both big and small. An accumulation of feedback could prompt the city to make an important change going forward, but a single tweet or text message could spark immediate action.

At the DC Office on Aging, senior citizens have been quick to take advantage of the opportunity to give feedback. Early in the program, residents made it clear that a meal served at one of the city’s wellness centers hadn’t been up to par. Based on the feedback, the problem was quickly fixed. By making the adjustment, the city proved it’s willing to listen, which has motivated residents to offer even more comments.

But the responses aren’t all negative, and that information is valuable to the city and its employees, Desjardins says: "You would think these agencies only get negative feedback, but a lot are getting positive comments, especially on Twitter and within our surveys. We're able to capture that and, in some cases, motivate personnel with positive comments. We couldn't do that before because we didn't have the resources."

Open Data, for Better or Worse

Because the feedback is data driven, the responses and grading should be as well. Anecdotal improvements, such as better food in the Aging Office, are key wins, but the program is designed to lift the entire city over the long term.

"The initial grades were C-minuses, but they've grown over the past year since we've been listening and responding. Most agencies are getting A’s,” Desjardins says. The data, however, isn’t robust enough to paint an accurate picture just yet. "We only have one year of data right now. We’re still identifying trends.”

Washington Post reporter Mike DeBonis questioned the value of the system, given the relatively small amount of data available:

While 15 city agencies are being graded, those agencies are not being reviewed evenly. In February, for instance, the system processed about 6,800 insights, but two-thirds dealt with one of four agencies: The Department of Transportation, the Department of Motor Vehicles, D.C. Public Library and the Office of Aging.

Former Indianapolis mayor and current Harvard professor Stephen Goldsmith believes that the initiative is about more than just data, according his article on Governing:

Years ago, when I was mayor of Indianapolis, we found that surveying customer satisfaction from outbound 311 calls provided meaningful comparative benchmarks by service and area. But that was rather crude compared to what Washington is able to do today with Grade.DC.Gov: taking the measurement of how well public services are delivered to a level that is both more pervasive and more collaborative.

The first of its kind, Washington, D.C., social data mining project is leading the way into uncharted territory. newBrandAnalytics says they are in discussing similar projects in other cities. Mayor Gray is pleased with the initiative so far, according to Desjardins: "The mayor's been really impressed with it. He's able to have a window into what's being said on social media about these agencies. He's used some of this data to call out agency directors and ask questions. It's been a good tool."