The Data Analytics and Sharing Tools Smart Cities Need

Why It Can Be Difficult to Share Data Across City Government  

Some government agencies resist sharing data with other departments because they’re focused on their own interests and needs, according to Bob O’Donnell, president of technology consulting firm Technalysis Research.

“In a lot of these organizations, there are little fiefdoms, in a sense,” O’Donnell says. “The transportation department may have data about the stoplights that would have value to the urban planning department, which is trying to figure out where different types of buildings could go. And they don’t necessarily share the data to determine traffic patterns and how they would be impacted by a new, huge office building.”

Departments also might be using separate data collection and management solutions.

“Different groups will pick different tools,” O’Donnell says. “One group’s set of data is being used in one particular tool, and it may not be that easy to share into another tool. Some of it’s just plain old software incompatibility or data set incompatibility, because there aren’t well-established standards, necessarily.” 

Pulling information from multiple sources into a data lake is one potential solution to allow cities to perform analytics work and share data between silos.

Data governance tools from Quest, RSA and other companies can help organizations securely and seamlessly share data, and a number of cloud-based providers, according to O’Donnell, offer a way to centralize and analyze data.

“We’ve seen over the past several years the ability to leverage data across multiple clouds,” he says. “Because the department of transportation may have something it’s done up with Google Cloud, whereas another division is doing something with AWS; in the past, that made it impossible. Now, there are tools some of the cloud providers have enabled to allow you to work on or transmit data, or that have an application that works on data that resides in multiple clouds.”

RELATED: Cities can now measure how “smart” they actually are.

Tech Solutions Help Cities Connect Data

In Carlsbad, Calif., data lake creation is one of the city’s planned data management efforts. The City Council has earmarked a significant amount of funding for developing systems that will make it easier for departments to use, analyze, visualize and share data, according to Chief Innovation Officer David Graham.

Carlsbad also conducted a thorough data inventory of roughly 4.5 terabytes of structured data, in addition to other items, and added a chief data officer-level position in recent years.

“One of the challenges that governments of all sizes are facing is the collection, governance, management and distribution of data,” Graham says. “How we deal with, share and distribute data is fundamental to getting our city operations right.”

Carlsbad uses the Tableau platform as its main data visualization and analytics tool. In preparation for eventually using a data lake, the city is trying to increase its amount of structured data and the number of systems that can access unstructured data, Graham says.

“The city has over 100 software applications,” he says. “We know we can probably consolidate and reduce that number.” So, as the city upgrades or replaces systems, it is looking for, “multiuse software applications that are easy to integrate with other systems, that are best in class for the area they are focused on handling, and that ultimately make it easier throughout the organization to access and use data,” Graham says.

The city’s chief data officer is now required to be involved in the acquisition of any new data storage or collection-related items to avoid having disparate systems that can’t be integrated.

EXPLORE: How does real-time data analysis benefit transit agencies?

“It doesn’t matter how many terabytes you have stored someplace; if it can’t be turned into insights for the organization, you may as well throw it on a hard drive and stick it in your filing cabinet,” Graham says. “It is the utilization and access of the data that ends up leading to having an impact with the data that you’ve collected.”

In Madison, Wis., agency technology choices can vary, according to Ellen Anderson, the city’s data and innovation manager. For example, the Public Works team uses a comprehensive asset management system that has reporting built in, whereas some other agencies rely on Excel spreadsheets.

The city’s data team uses tools such as Microsoft’s Power BI platform. It can combine multiple sources of information into interactive graphics to process and visualize data for efforts, including a program that involves coordinating paramedics, social workers and other entities responding to 911 calls.

“We hook it up to our raw data and use some functions like Power Query to join and clean our data,” Anderson says. “We’re actually using its data management tools as our primary data management, and we also use the visualization to share with our stakeholders what’s happening.”

The solution’s capabilities align with Madison’s ongoing data governance and use initiative, she says, which includes fostering a collaborative, data-informed culture.

“It’s a great way to empower people to work with their own data without needing outside experts … to come in and show them what to do, because it’s very user-friendly,” Anderson says. “But it’s also a pretty powerful way to look at what’s happening with a large amount of data.”

Click the banner below to get a customized smart city content experience by becoming an Insider.

How City Governments Can Make the Most of Their Data

Over the past two years, the pandemic may have put some cities in a better position to share information. Due to office closures and social distancing guidelines, municipalities were faced with a need to move in-person services and other functions online. In the process, they potentially enhanced or created additional data resources.

Agencies were particularly willing to exchange resources to respond to city service and other COVID-19-related needs in Madison, Anderson says. Typically, they did this by sending each other data app extracts or granting each other access to their respective systems.

“During the pandemic, we made some of our biggest leaps in breaking down data silos,” she says. “We always knew we had data that could be useful to other departments and parts of the city, and suddenly there was an urgency. People were really motivated to find ways to share data with each other.”

Madison will not be the only city to embrace that approach. Once more leaders and departments realize how much data is now available, O’Donnell says, a number will be interested in figuring out how to leverage it to make increasingly data-driven decisions. For some, it may require a cultural shift.

“There is much more awareness of the value data has,” O’Donnell says. “But there does have to be coordination. A lot of the changes that have to happen aren’t really technical. They’re more organizational. You’ve got to have the data to understand what’s actually going on before you can make those decisions.”

LEARN MORE: Philadelphia launched a smart streetlight pilot to collect data in real time.