Pulling information from multiple sources into a data lake is one potential solution to allow cities to perform analytics work and share data between silos.
Data governance tools from Quest, RSA and other companies can help organizations securely and seamlessly share data, and a number of cloud-based providers, according to O’Donnell, offer a way to centralize and analyze data.
“We’ve seen over the past several years the ability to leverage data across multiple clouds,” he says. “Because the department of transportation may have something it’s done up with Google Cloud, whereas another division is doing something with AWS; in the past, that made it impossible. Now, there are tools some of the cloud providers have enabled to allow you to work on or transmit data, or that have an application that works on data that resides in multiple clouds.”
RELATED: Cities can now measure how “smart” they actually are.
Tech Solutions Help Cities Connect Data
In Carlsbad, Calif., data lake creation is one of the city’s planned data management efforts. The City Council has earmarked a significant amount of funding for developing systems that will make it easier for departments to use, analyze, visualize and share data, according to Chief Innovation Officer David Graham.
Carlsbad also conducted a thorough data inventory of roughly 4.5 terabytes of structured data, in addition to other items, and added a chief data officer-level position in recent years.
“One of the challenges that governments of all sizes are facing is the collection, governance, management and distribution of data,” Graham says. “How we deal with, share and distribute data is fundamental to getting our city operations right.”
Carlsbad uses the Tableau platform as its main data visualization and analytics tool. In preparation for eventually using a data lake, the city is trying to increase its amount of structured data and the number of systems that can access unstructured data, Graham says.
“The city has over 100 software applications,” he says. “We know we can probably consolidate and reduce that number.” So, as the city upgrades or replaces systems, it is looking for, “multiuse software applications that are easy to integrate with other systems, that are best in class for the area they are focused on handling, and that ultimately make it easier throughout the organization to access and use data,” Graham says.
The city’s chief data officer is now required to be involved in the acquisition of any new data storage or collection-related items to avoid having disparate systems that can’t be integrated.
EXPLORE: How does real-time data analysis benefit transit agencies?
“It doesn’t matter how many terabytes you have stored someplace; if it can’t be turned into insights for the organization, you may as well throw it on a hard drive and stick it in your filing cabinet,” Graham says. “It is the utilization and access of the data that ends up leading to having an impact with the data that you’ve collected.”
In Madison, Wis., agency technology choices can vary, according to Ellen Anderson, the city’s data and innovation manager. For example, the Public Works team uses a comprehensive asset management system that has reporting built in, whereas some other agencies rely on Excel spreadsheets.
The city’s data team uses tools such as Microsoft’s Power BI platform. It can combine multiple sources of information into interactive graphics to process and visualize data for efforts, including a program that involves coordinating paramedics, social workers and other entities responding to 911 calls.
“We hook it up to our raw data and use some functions like Power Query to join and clean our data,” Anderson says. “We’re actually using its data management tools as our primary data management, and we also use the visualization to share with our stakeholders what’s happening.”
The solution’s capabilities align with Madison’s ongoing data governance and use initiative, she says, which includes fostering a collaborative, data-informed culture.
“It’s a great way to empower people to work with their own data without needing outside experts … to come in and show them what to do, because it’s very user-friendly,” Anderson says. “But it’s also a pretty powerful way to look at what’s happening with a large amount of data.”
Click the banner below to get a customized smart city content experience by becoming an Insider.