Security

DHS Researches Overcoming Occlusions in Video Surveillance for Public Safety

Computer vision and machine learning may help first responders overcome obstructions to what they can see.

Brian T. Horowitz is a writer and editor covering enterprise IT, innovation and the intersection of technology and government.

Video surveillance is a critical tool that can help law enforcement protect the public. However, as first responders rely on video to scan an emergency, occlusions, or obstructions, may block their view. Occlusions can include permanent fixtures, such as trees and buildings, as well as temporary objects, such as cars and dumpsters, says Honoré Nyuyse, physical and cybersecurity technology manager for the U.S. Department of Homeland Security’s Science and Technology Directorate (DHS S&T).

In a recent solicitation, DHS S&T challenged companies to create ways to mitigate occlusions and other threats toward soft targets and reduce attacks in crowded places. New technologies — including applications using computer vision, machine learning (ML) and drones, developed in response to the directorate’s solicitations — should be able to integrate with a municipality’s existing video management systems, says Melissa Oh, managing director at DHS S&T’s Silicon Valley Innovation Program.

DHS S&T is providing up to $2 million in funding per startup to address the Securing Soft Targets challenge, Oh says.

Click the banner below to explore integrating video feeds into a real-time crime center.

Computer Vision and Image Processing May Help Augment Video

Video analytics platforms, computer vision and image processing can potentially overcome occlusions. Many organizations are performing research to identify objects in a video frame, including bicycles or other vehicles, Nyuyse says.

“When you’re running a video management system, that algorithm can automatically identify an object and classify the object based on what it has been trained to identify,” Nyuyse says.

For ML to help first responders overcome occlusions, they will need to use data sets based on supervised learning, Nyuyse says. “With that type of data set, you can come up with applications or models that will help train the system to start conducting some automated functions.”

Nyuyse describes how NVIDIA is working with Audi, which offers a night vision tool that detects occlusions such as people or animals within 20 yards of a vehicle. NVIDIA is providing 3D segmentation and modeling.

“I think technologies like that have a huge impact in trying to solve some of the challenges we have, especially with occlusion,” Nyuyse says.

CONSIDER: 3 factors to assess when deploying video surveillance through edge computing.

Officials May Bypass Obstacles with Camera Handoff and Drones

First responders can use automated video processing to alert them to suspicious behavior. If a person walks behind shrubbery and takes longer than 5 seconds to emerge, they may have passed out, fallen down or been attacked, says Jim Davis, professor of artificial intelligence and machine learning and director of the computer vision lab at Ohio State University.

Smart cameras can pick up these cues, which are changes from normal behavior, such as a delay in the typical time it takes to emerge from behind an occlusion. A location-aware camera enhanced with AI and ML technology can provide high-level activity alerts to first responders, Davis says.

Another strategy is to use a camera handoff feature to follow a person or group of people walking through a public facility, such as a sporting arena, airport or train station, as they pass in and out of view from individual cameras. Operators can coordinate pan-tilt-zoom cameras to continually track people moving through an area.

“The fact that you can stitch observability of an individual, a crowd or a group of people across multiple cameras is one way to help combat the occlusion problem,” Davis says.

For the future, Davis suggests that mini drones could help first responders track emergencies in large gatherings such as sporting events. Just as celebrities like Lady Gaga have used drones for entertainment at stadiums, first responders could use drones in these outdoor venues to overcome occlusions.

In addition, 3D scene modeling and behavioral analysis will help first responders overcome occlusions in the same way that smart cars use 3D models, Davis predicts.

“The deeper it gets into 3D geometry, along with 3D behavioral analysis, the better it’s going to enable the AI to understand the environment, particularly in terms of these occlusions,” Davis says.