How Computer Vision Algorithms Work
The most sophisticated computer vision algorithms are based on a kind of artificial intelligence known as a convolutional neural network. A CNN is a type of artificial neural network and is most often used to analyze visual imagery.
What computer vision algorithms bring to the table is the scalability and aptitude to memorize outcomes. Instead of capturing and storing large amounts of video data, computer systems can, for example, store the fact that a person was present at a certain location at a certain time and wandered from point A to point B.
Computer vision can automate the process, extract that metadata about a video image and then store the metadata without the image.
MORE FROM STATETECH: How is artificial intelligence being used in state government?
Computer Vision Applications: Object Recognition
Some applications are better suited to a machine’s eyes. “AI vision is ideal for applications where a reliable steady visual coverage of the scene of interest is possible and there is a reasonable understanding of the range of object types or activities that will be present during use,” says Mike Piacentino, senior technical director of vision systems for SRI International, a nonprofit scientific research institute.
That’s why traffic management is a natural growth sector for the deployment of smart vision systems.
GRIDSMART uses a single fisheye camera that offers a full intersection view, including the dangerous center where cars, trucks, motorcycles, scooters, bicycles and pedestrians cross paths. “We blend traditional computer vision and deep neural networks,” says Gabriel McFadden, senior regional sales manager for GRIDSMART. “GRIDSMART detects and tracks all the moving objects in the scene and uses AI to determine exactly what they are.”
The platform provides real-time performance data such as time-stamped traffic volumes, turns, and average speeds for analysis and intersection light timing and delivers the information back to traffic managers. GRIDSMART cameras are currently in 10,000 locations across the United States, including in Tacoma, Wash., Seattle, San Francisco and Reno, Nev.
GRIDSMART’s cameras use AI to track objects at intersections. Source: YouTube
John Rader, a senior traffic signal electrician in Tacoma, says it usually takes four cameras to watch an intersection with a light signal. A single GRIDSMART camera supplies a 360-degree view, and some can tilt and zoom.
“It’s been working really well for us and it’s easy to put up,” Rader says. “It has a lot of features there that we like.” That includes some upgrades that can now detect bicycle riders and pedestrians and give them more time to cross. Rader says the city is also using the system to count cars. At one intersection, planners were surprised to find 45,000 automobiles moving through in a single day. “The reporting is strong,” he says.
Computer Vision Applications: Image Processing
Hayden AI offers a smart camera for use on transit buses, police cars and other municipal vehicles. The onboard device can detect if a car is parked on a bus lane and can also tag and report automobiles parked at bus stops.
The system also remembers historical data information to produce 3D maps. For example, it can spot when a car is parked in front of an unseen fire hydrant because the camera has noted the hydrant in earlier passes. The camera is a 21-teraflop device and catches data in 10-second evidence packages, which are transmitted to the cloud.
“We’re seeing the digital transformation of policing,” says Hayden AI CEO Chris Carson. “Europe has done a better job with it. We can’t hire enough police.” Many officers are worried about COVID-19 or the danger of pulling over a vehicle.
“Our technology has no bias; it just recognizes a car,” Carson says. Hayden AI has municipal customers in New York, California and Washington state. Its cameras will also soon be found on city vehicles in Washington, D.C., Carson says.
RELATED: How can cognitive engine technology aid public safety?
Computer Vision Applications: Motion Tracking
Smart vision systems are also finding their way into other specialty applications that can be used by state and local governments. Smart Vision Works is a company founded in 2012 with customers primarily in the food and agriculture sectors.
Smart Vision Works collaborated with the state of Michigan on a pilot to identify and remove invasive carp from its waterways. The native fish at risk is called the June sucker. The company trained its AI system with deep learning to identify the fish out of six or seven other species.
“You don’t know exactly what characteristics it’s cluing in on,” says Beau Tippetts, vice president of systems engineering at Smart Vision Works. “You need a lot of pictures of a lot of carp.”
Ultimately, Smart Vision Works created a trap-net design with a fence and camera. As fish swam by, they were identified and sorted through exit doors.
MORE FROM STATETECH: How can machine learning secure IoT data?
Computer Vision Applications: Drone Mapping and Monitoring
Levatas provides smart cameras for drones and fixed locations for oil and gas, telecommunications, and power-generation companies. It currently works with the largest public utility in Florida.
“We do a lot of helping organizations automate manual inspection tasks,” says Daniel Bruce, partner and chief product officer at Levatas. These include the rote, time-consuming safety checks that take up a lot of plant workers’ time. Drones are programmed to detect leaks and other anomalies in hard-to-reach places. Meanwhile, gauges can be watched for trouble by a stationary smart camera.
“These are mundane and frustrating jobs” says Bruce. “We are not here to replace humans; we’re here to augment them and give them superpowers.”
The Federal Aviation Administration forbids drones from flying beyond the operator’s line of sight. Fortem Technologies has created its SkyDome platform to provide autonomous 3D airspace monitoring. It uses AI-assisted radar to watch for airborne obstacles when a municipality wants to send a drone over the horizon to investigate an accident or crime scene. The system then signals an open flight path to the operator and the FAA provides a waiver.
“You need a service to tell you the route is clear,” says Fortem CEO Timothy Bean. “We provide that kind airspace data so people can do myriad autonomous drone operations.” The company is currently working with 14 state departments of transportation.
Edathil at the Texas Department of Information Resources spends his days working on new applications for machine learning at state agencies. A recent proof-of-concept application used AI biometric facial recognition to sign in visitors at the front desk.
“The main advantage of facial recognition is that it minimizes user contact,” Edathil says. “The system was set up to delete the image after sign-out,” so it doesn’t store personal identifiers without permission.
Edathil envisions many other uses for smart AI in public agencies around the state. These could include public-facing tasks at the Texas Parks and Wildlife Department, building inspections by the Texas Facilities Commission, or a toll system that detects expired tags and pulls information from registration stickers.
“It could also be used to help law enforcement,” he says. “The system can be trained to detect if the person in the driver’s seat is an authorized driver and potentially flag the vehicle for theft.”
