Security

What The Oldsmar Water Hack Taught Local Utilities

Utilities can take steps to shore up their defenses and protect operational technology from cyberattacks.

Twitter

Phil Goldstein is a former web editor of the CDW family of tech magazines and a veteran technology journalist. He lives in Washington, D.C., with his wife and their animals: a dog named Brenna and two cats, Grady and Princess.

While the fallout from the suspected Russian hack of federal agencies and private companies has consumed the federal government, another recent cyberattack is reverberating at the local level.

In February, malicious actors attempted to tamper with the water supply of a small Florida city by hacking into a water treatment plant in Oldsmar, Fla. The attackers were attempting to increase the level of lye in the city’s drinking water to dangerous levels but were quickly spotted, and the attack was mitigated before any changes were made to the drinking water.

Though local officials hailed the response as proof that redundant controls worked, cybersecurity experts say the attack exposed the vulnerability of local utilities to cyberattacks. The Oldsmar attack, they say, was not that sophisticated and could have easily ended in disaster.

A more savvy attacker could penetrate other utilities, and experts say that the Oldsmar incident should serve as a wake-up call to all levels of government to boost the cybersecurity of critical infrastructure facilities, especially water utilities.

That includes initiatives such as separating operational technology (OT) even more from IT and internet networks, enhancing password security and authentication at such facilities, and updating old software to newer and more secure versions.

“Frankly, they got very lucky,” retired Adm. Mark Montgomery, executive director of the federal Cyberspace Solarium Commission, tells ProPublica. “They shouldn’t celebrate like Tom Brady winning the Super Bowl,” he says. “They didn’t win a game. They averted a disaster through a lot of good fortune.”

Anatomy of the Oldsmar Water Plant Attack

The FBI, the Department of Homeland Security, the U.S. Secret Service and the Pinellas County Sheriff’s Office are investigating the attack in Oldsmar, and it is unclear where the attack originated from and what the motivations of the attacker or attackers were.

According to a Massachusetts state advisory describing FBI findings on the attack, on Feb. 5, unidentified malicious actors “obtained unauthorized access, on two separate occasions, approximately five hours apart, to the supervisory control and data acquisition (SCADA) system” used at the plant.

They accessed the SCADA system “via remote access software, TeamViewer, which was installed on one of several computers the water treatment plant personnel used to conduct system status checks and to respond to alarms or any other issues that arose during the water treatment process.”

According to ProPublica, the city had actually stopped using TeamViewer six months earlier, but never disconnected the program.

LEARN MORE: What are the main security vulnerabilities in a smart city?

Alarmingly, according to the advisory, all computers used by personnel at the Oldsmar plant were connected to the SCADA system and used an outdated, 32-bit version of the Windows 7 operating system. Even more worrisome, the Massachusetts advisory states, “computers shared the same password for remote access and appeared to be connected directly to the Internet without any type of firewall protection installed.”

A plant operator noticed the first intrusion, according to ProPublica, but “didn’t think much of it” Pinellas County Sheriff Bob Gualtieri said at a news conference. It wasn’t until after the second intrusion, when the attacker took over a computer and changed the amount of sodium hydroxide in the water from 100 parts per million to 1,100 parts per million, that the plant worker alerted his boss. The worker lowered the levels of sodium hydroxide and the city called the county sheriff’s office three hours later, ProPublica reports.

“This is dangerous stuff,” Gualtieri said, according to The New York Times. “It’s a bad act. It’s a bad actor. It’s not just a little chlorine, or a little fluoride — you’re basically talking about lye.”

Cybersecurity Lessons for Utilities and Critical Infrastructure Providers

Large utilities often have robust cybersecurity protections, but smaller water utilities, electric power utilities and other small critical infrastructure providers do not. That makes them easy targets, experts say.

“These are the targets we worry about,” Eric Chien, a security researcher at Symantec, tells the Times. “This is a small municipality that is likely small-budgeted and under-resourced, which purposely set up remote access so employees and outside contractors can remote in.”

There are several steps that utilities should take to harden their cybersecurity in the wake of the attack, according to experts and government agencies.

The FBI, DHS’ Cybersecurity and Infrastructure Security Agency, the Multi-State Information Sharing and Analysis Center and federal Environmental Protection Agency said in an alert that organizations should update to the latest versions of software they use, deploy multifactor authentication, use strong passwords to protect Remote Desktop Protocol (RDP) credentials and ensure anti-virus solutions, spam filters and firewalls are up to date, properly configured and secure.

These are the targets we worry about.”

Eric Chien Security Researcher, Symantec

Organizations should also take steps to secure TeamViewer software if they use it, including not using the unattended access features, such as “Start TeamViewer with Windows” and “Grant easy access.” IT leaders should also configure the TeamViewer service to “manual start,” so the application and associated background services are stopped when not in use, according to the FBI, and also set up security protocols to generate random 10-character alphanumeric passwords.

Water and wastewater plant operators, the alert says, should install independent cyber-physical safety systems. “These are systems that physically prevent dangerous conditions from occurring if the control system is compromised by a threat actor,” the alert notes. Separating SCADA and OT systems from IT systems and networks is crucial.

“Restrict all remote connections to SCADA systems, specifically those that allow physical control and manipulation of devices within the SCADA network,” the Massachusetts advisory states. “One-way unidirectional monitoring devices are recommended to monitor SCADA systems remotely.”

Examples of cyber-physical safety system controls include the size of the chemical pump and the chemical reservoir, as well as gearing on valves and pressure switches.

“The benefit of these types of controls in the water sector is that smaller systems, with limited cybersecurity capability, can assess their system from a worst-case scenario,” the FBI alert notes. “The operators can take physical steps to limit the damage. If, for example, cyber actors gain control of a sodium hydroxide pump, they will be unable to raise the pH to dangerous levels.”

What seems clear is that the threat is not going away anytime soon. Testifying before the House Homeland Security Committee in February, former CISA Director Chris Krebs said the Oldsmar’s vulnerability is “probably the rule rather than the exception,” according to ProPublica. “These are municipal facilities that do not have sufficient resources to have robust security programs. That’s just the way it goes.”

Terry J Alcorn/Getty Images