New York State of Mind

LAST AUGUST, while routinely monitoring the network traffic at its sites, the IT team at New York State's Division of Housing and Community Renewal was thrown for a loop. The Queens network was so flooded with Web traffic that the administrator in Albany couldn't even access its server to monitor the situation. The culprit: the Rbot-HB worm.

"It took a while to realize what was going on," says Housing and Community Renewal IT Director Bob Kelly. “By that time, a lot of machines were infected and a lot of phones were ringing.

"The infected PCs didn't stop working, but they ran very slowly because the worm was broadcasting to the Internet and creating lots of network traffic. The worm also disabled our virus signature server's updating engine. Re-enabling current signature updates allowed the workstation's virus software to recognize and quarantine infected files."

By the following day, Kelly and his team contained the virus by recloning the infected machines. In fact, all of the computers at that location were eventually recloned as a precaution since the Rbot-HB worm had made changes to the registry. The cleanup spread Kelly's staff thin, and employees were on edge until the malicious virus was removed.

"We can't go through this again," Kelly says, "so we better figure out what went wrong and make sure that it doesn't happen again."

Playing Defense

That's easier said than done. A majority of Kelly's resources are devoted to securing the agency's infrastructure, and ever-growing threats demand even more resources, which aren't available. It's a dilemma faced by public agencies around the country. However, for New York agencies such as Housing and Community Renewal, which has an office just blocks away from Ground Zero, the implicit threat of the conundrum carries a historically explicit message.

"It seems as if this is a never-ending thing, trying to deal with viruses, worms and the like," Kelly says. "It's like trying to fight a war when your enemy keeps changing. And we're always playing defense."

Playing defense leaves little time for proactive efforts. It's useful to be able to analyze logs for unusual patterns, but agencies first need to do the basics: Update virus definitions, download software patches, monitor firewalls and layer security.

"In an environment where it’s tough to be perfect, you've got to make sure you're good enough at addressing the threats coming through," Kelly says. To do that, agencies must first identify priorities, then pursue them vigilantly, he adds.

"Because of 9/11, New York state is further along that path," he says. "The state has taken a strong leadership role in this area."

After the Sept. 11 tragedy, the New York State Office of Cyber Security and Critical Infrastructure Coordination was established. As one of its key initiatives, the office developed a series of cybersecurity standards and led agencies in a gap analysis to see how close they were to meeting those standards.

"There were gaps identified that required attention," reports Greg Benson, the executive director of the New York State Forum, an agency that supports state and local government public IT departments.

Part of the Cyber Security office’s efforts to help agencies meet their compliance goals includes a series of training programs offered to state and local government employees across New York. In addition, the Office of Cyber Security, in conjunction with the New York State Forum, has started a Webcast training program that is available to government employees and to private sector and home users. The Webcasts focus on specific cybersecurity issues and offer attendees practical advice.

The Webcasts began as a New York state pilot in April 2004. Then Cyber Security Director Will Pelgrin, who also chairs the Multi-State Information Sharing and Analysis Center, took the idea to the Department of Homeland Security, which opened the Webcasts to a national audience. By October, the Webcasts had gone out to an international audience of more than 1,400 participants representing nine countries.

“There are a number of challenges that we all face,” Pelgrin says. “New York is not alone in that. The only way we’re going to be successful is if we all work together.”

Enemy at the Gate

Once upon a time, months could pass before new worms surfaced, hackers took advantage of vulnerabilities, and IT departments implemented patches to close security gaps. Now, the plethora of new worms and their variants are appearing at a daunting frequency, says Pelgrin. Hackers are taking advantage of any crack in cybersecurity opened by malicious code, often within hours, a practice called zero-day exploit.

“We’re all concerned about that zero-day exploit that we assume is just around the corner,” he says.

Responding to these cyberthreats can easily sap agencies’ available resources. However, the Office of Cyber Security helps to identify risks based on the vulnerabilities, which allows agencies to better prioritize their staff resources.

Its Web site ( www.cscic.state.ny.us) assigns risks based on five categories of users. For example, it will identify when a large agency might not face a risk from a particular virus, while a small agency might.

Another threat, Pelgrin says, is the growth of bot networks. A bot, short for robot, is an automated software program that can execute certain commands given by a central controller. A bot can sit in the background waiting for instructions and potentially can wreak havoc on infected machines. They can forward spam mail, steal personal data or perform denial-of-service attacks.

“They’re growing in size,” Pelgrin warns. The Office of Cyber Security found one bot network with 7,000 machines connected to it, and tales of networks with up to 30,000 connections abound. He describes bots as “soldiers in the wings waiting to attack.”

Instructions for cleaning up massive bot networks are available on the Internet, but they are extremely complex. Therefore, Pelgrin’s office recently issued a bot network case study to all agencies to explain the impacts and consequences at a level that everyone can understand.

Social engineering is another ever-present problem, Pelgrin adds. Phishing scams—e-mails disguised as notices from legitimate companies that direct victims to fraudulent sites in order to collect their account or credit card information—are becoming more and more sophisticated. He regards authentication as a potential research approach for addressing the problem of spoofed e-mail addresses and Web sites.

One of the agency’s Webcasts tackled the problem of phishing scams. It explained how such cons work and what they look like. The Webcast instructed users not to respond to or click links from e-mails soliciting personal information, and instead go directly to a company’s Web site or call to make sure the request is legitimate.

“If something doesn’t look right, question it,” Pelgrin advises.

An Office of Cyber Security mantra says that security isn’t about placing blame, and employees won’t get into trouble for reporting problems on their computers. Once people embrace that idea, the state will be better able to reach its goal of greater information sharing, Pelgrin points out. With Internet traffic steadily increasing, and the timeframe for a computer to be compromised rapidly shrinking, it’s critical that people report problems immediately, he stresses.

A challenge is getting agencies to address security at the executive level from the outset. If security is not implemented at the executive level during overall planning and mission development conversations, it will not work.

“The landscape’s never the same,” Pelgrin says. “That requires us to be vigilant every single day. My concern is that there’s complacency. My office strives to proactively address the current challenges and anticipate the future challenges to enhance our cybersecurity posture. By working collaboratively with all of our partners, we will be successful.”

Ensuring Continuity

On Sept. 11, 2001, Housing and Community Renewal‘s Manhattan office, which is just blocks from the World Trade Center, was evacuated and closed until smoke and debris could be cleared. Fortunately, the agency’s servers never went down, but housing attorneys who were due in court during the next several days couldn’t get to their paper case files.

The experience illustrated the need for electronic files and remote access. As a result, the agency has begun a number of initiatives to ensure business continuity, including a remote log-in system so employees can get onto the network even if they can’t get into the office.

Another initiative is to use storage area networks to back up transactions between sites. A third approach is using imaging and document management of active files so the information is available online.

“These initiatives will take time to complete, but this is the path we are going down,” Kelly says.

For other agencies, a challenge has been striking a balance between the demand for greater transparency in government through the Internet and the need to protect sensitive information that could render the agency or its constituencies vulnerable, says JoAnn Bomeisl, technical infrastructure manager for the New York State Insurance Department. “That’s a big struggle,” says Bomeisl, who is also co-chair of the NYS Forum’s Business Continuity Planning and Security subcommittee.

Limited resources present another challenge, she says. A gap analysis may highlight areas in which an agency is deficient, but if it has a small IT shop (less than 10 full-time staff members), officials must determine which issues they should tackle first. After agency officials make that call, they must determine where to get the expertise necessary to address those issues, “because it’s not necessarily going to be in their shop,” Bomeisl says. Even then, she adds, they must find a way to pay for the work.

“We have a lot of gaps to fill and very few resources to do it,” says Benson of the NYS Forum.

Sharing such concerns can be a big help, points out Kelly of Housing and Community Renewal. Under the Office of Cyber Security, large and small agencies—each with an independent mission—have come together for face-to-face meetings and through a listserv to share concerns and offer strategies and advice as they work toward closing the cybersecurity gaps within their agencies.

“We knew what we needed to do from an agency perspective, but it was interesting to see, based on cybersecurity, what we needed to do as a state,” Kelly says. “No one will ever be completely secure, but we’re headed in the right direction.”