For years, the default question a state CIO asked vendors was simple: “Is artificial intelligence in your product, and can we turn it off?” But according to a panel of state and local security leaders at GovForward’s FedRAMP Summit, that question may now be moot.
As state and local governments continue to expand their technology practices, they are confronting governance challenges due to the ubiquity of AI in vendor offerings. These challenges are exacerbated by the reality that many state and local governments rely on legacy architectures that were not designed to work with autonomous technology. That reality also opens up government systems to machine-speed cyberattacks.
“There is no ‘un-AI-able’ vendor service being provided in the cloud at this time period, and that should be your assumption,” said Elizabeth Herman, executive director of federal government relations at JPMorgan Chase.
That blunt assessment reframes the procurement conversation: AI is no longer a feature that buyers can approve or deny, because it’s thoroughly embedded in the services governments already acquire.
Click the banner below for insights into an AI strategy for your enterprise.
There Is No ‘AI Off Switch’ Anymore
The instinct to demand an “AI off switch” is understandable, and panelists agreed it’s still worth asking for. Maine CISO Charles Rote said the request to vendors is straightforward: Give teams a setting to turn AI off in case a security issue arises. But that’s not how the market behaves; vendors want “that shiny AI on the cover” to sell the product, Rote said
David Resler, GovRAMP’s COO and CTO, added the nuance that procurement teams miss: Turning AI off isn’t binary. Some products offer only on or off settings; others let administrators enable AI for one agency and disable it for another, or tune it by data boundary. The challenge is matching those controls to any given team’s risk tolerance, rather than assuming that a single toggle exists.
Georgia CIO Shawnzia Thomas described the practical move her team made: contract language requiring vendors to disclose AI usage.
“Most days, when we are putting language in our contracts, vendors have got to let us know if and how they are using AI,” Thomas said.
Thomas emphasized that state tech teams should stop treating AI as a yes/no gate at procurement, and start treating it as an ongoing discussion with vendors. Georgia also frames AI as a shared responsibility. Thomas had to correct her own team’s assumption that anything AI-related belonged solely to her AI officer, when it actually requires legal, cyber, procurement and business teams to work together.
Why Governments Should Adopt Continuous AI Disclosure
Point-in-time approval fails because AI features ship faster than buyers can react. Rote described integrators pulling AI feature sets into systems “even faster than ever before,” making current risk hard to track. And on the vendor side, “9 times out of 10, you don’t know what they’re using or how they’re using it” without a program that requires disclosure.
GovRAMP’s answer, per executive director Leah McGrath, is to treat added generative AI as a significant change event: When GenAI enters a product’s data boundary, participating agencies get a notification — a prompt to have the conversation before discovering the change later. Any state can build the same trigger.
All panelists agreed on one thing: Retire the “does it have AI?” question. They urged the audience to assume every cloud service provider is using AI, whether they advertise it or not. Write disclosure into contracts. Demand graduated, per-agency controls and make AI risk a shared responsibility across all teams — not just the CIO’s office.
UP NEXT: Here is a guide to AI governance for state and local agencies.
