Know What AI Can Reach Before It Reaches Production
The first step is an inventory of AI use across the organization and the data each service can reach. An AI assistant may have access to email, documents, case records, benefits information or other agency data, including regulated classes such as criminal justice information or federal tax information. Agencies should determine which data classes each AI service is permitted to access before connecting those services at scale.
Permissions deserve particular attention because AI can make existing access problems easier to exploit. A tool that searches agency information may surface data a user technically has permission to access but should not see. In that case, the permission model may be functioning as designed, but the permission state is still too broad. Overshared Microsoft SharePoint sites, stale user groups and poorly managed permissions become more consequential when an AI assistant can locate and summarize information in seconds.
Before connecting sensitive information to AI, agencies should establish clear data ownership, classification and access controls.
READ MORE: Agentic AI can manage government workflows.
Treat Identity and Networking as AI Control Points
Identity determines who, or what, can use an AI service. Single sign-on, multifactor authentication and least privilege should apply to AI just as they apply to conventional applications. Agents add another layer of complexity. When an agent acts through APIs or connectors, teams need to know which identity is acting and what that identity is allowed to do.
Networking requirements depend on the AI delivery model, and that distinction often gets lost. Platform deployments may support private network paths with defined Domain Name System, routing and inspection. SaaS assistants often rely more heavily on conditional access, tenant and agent settings, and data loss prevention policy. Agencies should decide on these patterns at the infrastructure level so individual projects do not invent their own controls.
DIVE DEEPER: AI can transform government operations.
Watch the Connectors, Endpoints and Logs
Connectors should extend existing controls, not sidestep them. A connector should not bypass permissions or logging already applied to the source system. APIs need authentication, limits on use and records of activity. Endpoints matter as well: Agencies should know whether devices accessing AI meet security requirements and whether AI-generated output can move to unmanaged tools.
Retrieval scope is a security boundary, not a productivity setting. Any assistant that retrieves untrusted content — whether from an email, shared document or web page — can be influenced by instructions hidden inside that content. That is why agencies should separate trusted instructions from untrusted data and limit what connected agents can access or do.
Logging also requires more than a single switch. Activity records, details about accessed resources, interaction content and retention periods are often handled through separate audit, compliance and e-discovery capabilities. Access to prompt and response text may depend on licensing, configuration and role assignments. Agencies should also assume that prompts and outputs may become public records subject to state retention schedules and disclosure requests.
Don’t Make Every AI Project a Compute Project
Compute should follow the use case. Agencies running local inference or other high-density workloads may need specialized infrastructure. But most agencies can begin by strengthening the environment they already have: documenting AI use, reducing excessive permissions, approving network paths, governing connectors and ensuring the right logs are available.
Buying GPUs will not fix excessive permissions, unmanaged connectors or missing logs. For many state and local agencies, those are the infrastructure issues that will determine whether AI can move from a promising pilot to a service that operates safely at scale.
