Agentic AI at Scale: Use Cases, Costs and Ramifications

AI Specialists Comprise ‘Teams’ for Complex Functions

An AI agent might work within a “team” of other agents. For example, a DMV agent, a financial-advisor agent and a healthcare assistant agent could “talk” to each other and solve complex issues for users, whether they’re citizens or government employees. Put simply, an AI agent from one discipline can work with an agent from another to vastly expedite entire workflows. 

Each agent is a specialist trained with focused knowledge. By narrowing a single AI agent’s scope and connecting it to the right APIs and databases, we can reduce noise and boost precision, while confining it to certain areas of expertise.

We can then compound their capabilities by allowing them to interface with other agents. 

Governments Must Carefully Justify Use Cases

At the most basic level, agentic AI can automate certain functions. It can reduce workloads at call centers and provide 24/7 service, for instance.

But agentic AI has the capacity to solve some of government’s most complex challenges. For example, state and local jurisdictions are chronically constrained by funding. Some states are already investigating how AI agents might comb through budgets to identify cost-saving measures and even simulate specific budget scenarios.

As noted in a recent StateTech column, there’s profound value in using AI to shape “balanced budgets that align with strategic priorities, enhancing fiscal responsibility and resource allocation.” The ability to do this without raising state or local taxes is highly appealing to jurisdictions.

What’s more, AI agents aren’t free, and not even necessarily cheap. They’re incredibly cost-effective at best, and money pits at worst. Use cases should be forward-thinking and do justice to agentic AI’s potential and its costs. The last thing you want is to ask 20th-century questions of a 22nd-century technology that came too soon.

AI models, GPUs, cloud storage and in some cases, even virtual avatars, all require resources. Different platforms (AWS, Azure, Google Cloud Platform) have their own pricing models, and costs can easily spiral out of control. Over time, especially when agents interface with one another 24/7, compute and storage costs will compound, and micro-transactions will add up. 

Cost Will Need to be a Long-Term Consideration

Within the next three to five years — maybe sooner — IT departments will come face-to-face with the reality of how much agentic AI at scale will cost them.

There are many ways to do the math when forecasting costs for agentic AI: What’s the price per conversation between an agent and a human, or between an agent and another agent? How many agents are running? What’s the energy cost? For that matter, what are the returns?

The key is being able to track what agents are doing, what resources they're using and how much each prompt costs. At the risk of getting meta here, you could even task a “financial agent” on every team to estimate the compute cost of each task and set budgets accordingly.

It’s also worth implementing financial circuit breakers that pause tasks when they exceed a certain budget, prompting a human to step in for approval. 

The public cares about how AI dollars are spent, and they deserve that transparency. 

RELATED: Take these steps to ready your IT infrastructure for AI.

Agentic AI Is a Team Sport and a Long Game

Agentic AI will touch education, healthcare, crisis response, public safety and more. As bastions of social infrastructure, state and local jurisdictions must be prepared for that reality. That means involving financial and legal stakeholders now, not after the fact. 

Mayors, governors and local leaders must work with their staff and private-sector partners to understand what’s possible, what it will cost and ultimately, what projects are worth pursuing. 

Don’t rush into the wrong use cases, but don’t backpedal either. Agentic AI builds on generative AI and it is the next wave of AI technology. Ask questions now to avoid mistakes later.

This article is part of StateTech’s CITizen blog series.