AI PCs: Everything State and Local Agencies Need To Know

What Makes AI PCs Unique?

“By definition, an AI PC has a CPU and a GPU, but then it also has an NPU, or neural processing unit,” Larson says.

Neural processing units significantly accelerate AI workloads while saving battery. This gives software providers opportunities to run powerful AI functions locally.

“CrowdStrike, for example, was able to bring a security workload to the NPU that in the cloud would have taxed the CPU by about 20%, and we’re able to utilize less than 1% of the CPU,” Larson says. “And then there's things such as Teams background segmentation that we have offloaded to our NPU to also really boost battery life.”

Chipmakers such as Intel and laptop manufacturers such as HP have witnessed a spike in AI features among independent software providers, many drawn in by the chance to save money on cloud spending by offloading AI workloads to local devices.

“About 60% of the folks that write software are updating it to incorporate AI features,” says Alex Thatcher, senior director for cloud clients and AI experiences at HP. “Some software providers we’ve talked to say they save as much as $40 per user per month when they take some of their AI features and move them locally to the PC.”

In addition to being better at sustaining AI functions from software providers, AI PCs are better equipped to support Copilot and other AI features native to Windows 11. This includes the ability to interface in 40 languages and handle real-time translation of those supported languages.

AI PCs can also locally run small language models for custom workflows, reducing reliance on cloud-based tools and creating opportunities for state and local agencies to pare back some of their cloud-based subscriptions.

“That doesn't mean that everything’s coming to the PC,” Thatcher says. “Some things are still best run in the cloud, like the large language models, but there are so many functions that are better done locally, such as voice, translation and background removal.”

Potential Benefits of AI PCs for State and Local Agencies

The core benefit AI PCs offer state and local government agencies is their ability to keep more AI workloads on-premises.

According to Thatcher, they create an opportunity to “be very deterministic about where they’re going to invest in the cloud.” Productivity can be “jump-started with an AI PC and the things that are tangible, almost free, with just a slight investment and a slightly better PC that can get you huge gains in everyday workflows.”

He added: “You pull on the big brain of the cloud only when you need it, and that can give you much more privacy and much more security, and it can have a tremendous impact on the cost.”

Most states employ hybrid cloud models, and AI PCs may act as a gateway to AI for agencies that are wary of cloud LLMs. They can reduce the risks of shadow AI and tighten governance while making it possible to safely leverage local generative AI and even create their own small language models.

“A lot of government organizations are feeling really driven to leverage AI,” Herbert says. “AI PCs are the easiest to begin with, and that’s where they’re starting, because a lot of them are already having to shore up their PC technology anyway.”

DIVE DEEPER: Advanced processors support the rise of ‘invisible’ AI.

To Wait, or Not To Wait?

Whether or not to wait depends on multiple factors, such as how recently PC fleets were refreshed and whether current PCs can support Windows 11. It’s difficult for an agency running two-year-old, Windows 11-compatible devices to justify an across-the-board investment in first-generation AI PCs, for instance.

But many organizations are reaching a decision point, and experts agree that within the next few years, every new model of laptop and PC will have native AI capabilities.

“It's maybe not happening right now, today, but we know it's going to happen in the future,” Herbert says. “How do they get ahead of the technology curve? AI is really moving faster than any other initiative we’ve seen in the past couple of years.”

The biggest unknown concerns just how quickly AI is expected to evolve over the next few years, and some experts say that purchasing non-AI PCs risks becoming a sunk cost.

“Make the transition now, while you know you're going to be investing in a fleet of new PCs, as opposed to waiting and investing in the same technology we’ve been using for the past 15 years,” says Tommy Gardner, CTO at HP Federal. “Otherwise, you’re going to spend the money twice.”

DISCOVER: What does the rising cost of AI means for state and local governments?

Alan Shark, executive director at the Public Technology Institute, agrees that all new PCs will eventually be AI PCs, but says that not every user will necessarily require on-premises AI capabilities in the immediate future.

“We’ve never gotten away from the fact that it’s different configurations for different users. It’s not one-size-fits-all. If you’re a light user, this is what you get, and if you’re a heavy user, you might need an AI PC.”

Technology is moving so quickly that there’s always uncertainty in any IT purchase, Shark says. But he also says that waiting to more broadly roll out AI PCs won’t make or break an agency’s ability to stay competitive with AI. Instead, he recommends being selective about which departments or users to roll AI PCs out to.

“There is no one right answer that will work for everyone, but it helps to ask yourself: Can you start with a pilot and build out from there?”

Click the banner below to get weekly updates from StateTech.