Google Cloud Claims 10x Cost Cut for Idle AI Agents on Kubernetes

Quick Facts

• Google says Agent Substrate cuts the cost of running AI agents on Kubernetes by 10x compared to dedicated virtual machines and containers.
• Google donated Agent Substrate, gVisor, and OpenRL to the CNCF Sandbox at KubeCon North America 2026 in Salt Lake City.
• Agent Substrate can pack more than 1,000 dormant agents per host and restore any of them in under 500 milliseconds.

Google Cloud arrived at KubeCon North America 2026 with a direct answer to one of the most expensive problems in AI infrastructure: agents that do nothing most of the time. AI agents sit idle 90% to 95% of the time, waiting on model responses, test runs, or human approvals, while their allocated compute keeps billing.

Allan Naim, director of product management for AI infrastructure at Google Cloud, put the cost claim plainly: "We've seen 10x reduction in cost in terms of running Agent Substrate on Kubernetes versus running these agents directly on dedicated virtual machines and containers."

The fix is Agent Substrate, an open-source runtime Google built to run AI agents in isolated environments on Kubernetes. When an agent stalls, the runtime snapshots its memory and local files, frees the compute, and restores full state in under 500 milliseconds when the agent is needed again.

The architecture separates stateful Actors from pre-warmed Worker pods. That split lets a single host hold more than 1,000 dormant agents without losing the ability to resume any of them quickly. Jaana Dogan, principal engineer at Google Cloud, described the customer complaint that drove the design: "Hey, majority of the time our agents are idle, but we have to allocate a sandbox all through the lifetime of a session."

Brandon Royal, product manager for AI agents and infrastructure at Google Cloud, said customers have raised the same problem for the past 18 months. Agent Substrate on Google Kubernetes Engine is now generally available as a managed runtime.

Three Projects Head to the CNCF

Google announced the donation of Agent Substrate, gVisor, and OpenRL to the CNCF following their acceptance into the CNCF Sandbox. Drew Bradstock, VP of product management for Kubernetes and orchestration at Google, confirmed the move in an official blog post. The goal mirrors what happened with Kubernetes: establish a vendor-neutral standard that any conformant cluster can adopt.

gVisor is the sandboxing layer that makes Agent Substrate's isolation possible. The open-source application kernel, written in Go and first released by Google in 2018, intercepts system calls in user space and keeps untrusted code away from the host kernel. Google says gVisor already runs millions of sandboxes a day inside GKE Sandbox and Cloud Run. Anthropic and OpenAI both use the technology.

OpenRL, the third donated project, targets reinforcement learning post-training. It supports PyTorch and vLLM, with Gemma and Qwen validated out of the box. Google started OpenRL as a research preview in February 2026 and moved it under CNCF governance early so hardware makers and model communities can shape the project before its design is fixed.

Distributed Inference Gets a Stable Release

Google also highlighted llm-d 1.0, a Kubernetes-native distributed LLM inference framework that reached its first production-stability release on October 20 at the PyTorch Conference North America. The project counts over 500 engineers across 25-plus organizations as contributors, including Google Cloud, Red Hat, IBM Research, CoreWeave, and NVIDIA.

llm-d turns standalone inference engines like vLLM and SGLang into a coordinated cluster-wide serving fabric. Google Cloud said it is integrating llm-d 1.0 into GKE and enabling it across its Google TPU and NVIDIA GPU portfolio. Tesla, Mistral, Prime Intellect, and DigitalOcean are running the framework at scale.

One briefing cited up to 14x more throughput on the same accelerators under a specific workload, though that figure represents a best case rather than a general benchmark. Google Cloud described the goal as letting customers scale multi-tenant environments and lower serving costs on an open-source foundation.

The combined announcements position Google Cloud as the primary architect of Kubernetes-based AI agent infrastructure, using open-source donations to push its preferred patterns toward becoming defaults across the industry.