Google Cloud Report: 83% of Firms Must Upgrade for Agentic AI

Enterprise AI has shifted from conversational chatbots to autonomous agents that execute complex tasks — and legacy infrastructure cannot keep up.

For years, enterprise AI meant customer service bots and digital assistants. Today, the market has moved to AI that takes action, automates workflows, and executes complex tasks on its own. This shift unlocks new use cases but places significant stress on the underlying infrastructure. According to Google Cloud's latest research, the gap between AI ambition and infrastructure reality is widening rapidly.

What was announced

Google Cloud published the State of AI Infrastructure report, based on a survey of more than 1,400 senior IT leaders. The headline finding: 83% of organizations require infrastructure upgrades to support production-grade agentic AI. Additional data points include:

• 62% of leaders face a significant inference tax driven by data egress fees, storage bloat, and idle specialized hardware.
• 81% cite operational complexity as a hidden cost of scaling AI.
• 79% rank security, governance, and MLOps as their top challenge to scaling inference.
• 78% now source generative AI solutions directly from their primary cloud partner — a 30-point jump from 2025.
• 52% use a hybrid multicloud architecture, and 48% prioritize infrastructure with strict data residency controls.
• 90% rank edge deployment as important for AI initiatives; 72% call it extremely or very important.
• 91% factor power consumption into hardware selection.

Why this matters

For CIOs, CISOs, IT directors, and procurement leaders, agentic AI changes the economics and risk profile of every deployment. A single agent prompt can trigger hundreds of downstream actions, requiring massive context windows in memory and continuous reasoning loops. Running these workloads on legacy architecture is financially unsustainable and operationally fragile.

The report highlights new operational risks such as agent sprawl — thousands of autonomous agents scattered across platforms, reading emails, querying databases, and executing workflows without unified oversight. Governance, identity, and audit controls must precede innovation. At the same time, digital sovereignty, data residency, and energy constraints (for example, Germany's PUE 1.2 requirement for new data centers) are reshaping where and how AI can be deployed.

You cannot solve the challenges of tomorrow's agentic systems with yesterday's architecture

Technical details

• Fluid compute: Dynamically match silicon to workload — TPU 8t for heavy training, TPU 8i for low-latency inference, and Arm-based Google Axion CPUs for orchestration and reinforcement learning simulations.
• Agent Gateway: Centralized control plane for agent permissions, identity, read/write scopes, audit trails, and human-in-the-loop approvals.
• Unified data layer: Smart Storage auto-annotates unstructured data; Cross-Cloud Lakehouse lets agents read data natively without custom pipelines.
• Hybrid multicloud and sovereignty: Google Distributed Cloud enables air-gapped isolation and strict data residency where required.
• Edge AI: Optimized models on smartphones, IoT devices, and local servers reduce latency, sustain operations during connectivity loss, and cut per-token costs.
• Energy efficiency: TPU 8t delivers nearly 3x the performance of the prior generation while being up to twice as energy-efficient.
• AI Hypercomputer: Co-designed silicon (TPUs, GPUs, CPUs) engineered with Virgo Network, Managed Lustre, Hyperdisk storage, and GKE orchestration.

Softprom and Google

Softprom is the official partner of Google. Enterprise customers can plan and implement Google Cloud infrastructure for agentic AI — from AI Hypercomputer and TPU-based compute to Agent Gateway governance and hybrid deployments with Google Distributed Cloud.

This content was prepared as part of the Softprom DistriFlow project — an automated system for monitoring and adapting vendor news. Original source: original article.