43% of IT leaders say legacy systems are holding back agentic AI: Google Cloud report

The next challenge in enterprise AI is no longer the capability of the models. It is the infrastructure supporting them. According to Google Cloud's State of Infrastructure in the Agentic AI Era report, 43% of IT leaders identify difficulty integrating legacy APIs and data sources as the biggest infrastructure gap preventing organisations from scaling agentic AI.

The findings highlight a growing disconnect between rapid advances in AI models and the readiness of enterprise technology stacks. As organisations move from AI pilots to production, fragmented data, ageing systems and engineering complexity are emerging as significant obstacles.

Legacy infrastructure is slowing enterprise AI adoption

According to Google Cloud, agentic AI systems require continuous access to business context spread across operational databases, analytics platforms and legacy applications. Many enterprises, however, continue to rely on disconnected architectures that make accessing this information difficult.

The report notes that agents operating without sufficient business context are more likely to generate incomplete or inaccurate results.

Google Cloud says enterprises cannot simply migrate vast volumes of data into AI platforms because doing so can increase both operational complexity and infrastructure costs.

Instead, organisations need data environments capable of allowing AI systems to access information across multiple environments without creating additional latency or duplication.

Infrastructure challenges extend beyond legacy systems

Legacy integration is only one part of a broader infrastructure challenge identified in the report. Key findings include:

• 43% of IT leaders cite integrating legacy APIs and data sources as the biggest infrastructure gap for agentic AI.
• 83% of organisations believe they require infrastructure upgrades to support production-grade agentic AI systems.
• 81% of leaders identify operational complexity and engineering overhead as major unforeseen costs when scaling AI.
• 36% of leaders say the lack of specialised, high-throughput vector databases limits their ability to provide AI models with sufficient business context.

According to Google Cloud, these challenges increase the amount of manual engineering required to connect AI agents with enterprise systems and data sources.

Context has become the competitive advantage

The report suggests organisations should rethink data architecture as AI systems increasingly move beyond answering questions to performing actions.

Unlike traditional AI assistants, agentic systems can independently browse information, query databases and execute workflows across multiple applications from a single prompt. This places greater pressure on compute, networking and storage infrastructure.

Google Cloud argues enterprises need to move from passive systems of record towards connected systems capable of activating trusted business data in real time.

To support this approach, the company introduced its Agentic Data Cloud at Google Cloud Next 2026, bringing together AI models, operational databases and analytics platforms within a single architecture designed for agentic workloads.

The platform combines services including BigQuery, Spanner, Apache Spark and Apache Iceberg to help AI agents access data across environments without relying on traditional movement of large datasets.

Reducing engineering overhead becomes a priority

Beyond technology performance, Google Cloud highlights the operational burden placed on engineering teams.

The report says organisations continue spending significant time connecting AI agents across fragmented systems instead of deploying production-ready applications.

According to Google Cloud, vertically integrating AI models, data platforms and infrastructure reduces network hops, improves tooling integration and allows AI agents to execute transactions using real-time operational and analytical data with less engineering effort.

The company also highlights the importance of knowledge layers that enrich enterprise data with business meaning, enabling AI agents to retrieve relevant context before making decisions.

From AI pilots to production

As enterprises expand AI deployments, infrastructure readiness is becoming as important as model capability.

Google Cloud concludes that organisations seeking long-term competitive advantage will need connected data ecosystems capable of securely delivering trusted information to AI agents at scale.

The report suggests future AI leaders will be distinguished not simply by the intelligence of their models, but by their ability to provide those models with accurate, contextual knowledge across enterprise systems while maintaining speed, reliability and cost efficiency.