Report: Infrastructure Is the Missing Piece in Gen AI Strategy

• By David Ramel
• 04/15/25

While generative AI has become central to digital transformation strategies, a new report from Google Cloud reveals most organizations aren't yet equipped to support it at scale. The report suggests that success hinges not on tech alone, but on the strength of the underlying infrastructure, which needs to be secure, scalable, and ready for real-world complexity. From hybrid cloud to edge deployments, infrastructure has emerged as the missing piece in the gen AI puzzle, Google asserted.

As the company's "2025 State of AI Infrastructure" report notes: "Gen AI is rapidly becoming mainstream: 98% of organizations are actively exploring its use, with 39% already deploying it in production. For technology leaders, understanding the infrastructure implications of this shift is crucial. Widespread adoption of gen AI is transforming businesses across industries, and the right cloud infrastructure is critical for success."

The success of gen AI depends on having the right infrastructure — not just adding capacity, but building a secure, scalable, and cost-efficient foundation that supports AI's complex demands across cloud, data center, and edge environments. As gen AI reshapes IT strategy, organizations must choose infrastructure that serves not only AI specialists but also developers and business teams enterprise-wide.

"While the explosion of interest in AI has sparked widespread experimentation, the focus has now shifted to practical application," said Google Cloud exec Nirav Mehta. "Tech leaders are prioritizing 'how' over 'if' — how to integrate, secure, and scale gen AI to deliver sustainable value. This transformative phase demands a strategic approach to platform management and cost-effective deployment. However, this transformation is far from straightforward. Organizations face significant challenges, including legacy system integration, complex migrations, and the need to tailor AI solutions to their unique business contexts."

Key findings of the report as presented by Google Cloud emphasize that optimized infrastructure is crucial for gen AI success:

• Gen AI adoption is nearly universal: 98% of organizations are experimenting with, developing, or using gen AI in production.

  

• Dual focus on internal and external use cases: 64% of organizations prioritize both enhancing customer experience and improving internal operations with gen AI.
• Security and data are top concerns: 70% of organizations face challenges with data governance, integration, and training data quality.

  

• Cost-efficiency is critical: 83% of tech leaders prioritize cost when evaluating gen AI infrastructure solutions.
• Robust AI platforms are essential: Success hinges on infrastructure that is secure, scalable, performant, and cost-efficient.
• Edge computing is expanding AI's reach: 73% consider deploying gen AI models on distributed or edge devices to be important.
• Hybrid cloud dominates AI infrastructure strategy: 74% prefer a hybrid approach to balance flexibility, performance, and compliance.
• Cloud providers are central to gen AI success: Most organizations rely on cloud vendors for managed services, infrastructure, and scalable AI tooling.

The Google Cloud report is based on quantitative research conducted in August and September 2024, surveying 513 technology leaders from companies in North America, Europe, and Asia-Pacific. Respondents were required to have significant decision-making authority or influence over technology adoption and infrastructure, and represented organizations either currently using AI or planning to implement it within 12 months. Participants had varying levels of AI expertise and were familiar with multiple gen AI applications. The survey was conducted independently — respondents were unaware that Google Cloud was the research sponsor.

For the full report, visit the Google Cloud site (registration required).