In Part 1 of our Understanding Modern AI series, we explored the infrastructure technologies powering modern Large Language Models (LLMs), including GPU architectures, NVIDIA CUDA, Linux, containerization, Kubernetes, high-performance networking, storage, and model inference.

But how do the companies behind ChatGPT, Claude, Gemini, and other major LLMs actually operate their infrastructure?

Do they own and operate their own data centers, rely entirely on public cloud providers such as Microsoft Azure, Amazon Web Services (AWS), and Google Cloud, or combine different infrastructure models?

And what hardware powers these enormous AI environments? Are they all using NVIDIA GPUs, or are alternatives such as Google’s Tensor Processing Units (TPUs) and AWS Trainium becoming increasingly important?

In this second part, we will take a closer look at the real-world infrastructure strategies of major AI developers, including OpenAI, Anthropic, Google DeepMind, Meta, xAI, and DeepSeek.

We will explore their publicly documented hardware platforms, cloud partnerships, dedicated AI data centers, and the growing importance of specialized infrastructure providers, often referred to as neoclouds.

We will also distinguish between training infrastructure, used to develop and improve AI models, and inference infrastructure, responsible for serving millions of user requests.

The objective is to understand not only which technologies these companies use, but also who provides the computing resources, who owns the hardware, and who actually operates the underlying infrastructure.

Because while the AI models themselves receive most of the attention, the infrastructure required to develop and operate them is equally fascinating.


Table Of Contents
  1. AI Infrastructure Models – Hyperscalers, Dedicated Data Centers, and Neoclouds
  2. OpenAI – Microsoft Azure, Oracle, NVIDIA, and Stargate
  3. Anthropic – AWS Trainium, Google TPUs, and Fluidstack
  4. Google DeepMind – Gemini and Google's Own AI Infrastructure
  5. Meta, xAI, and DeepSeek – Different Approaches to AI Infrastructure
  6. NVIDIA GPUs vs. Google TPUs vs. AWS Trainium
  7. Training vs. Inference – Where the Infrastructure Is Used
  8. Conclusion – Comparing the Infrastructure Strategies
  9. Links


AI Infrastructure Models – Hyperscalers, Dedicated Data Centers, and Neoclouds

Developing and operating modern Large Language Models (LLMs) requires enormous computing resources, particularly for training models with billions of parameters and serving millions of inference requests.

However, not every AI company owns or operates the underlying infrastructure. Companies such as OpenAI and Anthropic rely on different combinations of cloud providers, dedicated computing facilities, and specialized AI infrastructure partners.

Generally, we can distinguish between three major infrastructure models.

Hyperscalers – Public Cloud Infrastructure

Hyperscalers such as Microsoft Azure, Amazon Web Services (AWS), and Google Cloud operate massive global data center infrastructures and provide access to GPU and other AI accelerator resources.

AI developers can use these platforms without having to build and operate their own physical data centers.

Dedicated AI Data Centers

Some AI developers invest in dedicated infrastructure specifically designed for their training and inference requirements.

These facilities may contain thousands of interconnected AI accelerators, high-performance networking fabrics, specialized cooling systems, and enormous power capacities.

Importantly, financing or owning dedicated computing infrastructure does not necessarily mean operating the physical data center. Construction, facility management, and hardware operations may be handled by infrastructure partners.

Neoclouds – Specialized AI Infrastructure Providers

Neoclouds are infrastructure providers specializing in GPU-accelerated computing rather than offering the broad range of services associated with traditional hyperscalers.

Examples include CoreWeave, Lambda, and Fluidstack.

These providers typically focus on high-performance AI computing capacity, including GPU clusters, bare-metal servers, and dedicated infrastructure for AI training and inference.

Comparing the Infrastructure Models

Infrastructure modelMain characteristic
HyperscalerAI computing resources provided through large public cloud platforms
Dedicated AI data centerInfrastructure specifically designed or reserved for particular AI workloads
NeocloudSpecialized provider focused on GPU and AI computing infrastructure


These models are not mutually exclusive. A major AI developer may use several simultaneously, depending on computing requirements, hardware availability, cost, and strategic partnerships.

In the following sections, we will examine how major LLM developers combine these approaches to build and operate their AI infrastructure.

OpenAI – Microsoft Azure, Oracle, NVIDIA, and Stargate

OpenAI, the company behind ChatGPT and the GPT model family, provides an interesting example of how AI infrastructure strategies evolve as computing requirements increase.

Originally relying heavily on Microsoft Azure, OpenAI has expanded its infrastructure strategy to include additional cloud providers, dedicated AI data centers, and partnerships with hardware manufacturers.

Microsoft Azure – The Foundation

Microsoft and OpenAI have collaborated on AI infrastructure since 2016, with their strategic investment partnership beginning in 2019.

Azure provided the large-scale GPU computing resources required for OpenAI’s research, model training, and inference workloads.

Microsoft remains OpenAI’s primary cloud partner, but their agreements have evolved to allow OpenAI greater flexibility in using additional infrastructure providers.

Stargate – Dedicated AI Infrastructure

In January 2025, OpenAI announced The Stargate Project, an initiative originally targeting up to $500 billion in investment and 10 gigawatts of AI infrastructure capacity in the United States.

The initial partners include:

  • OpenAI: AI development and operational responsibility within the Stargate partnership.
  • SoftBank: Financial leadership and infrastructure investment.
  • Oracle: Cloud infrastructure and data center development.
  • NVIDIA: GPU accelerators and high-performance networking technology.
  • Microsoft and Arm: Additional technology partners.


One of the first Stargate facilities is located in Abilene, Texas, where Oracle Cloud Infrastructure (OCI) provides computing capacity using NVIDIA GB200 GPU systems.

The facility began supporting initial AI training and inference workloads in 2025, while additional data center locations and capacity expansions were announced.

Expanding Beyond Microsoft Azure

OpenAI is also diversifying its computing resources through additional partnerships.

In November 2025, OpenAI announced a $38 billion agreement with AWS, providing access to large-scale NVIDIA GPU infrastructure through Amazon EC2 UltraServers.

Other infrastructure partnerships include CoreWeave, a specialized AI cloud provider, and additional accelerator manufacturers.

OpenAI has also expanded beyond NVIDIA-only hardware strategies, exploring alternative accelerators and developing custom inference hardware to improve performance and reduce operating costs.

Training and Inference Infrastructure

OpenAI requires enormous computing resources for two different purposes:

  • Training: Large GPU and accelerator clusters process massive datasets to develop and improve AI models.
  • Inference: Distributed computing infrastructure executes trained models to serve requests from ChatGPT users, API customers, and enterprise applications.


While training emphasizes massive parallel processing and high-speed communication between accelerators, inference infrastructure must also optimize response latency, GPU memory utilization, throughput, and operating costs.

OpenAI’s Infrastructure Strategy

OpenAI’s infrastructure strategy increasingly combines hyperscaler cloud platforms, dedicated AI data centers, specialized AI infrastructure providers, and custom hardware development.

Rather than depending on a single cloud provider or accelerator architecture, the company is building a diversified computing ecosystem to support growing training and inference requirements.

It is important to distinguish between announced investments, contracted capacity, and infrastructure already operating in production. Large-scale AI data center projects frequently require several years to reach their full planned capacity.

Anthropic – AWS Trainium, Google TPUs, and Fluidstack

Anthropic, the company behind the Claude family of Large Language Models, follows a diversified infrastructure strategy that combines major cloud providers, specialized AI accelerators, and dedicated data center investments.

Unlike companies that primarily rely on NVIDIA GPUs, Anthropic uses several hardware platforms to support model training and inference, including AWS Trainium, Google Tensor Processing Units (TPUs), and NVIDIA GPUs.

Amazon AWS – The Primary Cloud and Training Partner

Amazon Web Services (AWS) has been Anthropic’s primary cloud and training partner since 2023.

A major component of this collaboration is Project Rainier, a large-scale AI computing infrastructure built around Amazon’s custom Trainium accelerators.

In April 2026, Anthropic and Amazon announced an expanded agreement covering up to 5 gigawatts of additional computing capacity. Anthropic also reported using more than one million Trainium2 chips for training and serving Claude.

The partnership demonstrates how AWS is developing its own AI accelerator ecosystem rather than relying exclusively on NVIDIA hardware.

Claude models are also available to enterprise customers through Amazon Bedrock, AWS’s managed generative AI platform.

Google Cloud – Tensor Processing Units (TPUs)

Anthropic also relies heavily on Google’s custom-designed Tensor Processing Units (TPUs).

In October 2025, Anthropic announced plans to expand its Google Cloud infrastructure by up to one million TPUs, representing more than one gigawatt of planned computing capacity during 2026.

An additional agreement with Google and Broadcom, announced in April 2026, provides for several gigawatts of next-generation TPU capacity beginning in 2027.

TPUs are specialized AI accelerators developed by Google to efficiently execute machine learning operations.

This gives Anthropic an alternative to NVIDIA GPUs and AWS Trainium, allowing it to distribute workloads across different hardware platforms.

Fluidstack – Dedicated AI Data Centers

In November 2025, Anthropic announced a $50 billion investment in American AI infrastructure, partnering with specialized AI infrastructure provider Fluidstack.

The initiative includes custom-built data centers in Texas and New York, with initial facilities scheduled to come online throughout 2026.

Unlike conventional public cloud services, these facilities are designed specifically around Anthropic’s computing requirements.

However, dedicated infrastructure does not necessarily mean Anthropic physically owns or operates every facility. Fluidstack is responsible for delivering the custom data center infrastructure.

This illustrates the growing trend toward dedicated AI computing capacity operated through specialized infrastructure partnerships.

Microsoft Azure and NVIDIA

Anthropic further expanded its infrastructure strategy in November 2025 through partnerships with Microsoft and NVIDIA.

The agreement includes a commitment to purchase $30 billion of Azure computing capacity, alongside plans for additional capacity using NVIDIA Grace Blackwell and future Vera Rubin architectures.

This also makes Claude available through Microsoft’s enterprise AI ecosystem, complementing its availability on AWS and Google Cloud.

Anthropic’s Infrastructure Strategy

Anthropic’s approach combines several infrastructure models:

  • AWS: Primary cloud and training partner, using custom Trainium accelerators.
  • Google Cloud: TPU-based infrastructure for training and inference.
  • Microsoft Azure: Additional computing capacity using NVIDIA hardware.
  • Fluidstack: Dedicated AI data centers designed for Anthropic’s workloads.


By distributing computing requirements across different providers and accelerator architectures, Anthropic aims to improve scalability, hardware availability, cost efficiency, and operational resilience.

The key difference from a traditional single-cloud strategy is that Anthropic does not depend exclusively on one provider or one type of AI accelerator.

As with other large AI infrastructure projects, announced investments and contracted computing capacity should not be confused with hardware already deployed and operational.

Google DeepMind – Gemini and Google’s Own AI Infrastructure

Google follows a different infrastructure strategy compared with OpenAI and Anthropic. Through Google DeepMind and Google Cloud, the company develops its own AI models, designs custom AI accelerators, and operates a global network of data centers.

This allows Google to integrate hardware, networking, and software into a highly optimized infrastructure platform for training and serving its Gemini models.

Google Tensor Processing Units (TPUs)

Rather than relying exclusively on NVIDIA GPUs, Google has developed its own Tensor Processing Units (TPUs), specialized AI accelerators designed for machine learning workloads.

Google has introduced several TPU generations, including:

  • TPU v5p: Designed for large-scale AI training.
  • Trillium (TPU v6e): Improved compute performance, memory capacity, and energy efficiency.
  • Ironwood (TPU v7): Designed for large-scale training, reasoning, and inference, supporting configurations of up to 9,216 chips per pod.
  • TPU 8t and 8i: Eighth-generation accelerators optimized for large-scale training and inference respectively.


Google uses TPUs to train and serve Gemini models, while also making TPU computing resources available to customers through Google Cloud.

Google’s AI Data Centers and Networking

Google operates its own hyperscale data centers, combining custom-designed hardware with high-performance networking infrastructure.

Large TPU clusters are interconnected using technologies such as:

  • Inter-Chip Interconnect (ICI): High-speed communication between TPU accelerators.
  • Optical Circuit Switching (OCS): Dynamically configurable optical connections between TPU groups.
  • Jupiter and Virgo networking: Google’s data center networking technologies for connecting large-scale computing infrastructure.


These technologies allow thousands of AI accelerators to communicate efficiently during distributed model training and inference.

Software Infrastructure – JAX and Pathways

Google also develops software technologies specifically designed for its AI infrastructure.

JAX is a Python-based numerical computing framework that supports automatic differentiation, compilation, and parallel execution across accelerators.

Pathways is Google’s distributed machine learning system for coordinating large-scale AI workloads across multiple TPU devices.

Together with Google’s XLA compiler, these technologies enable efficient execution of AI models on Google’s custom hardware.

Google also supports Kubernetes-based AI workloads through Google Kubernetes Engine (GKE), although the complete internal orchestration architecture used for Gemini is not publicly documented.

Google’s Infrastructure Strategy

Google’s primary advantage is its ability to develop and operate much of its AI infrastructure internally, including:

  • Custom TPU accelerators and server hardware.
  • Global hyperscale data centers.
  • High-performance networking and storage.
  • AI frameworks and distributed training systems.
  • Gemini models and commercial AI services.


Unlike AI developers that depend heavily on external infrastructure providers, Google controls much of the underlying technology stack.

Google’s approach demonstrates how designing AI hardware, software, and infrastructure together can improve performance, scalability, and operational efficiency.

Meta, xAI, and DeepSeek – Different Approaches to AI Infrastructure

While OpenAI, Anthropic, and Google rely on different combinations of cloud platforms and custom hardware, other AI developers follow their own infrastructure strategies.

Meta, xAI, and DeepSeek provide three interesting examples of how companies approach large-scale AI computing.

Meta – Llama, NVIDIA GPUs, and Custom AI Accelerators

Meta, the company behind Facebook, Instagram, and the Llama family of LLMs, operates its own large-scale data center infrastructure.

For training Llama models, Meta has invested heavily in NVIDIA GPU clusters. In 2024, the company introduced two training clusters containing 24,576 NVIDIA H100 GPUs each, using InfiniBand and Ethernet with RoCE networking respectively.

Meta subsequently expanded its infrastructure significantly, including its Prometheus AI supercluster in Ohio. In October 2026, Meta reported that its first phase was operational, with hundreds of thousands of interconnected GPUs and plans to scale toward approximately one million GPUs.

Alongside NVIDIA hardware, Meta develops its own Meta Training and Inference Accelerator (MTIA) chips, primarily targeting recommendation systems and increasingly generative AI inference.

Meta therefore combines custom-designed data center infrastructure, commercial NVIDIA GPUs, proprietary accelerators, and open-source technologies such as PyTorch.

xAI – Grok and the Colossus Supercomputer

xAI, the developer of the Grok language models, follows an infrastructure strategy centered around massive dedicated GPU computing facilities.

Its Colossus supercomputer, located in Memphis, Tennessee, was initially built with approximately 100,000 NVIDIA H100 GPUs and subsequently expanded to 200,000 GPUs.

Colossus uses NVIDIA Spectrum-X Ethernet networking, including RDMA capabilities, to provide high-bandwidth communication between GPU servers.

Unlike a conventional cloud deployment, Colossus represents a dedicated AI computing environment designed primarily for training and developing xAI’s models.

xAI has announced plans for further expansion, although projected GPU counts should be distinguished from already operational capacity.

DeepSeek – Efficient Training with NVIDIA H800 GPUs

DeepSeek, a Chinese AI developer, has attracted attention through its focus on computational efficiency when training large language models.

According to its published technical research, DeepSeek-V3 was trained using a cluster of 2,048 NVIDIA H800 GPUs.

The model uses a Mixture-of-Experts (MoE) architecture, activating only a subset of its total parameters for each token rather than processing every parameter during each operation.

DeepSeek also employs techniques such as Multi-head Latent Attention (MLA) and optimized communication between GPUs to reduce computational and memory requirements.

The published DeepSeek-V3 training report specifies approximately 2.788 million H800 GPU-hours for full training. This figure represents accelerator usage for the reported training run, not the complete cost of developing, operating, or maintaining DeepSeek’s AI infrastructure.

Unlike Meta and xAI, DeepSeek has published comparatively limited information about its complete physical data center architecture and inference infrastructure.

Comparing the Infrastructure Strategies

CompanyPrimary approachNotable technologies
MetaLarge-scale owned infrastructure and custom siliconNVIDIA H100, MTIA, InfiniBand, RoCE
xAIDedicated GPU supercomputing facilitiesColossus, NVIDIA H100, Spectrum-X
DeepSeekEfficient model architecture and GPU utilizationNVIDIA H800, MoE, MLA


These examples demonstrate that developing competitive LLMs is not exclusively about acquiring the largest possible number of GPUs.

Infrastructure efficiency, model architecture, networking, hardware utilization, and operating costs are equally important factors when building and scaling modern AI platforms.

NVIDIA GPUs vs. Google TPUs vs. AWS Trainium

As we have seen in the previous sections, major AI developers do not necessarily rely on the same processor architectures.

While NVIDIA GPUs remain widely used for AI training and inference, companies such as Google and Amazon have developed their own specialized AI accelerators to optimize performance, energy efficiency, and infrastructure costs.

NVIDIA GPUs – The Established AI Computing Platform

NVIDIA GPUs, including the H100, H200, and Blackwell generations, provide massively parallel processing capabilities and specialized Tensor Cores for accelerating neural network calculations.

A major advantage is NVIDIA’s mature CUDA software ecosystem, supporting frameworks such as PyTorch and a wide range of optimized AI libraries.

NVIDIA GPUs are available across different cloud providers and dedicated AI infrastructure platforms.

Google TPUs – Custom Accelerators for AI

Google’s Tensor Processing Units (TPUs) are application-specific integrated circuits (ASICs) designed primarily for machine learning.

They use specialized Matrix Multiplication Units (MXUs) to efficiently execute the mathematical operations required by neural networks.

TPUs are closely integrated with Google’s software ecosystem, including JAX, TensorFlow, PyTorch, and the XLA compiler.

Google uses TPUs internally for AI workloads and makes them available to customers through Google Cloud.

AWS Trainium – Amazon’s AI Accelerator

AWS Trainium is Amazon’s custom-designed AI accelerator family, developed to support large-scale model training and inference.

Trainium uses dedicated NeuronCores and Amazon’s AWS Neuron SDK, which provides compilers, runtime libraries, and integration with frameworks such as PyTorch and JAX.

AWS also develops Inferentia, a separate accelerator family specifically designed for inference workloads.

These processors allow Amazon to offer alternatives to NVIDIA-based GPU instances within its cloud infrastructure.

Comparing the AI Accelerator Platforms

FeatureNVIDIA GPUsGoogle TPUsAWS Trainium
ManufacturerNVIDIAGoogleAmazon AWS
ArchitectureParallel GPU with Tensor CoresCustom ASIC with MXUsCustom ASIC with NeuronCores
Software ecosystemCUDA, PyTorch, TensorFlowXLA, JAX, PyTorch, TensorFlowAWS Neuron, PyTorch, JAX
Main workloadsTraining and inferenceTraining and inferenceTraining and inference
AvailabilityMultiple cloud providers and on-premisesPrimarily Google CloudAWS
Main advantageBroad software ecosystem and hardware flexibilityTight hardware/software integrationAWS integration and cost optimization

Why AI Providers Develop Their Own Accelerators

The development of custom AI processors is driven by several factors, including performance per watt, hardware availability, operating costs, and reduced dependency on external chip suppliers.

However, custom accelerators also require substantial investment in hardware design, compilers, drivers, and software frameworks.

There is no universally superior architecture. Actual performance and cost efficiency depend on the model, workload, accelerator generation, software optimization, and infrastructure configuration.

Ultimately, NVIDIA GPUs, Google TPUs, and AWS Trainium represent different approaches to solving the same challenge: executing increasingly complex AI workloads efficiently at massive scale.

Training vs. Inference – Where the Infrastructure Is Used

As discussed in Part 1, training involves developing or improving an AI model by adjusting its internal parameters, while inference uses an already trained model to process requests and generate responses.

Although both require substantial computing resources, their infrastructure requirements differ.

Training Infrastructure

Training large language models typically involves thousands of interconnected AI accelerators working together on computationally intensive tasks.

Providers such as OpenAI, Anthropic, Google, and xAI use large GPU or custom accelerator clusters with high-performance networking and storage to distribute training workloads across multiple systems.

The primary requirements are computing throughput, accelerator memory, high-speed interconnects, and efficient synchronization.

Inference Infrastructure

Inference infrastructure serves trained models to users and applications, such as ChatGPT, Claude, and Gemini.

Unlike large-scale training, inference must also handle unpredictable request volumes, concurrent users, and strict response-time requirements.

Important considerations include:

  • Low latency: Generating responses quickly.
  • High throughput: Processing large numbers of requests efficiently.
  • Scalability: Adding computing capacity as demand increases.
  • Cost efficiency: Minimizing the infrastructure cost per generated token.


Inference workloads may run on NVIDIA GPUs, Google TPUs, AWS Trainium, or other supported accelerators, depending on the provider.

Comparing Training and Inference

CharacteristicTrainingInference
Primary objectiveDevelop or improve modelsGenerate predictions and responses
Main priorityComputational throughputLatency, throughput, and cost per token
Workload patternLong-running compute-intensive jobsContinuous or variable user requests
InfrastructureLarge interconnected accelerator clustersDistributed model-serving infrastructure
Model parametersUpdated during trainingNormally remain unchanged


Major AI providers increasingly optimize their infrastructure separately for training and inference, selecting different accelerator architectures, networking configurations, and software platforms according to workload requirements.

As LLM adoption grows, inference infrastructure becomes particularly important because every user request consumes computing resources, even after the model has completed its training.

Conclusion – Comparing the Infrastructure Strategies

The infrastructure behind modern Large Language Models (LLMs) is as diverse as the companies developing them.

Throughout this article, we explored how major AI providers such as OpenAI, Anthropic, Google, Meta, xAI, and DeepSeek approach the enormous computing requirements of model training and inference.

While OpenAI and Anthropic increasingly combine hyperscaler platforms, specialized AI infrastructure providers, and dedicated data centers, Google and Meta operate extensive infrastructure of their own and invest heavily in custom AI accelerators. Meanwhile, xAI focuses on massive dedicated GPU clusters, and DeepSeek demonstrates how architectural and computational efficiency can influence infrastructure requirements.

We also examined how NVIDIA GPUs, Google TPUs, and AWS Trainium represent different approaches to accelerating AI workloads, each with its own hardware architecture and software ecosystem.

One important conclusion is that there is no single infrastructure architecture shared by all major LLM providers. Their strategies depend on computing requirements, hardware availability, costs, partnerships, and the balance between training and inference workloads.

As AI adoption continues to grow, infrastructure capacity, energy efficiency, and access to computing resources will remain critical factors in the development of increasingly capable AI models.

What’s Next – Understanding Modern AI Part 3

In Part 3: How Large Language Models Actually Work – From Tokens to Transformers, we will move beyond the underlying infrastructure and explore what actually happens inside an LLM.

We will examine tokenization, embeddings, neural networks, attention mechanisms, Transformer architectures, and how models generate responses from user prompts.

Later in the series, we will explore Agentic AI and build our own practical AI infrastructure lab using two Dell XPS notebooks, Ubuntu Server, Kubernetes, and NVIDIA GPU acceleration.

After exploring the infrastructure powering modern AI, it’s time to understand how the models themselves actually work.

Links

Announcing The Stargate Project
https://openai.com/index/announcing-the-stargate-project/

Anthropic and Amazon expand collaboration for up to 5 gigawatts of new compute
https://www.anthropic.com/news/anthropic-amazon-compute

Tensor Processing Units
https://cloud.google.com/tpu

Building Meta’s GenAI Infrastructure
https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/

xAI
https://x.ai/colossus

DeepSeek-V3 Technical Report
https://arxiv.org/abs/2412.19437

CUDA Programming Guide
https://docs.nvidia.com/cuda/cuda-programming-guide/

AWS Neuron – SDK to optimize AI and deep learning on AWS Trainium and AWS Inferentia
https://aws.amazon.com/ai/machine-learning/neuron/