Every other frontier lab rents its intelligence from the same landlord. OpenAI, Anthropic, Meta, xAI — all of them ultimately train and serve their models on NVIDIA GPUs. Google is the exception. For a decade, Google has been quietly designing, building, and deploying its own AI chips, the Tensor Processing Units (TPUs), and that vertical integration is now paying off in a way competitors are struggling to replicate.

A decade of silicon nobody noticed#

Google's chip story started in 2015 with TPU v1, an application-specific integrated circuit (ASIC) hardwired for machine learning. The logic was straightforward: GPUs are general-purpose parallel processors that were originally designed for rendering graphics; an ASIC can do one thing — matrix multiplication — faster and with far less energy. At Google's scale, with billions of users hitting Search, Gmail, and YouTube daily, even small per-query efficiency gains compounded into enormous savings.

Fast-forward to 2025 and Google is on its seventh generation. The TPU line has grown through Trillium (v6, announced May 2024) to Ironwood (v7, announced at Google Cloud Next 25 in April 2025), and Google claims roughly 30x improvement in power efficiency versus its first Cloud TPU from 2018.

Ironwood by the numbers#

Ironwood is the first TPU Google has designed specifically for inference — running models rather than training them — which Google calls the "age of inference." The headline specs, from Google's own announcement:

  • 4,614 TFLOPs of peak FP8 compute per chip
  • 192 GB of high-bandwidth memory per chip (6x Trillium)
  • 7.37 TB/s of memory bandwidth per chip (4.5x Trillium)
  • 1.2 TBps of bidirectional Inter-Chip Interconnect bandwidth (1.5x Trillium)
  • Pods scaling to 9,216 liquid-cooled chips delivering 42.5 Exaflops — more than 24x the per-pod compute Google attributes to the El Capitan supercomputer
  • 2x the performance per watt of Trillium

On paper, that's squarely in the same class as NVIDIA's Blackwell-generation hardware. Where Ironwood pulls ahead is the plumbing: the interconnect is integrated tightly enough that Google treats a 9,216-chip pod as a single coherent supercomputer, avoiding some of the networking latency and cost penalties that modular GPU clusters carry.

Hardware is only half the stack. Google's Pathways runtime lets developers compose hundreds of thousands of chips as one machine, and the PyTorch/XLA bridge (plus the newer Torchax layer) means researchers can now run PyTorch code directly on TPUs without rewriting it for JAX — closing much of the software gap that once made CUDA the path of least resistance.

Gemini runs on Gemini's silicon#

The most important detail isn't the specs — it's what's running on them. Google confirms that Gemini 2.5 and AlphaFold run on TPUs. And Gemini 3, released in November 2025 and widely rated among the top models on benchmark leaderboards, was reportedly trained entirely on TPUs — no NVIDIA GPUs involved, according to analysts at Bloomberg Intelligence.

That's a milestone the industry hasn't seen before: a frontier-leading model built start to finish on custom silicon. It also validates the strategy for Google's biggest bet, its partnership with Anthropic — a multi-billion-dollar deal giving Anthropic access to up to a million TPU chips through Google Cloud, with Ironwood as the workhorse.

Where the cost advantage comes from#

The economics are the heart of the story:

  • Margins captured internally. NVIDIA's gross margins on data-center GPUs have famously hovered around 75%. Google builds TPUs with design partner Broadcom and pays roughly at cost — it captures that margin instead of paying it.
  • Energy is the new bottleneck. Power availability, not just chips, now constrains AI deployment. Ironwood's 2x perf-per-watt gain over Trillium (and 30x over the first Cloud TPU) means Google can squeeze more inference out of every megawatt.
  • Inference at scale is the game. Training is expensive but one-time; serving models to billions of users is a perpetual cost. Purpose-built inference silicon is where custom chips compound their advantage.

This doesn't mean TPUs are better at everything. GPUs remain far more flexible, NVIDIA's CUDA software ecosystem is far more mature, and TPUs live inside Google Cloud's walled garden — you can't buy one and rack it yourself. For researchers and startups, CUDA is still the default.

The rest of the field is catching up — differently#

Google isn't alone in vertical silicon anymore. Amazon has Trainium and Inferentia, Microsoft has the Maia 100, and Meta has been exploring its own accelerator strategy. Even Meta has reportedly held talks with Google about using TPUs, which would turn Google's in-house advantage into a direct attack on NVIDIA's core business.

Meanwhile, NVIDIA is pushing back hard with its Vera Rubin platform and a claim that it's "a generation ahead" of custom silicon. The chip war is now a two-front fight: performance and openness versus cost and integration.

What's next#

Google is already previewing the eighth TPU generation, with reports pointing to a split architecture — separate training and inference designs built on TSMC's 2nm process — arriving as soon as 2027. Google has also committed to enormous infrastructure spending, reportedly approaching $185 billion in 2026, much of it flowing into this vertical stack.

The takeaway#

Google's TPU program is a ten-year bet that is only now becoming legible: owning the silicon lets a frontier lab train and serve models at structural costs no GPU renter can match. Whether the moat holds depends on two things — whether NVIDIA's software edge (CUDA) outweighs Google's hardware edge, and whether Google can sell TPUs to competitors without arming them. For now, though, Gemini is the living proof that the "NVIDIA tax" is optional. And in the age of inference, that's the most expensive tax there is.