Google has open-sourced TorchTPU, its native PyTorch stack for Google Cloud TPUs, the company announced at PyTorch Conference 2026 on October 20, 2026. Co-developed with the PyTorch team at Meta, the project makes TPUs a native PyTorch device alongside GPUs and CPUs, letting existing PyTorch applications move to Google’s AI chips with what the company calls a simple configuration change and no code rewriting.
What Was Announced?
In a blog post by Mark Lohmeyer, VP and GM of AI and computing infrastructure, and Bill Jia, VP of engineering, Google said: “With TorchTPU, TPUs join GPUs and CPUs as a native option for PyTorch. You can now train, serve, and deploy models on TPUs using the tools you already rely on, including vLLM, SGLang, llm-d, TorchTitan, Ray, TorchRec, Hugging Face, LiteRT.”
The repository is released under Apache 2.0, with version 0.2.0 available on PyPI supporting Python 3.11 and above. Google says TorchTPU replaces PyTorch/XLA and recommends it for all new workloads, positioning it as the standard way to run PyTorch on TPUs going forward.
Image: Google Cloud
Key Features and Details
TorchTPU plugs into PyTorch’s PrivateUse1 interface, exposing TPUs as a native device through torch.device(“tpu”). Tensors are ordinary PyTorch tensors, with no subclasses or wrappers. It runs eager by default for debugging, then compiles for production performance through torch.compile and XLA, matching the normal PyTorch workflow developers already know.
Four repositories shipped together: TorchTPU itself, plus native integrations for vLLM, SGLang and TorchTitan, developed in collaboration with Inferact (the maker of vLLM) and RadixArk (the maker of SGLang). Google also says that in Hugging Face Transformers v5, deploying models to TPUs is as simple as passing device_map=”tpu”, and that Diffusers runs natively on TPUs.
On hardware, TorchTPU is tested on TPU generations from v5e all the way up to clusters of 8,192 Ironwood (TPU7x) chips. Serving supports FP8, MXFP4 and NVFP4 checkpoints. One self-reported figure: 52% model FLOPS utilization training Qwen 3 8B with TorchTitan on Ironwood. Google also introduced a TPU agent suite and updated LiteRT to improve PyTorch portability across its chips, cloud services and edge devices.
Graphic: GPTOnline, using Google Cloud assets
Why It Matters
As TechCrunch put it, Google has joined Amazon in challenging NVIDIA’s hold on AI app developers. PyTorch is the toolkit AI developers and startups use to build, train and run models, and much of its ecosystem has long been optimized for NVIDIA GPUs by default. The software layer, not just the silicon, is what makes switching hardware expensive: training code, serving engines and tutorials all assume one vendor, so moving meant rewriting.
The timing is pointed. TorchTPU landed one day after Amazon released TorchNeuron, its own PyTorch variant for Trainium chips and AWS. Amazon also said it is contributing code to the core PyTorch project to decouple features from NVIDIA-specific dependencies and make the framework hardware-agnostic by default. Google may be going further on ecosystem compatibility: it worked directly with Meta, Hugging Face, Inferact and RadixArk so TorchTPU’s libraries slot into the tools developers already use.
For Google Cloud, the strategic goal is clear: lower the friction of moving PyTorch workloads onto TPUs and, by extension, onto Google Cloud. For developers, the near-term question is how much of a GPU-tuned stack actually ports over cleanly, which only real-world testing will answer.
