Google's latest innovation in the field of artificial intelligence (AI) is the unveiling of its eighth-generation Tensor Processor Units (TPUs), specifically designed for the agentic era. This new generation, comprising TPU 8t and TPU 8i, is a testament to Google's commitment to pushing the boundaries of AI technology. These chips are not just about processing power; they are about revolutionizing the way AI agents operate and interact with their environment.
A Chip for Every Task
TPU 8t and TPU 8i are purpose-built to handle the diverse demands of AI, from training massive models to running complex agents. TPU 8t, the training powerhouse, is designed to accelerate the development of large-scale models, reducing the time from months to weeks. It boasts an impressive scale, with a single TPU 8t superpod capable of handling 9,600 chips and two petabytes of shared high-bandwidth memory, delivering 121 ExaFlops of compute. This level of performance is further enhanced by 10x faster storage access and TPUDirect, ensuring maximum utilization of the system.
On the other hand, TPU 8i is tailored for the intricate, collaborative work of AI agents. It addresses the 'waiting room' effect by breaking the 'memory wall' and optimizing the system for superior performance. With 288 GB of high-bandwidth memory and 384 MB of on-chip SRAM, TPU 8i keeps models' active working sets on-chip, ensuring minimal lag. The chip's Axion-powered efficiency and custom Boardfly architecture further contribute to its exceptional performance-per-dollar, enabling businesses to serve nearly twice the customer volume at the same cost.
Co-Design Philosophy
Google's TPU 8i and 8t chips are the result of a meticulous co-design process, where every specification is tailored to overcome AI's most significant challenges. The Boardfly topology, for instance, is specifically designed to meet the communication demands of advanced reasoning models. Similarly, the SRAM capacity in TPU 8i is sized to accommodate the KV cache footprint of reasoning models at production scale. This level of customization ensures that the hardware is perfectly aligned with the software and model requirements, leading to unprecedented performance and efficiency.
Power Efficiency and Infrastructure
Google's focus on power efficiency is evident in the TPU 8t and 8i chips, which deliver up to two times better performance-per-watt compared to the previous generation, Ironwood. This efficiency is not limited to the chips themselves but extends to the entire stack, including the data center. Google's fourth-generation liquid cooling technology and the integration of network connectivity with compute on the same chip significantly reduce power costs. Moreover, the company's data centers are co-designed with the TPUs, enabling them to deliver six times more computing power per unit of electricity than just five years ago.
The Agentic Era
As AI agents become more sophisticated, the infrastructure must evolve to meet their demands. TPU 8t and TPU 8i are Google's answer to this challenge, offering specialized architectures that redefine what is possible in AI. From building the most advanced models to orchestrating swarms of agents, these chips are poised to drive the next wave of AI innovation. With their general availability later this year, TPU 8t and TPU 8i will be a powerful addition to Google's AI Hypercomputer, providing a unified stack of purpose-built hardware, open software, and flexible consumption models.
In conclusion, Google's eighth-generation TPUs represent a significant leap forward in AI technology, showcasing the company's commitment to innovation and its vision for the future of AI. As the agentic era unfolds, these chips will play a pivotal role in shaping the capabilities and possibilities of AI, from training massive models to running complex agents.
