Google announced two new TPU chips at Cloud Next — TPU 8i and TPU 8t — each targeting a distinct bottleneck in modern AI workloads.
TPU 8i is built for inference in agentic workflows. AI agents that reason, plan, and execute multi-step tasks need low-latency responses to feel usable. Google designed 8i specifically to hit that speed bar. TPU 8t takes the other end: training. It runs large, complex models on a single unified memory pool, removing the need to shard across multiple chips for the biggest workloads.
The split architecture signals where Google sees the compute problem heading. Training and inference have different hardware requirements — training needs massive memory bandwidth for large parameter counts, inference needs fast, responsive throughput for real-time task execution. Shipping two dedicated chips instead of one general-purpose chip is a direct acknowledgment of that divergence.
NOTE
TPU 8t’s single-pool memory design matters for teams training frontier-scale models. Sharding across chips introduces coordination overhead and failure points. A unified pool simplifies that considerably.
For teams building on Google Cloud, the practical question is which chip maps to which workload. If you’re running agent pipelines where response latency affects user experience, 8i is the relevant hardware. If you’re training large models and hitting memory limits that force awkward parallelism strategies, 8t is worth evaluating. Google hasn’t published pricing or availability dates in this announcement, so the immediate action is to watch Google Cloud’s TPU documentation for access details.