How Multi-Token Prediction drafters accelerate Gemma 4 inference by up to 3x

Summary

This article explains how Google's Gemma 4 models achieve up to 3x faster inference through Multi-Token Prediction (MTP) drafters and speculative decoding. It describes the technical approach where smaller draft models predict multiple tokens simultaneously, which are then verified by the main model, overcoming the memory-bandwidth bottleneck that typically limits LLM inference speed. The article covers the architecture, training methodology, and performance benefits of this approach.