Google Gemma 2 — Distillation, Sliding-Window Attention, and Open Weights Done Right

Google's Gemma 2 family (June–July 2024) introduced a 27B model that outranks Llama 3 70B on Chatbot Arena despite being 2.5× smaller, through interleaved sliding-window attention, grouped-query attention, logit soft-capping, and knowledge distillation. Three sizes: 2B, 9B, 27B. 8K context. Gemma Terms license. Rating: 4/5.