Understanding Continuous Batching in Large Language Models: From Attention Mechanisms to Throughput Optimization

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

arXiv:2607.08930v1 Announce Type: new Abstract: Efficient serving of diffusion large language models (dLLMs) is hindered by convergence hete

Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

arXiv:2607.08057v1 Announce Type: cross Abstract: Despite the rapid advancements of large language models (LLMs), LLM serving systems remain

A Practical Guide to Scaling Language Models: From Single Accelerators to Thousands

Training LLMs often feels like alchemy, but understanding and optimizing the performance of your models doesn't have to. This book aims to d

A Practical Guide to Scaling Language Models: From Single Accelerators to Thousands

Training LLMs often feels like alchemy, but understanding and optimizing the performance of your models doesn't have to. This book aims to d

How continuous batching enables 23x throughput in LLM inference while reducing p50 latency

Why LLMs Are So Expensive: The Quadratic Cost of Dense Attention — and How Subquadratic Claims to Fix It

Dense attention's quadratic compute scaling has been the hidden cost driver behind enterprise AI since 2017. Subquadratic's SubQ model posts

Flash-MSA Method Aims to Speed Up AI Training on Million-Token Sequences

Researchers have introduced Flash-MSA, a technique designed to accelerate the training of large language models on very long sequences of up