Weaver: Autoregressive drafting with factorized priors for efficient speculative decoding
Speculative decoding greatly increases the interactivity of autoregressive language models by trading off computation for extra tokens generated in a single forward pass. Factorized draft models are…
Read the full articleYou might also wanna read
Speculative Speculative Decoding: Parallelizing LLM Inference for Faster Performance
Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by
arxiv.org·4mo ago
DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding
arXiv:2607.08642v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting several tokens and verifying them
Three training-time interventions improve diffusion-based speculative decoding by 21-76%
Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs s
arxiv.org·22d ago
DSpark: A Speculative Decoding Framework Using Semi-Autoregressive Generation and Confidence-Scheduled Verification for LLM Inference Acceleration
Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent
arxiv.org·9d ago
LK Losses: A New Training Objective to Optimize Acceptance Rate in Speculative Decoding for LLMs
Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate
arxiv.org·1mo ago
Fast-dLLM: Training-Free Acceleration Method for Diffusion Language Models Using KV Cache and Parallel Decoding
Diffusion-based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation with parallel decoding capa
arxiv.org·8mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.