Hardware-Aware Dynamic Speculative Decoding: Optimizing LLM Inference Through Adaptive Draft Token Selection
See how DSD overcomes the limitations of standard SD by controlling the optimal K based on hardware constraints.
Read the full articleYou might also wanna read
Speculative Speculative Decoding: Parallelizing LLM Inference for Faster Performance
Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by
arxiv.org·4mo ago
DSpark: A Speculative Decoding Framework Using Semi-Autoregressive Generation and Confidence-Scheduled Verification for LLM Inference Acceleration
Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent
arxiv.org·4d ago
DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding
arXiv:2607.08642v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting several tokens and verifying them
LK Losses: A New Training Objective to Optimize Acceptance Rate in Speculative Decoding for LLMs
Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate
arxiv.org·1mo ago
DeepSpec/DSpark_paper.pdf at main · deepseek-ai/DeepSpec
DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms - deepseek-ai/DeepSpec
github.com·14d ago
Research Directions for Overcoming Memory and Interconnect Challenges in Large Language Model Inference Hardware
Large Language Model (LLM) inference is hard. The autoregressive Decode phase of the underlying Transformer model makes LLM inference fundam
arxiv.org·5mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.