DSpark: A Speculative Decoding Framework Using Semi-Autoregressive Generation and Confidence-Scheduled Verification for LLM Inference Acceleration
By
[Submitted on 6 Jul 2026]
Summary
DSpark is a speculative decoding framework for Large Language Models (LLMs) that improves inference efficiency by combining semi-autoregressive generation with confidence-scheduled verification. It addresses two key problems in existing parallel drafters: rapid acceptance decay due to lack of inter-token dependencies, and wasted batch capacity from indiscriminately verifying high-risk tokens. DSpark uses a semi-autoregressive architecture (parallel backbone + lightweight sequential module) to maintain draft quality, and employs confidence-scheduled verification that dynamically adjusts verification length per request based on survival probabilities and throughput profiles. When deployed in DeepSeek-V4's serving system, DSpark accelerated per-user generation speeds by 60-85% over the production baseline (MTP-1) at matched throughput levels, and enabled previously unattainable performance tiers under strict interactivity constraints.
Source
bskyDSpark: A Speculative Decoding Framework Using Semi-Autoregressive Generation and Confidence-Scheduled Verification for LLM Inference Accelerationarxiv.orgKey quotes
· 5 pulledDSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay.
DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles.
Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels.
By preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.
While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies.
You might also wanna read
Speculative Speculative Decoding: Parallelizing LLM Inference for Faster Performance
Researchers introduce speculative speculative decoding (SSD), a novel technique to accelerate large language model inference by parallelizin
arxiv.org·4mo ago
DeepSpec/DSpark_paper.pdf at main · deepseek-ai/DeepSpec
github.com·10d ago
speculators/examples/train/dspark_qwen3_0_6b_sharegpt_online.sh at main · vllm-project/speculators
github.com·5d ago
Fast-dLLM: Training-Free Acceleration Method for Diffusion Language Models Using KV Cache and Parallel Decoding
Researchers introduce Fast-dLLM, a training-free acceleration method for diffusion-based large language models that addresses their slower i
arxiv.org·8mo ago
ChunkLLM: A Lightweight Framework for Accelerating Large Language Model Inference
ChunkLLM is a lightweight, pluggable framework designed to accelerate large language model inference by addressing computational inefficienc
arxiv.org·8mo ago
ATLAS: Adaptive Learning System for Faster LLM Inference Without Manual Tuning
Together AI introduces ATLAS (AdapTive-LeArning Speculator System), a novel runtime-learning accelerator for LLM inference that automaticall
together.ai·8mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.