DFlash Speculative Decoding Boosts NVIDIA Blackwell Inference Performance Up to 15x
As AI systems move from single-turn interactions to coordinated multiagent workflows, low-latency inference becomes increasingly important.
Read the full articleYou might also wanna read
NVIDIA Tests DFlash, a Block-Diffusion Method to Accelerate LLM Inference on GPUs
DFlash replaces sequential speculative drafting with block-diffusion token prediction on NVIDIA GPUs, aiming to raise throughput for latency
stechtimes.com·14d ago
DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding
arXiv:2607.08642v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting several tokens and verifying them
Speculative Speculative Decoding: Parallelizing LLM Inference for Faster Performance
Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by
arxiv.org·4mo ago
Three training-time interventions improve diffusion-based speculative decoding by 21-76%
Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs s
arxiv.org·15d ago
Fast-dLLM: Training-Free Acceleration Method for Diffusion Language Models Using KV Cache and Parallel Decoding
Diffusion-based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation with parallel decoding capa
arxiv.org·8mo ago
Optimizing LLM Inference by Combining NVIDIA DGX Spark and Apple Mac Studio Architectures
Disaggregating Prefill and Decode: Faster First Tokens, Faster Streams
Comments
Sign in to join the conversation.
No comments yet. Be the first.