vLLM 0.25.1

Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices

arXiv:2607.08786v1 Announce Type: new Abstract: With the growing deployment of large language models (LLMs), LLM inference cost has become a

How continuous batching enables 23x throughput in LLM inference while reducing p50 latency

BIFE: A New Framework for Stable Minute-Long Video Generation Using Semantic Sparse KV Cache and Block Forcing

Long video generation is a critical step toward building realistic world models, requiring both high visual fidelity and long-range interact

DepthWeave-KV: Unlocking Long Context Efficiency

DepthWeave-KV tackles long-context LLM memory bottlenecks with token-adaptive cache compression, achieving 8.3x reduction and high throughpu

6cy: High-Performance Streaming Container Format with Rust Reference Implementation

High-performance, streaming-first container format with per-block codec polymorphism and robust data recoverability. Reference implementatio

A New Streaming API for Node v0.10