DepthWeave-KV: Unlocking Long Context Efficiency
DepthWeave-KV tackles long-context LLM memory bottlenecks with token-adaptive cache compression, achieving 8.3x reduction and high throughput.
Read the full articleYou might also wanna read
LMCache: A KV Cache Management Layer for Scalable LLM Inference
Learn how LMCache reduces TTFT and improves throughput for LLM inference with tiered KV cache offloading, non-prefix reuse, PD disaggregatio
Expected Attention: KV Cache Compression Method for Efficient LLM Inference
Memory consumption of the Key-Value (KV) cache represents a major bottleneck for efficient large language model inference. While attention-s
arxiv.org·9mo ago
Attention Matching: Fast KV Cache Compaction for Language Models
Scaling language models to long contexts is often bottlenecked by the size of the key-value (KV) cache. In deployed settings, long contexts
arxiv.org·4mo ago
How New Open-Weight LLMs Are Reducing Long-Context Costs: KV Sharing, Attention Budgeting, and Compressed Attention
From Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs
magazine.sebastianraschka.com·1mo ago
Sequential KV Cache Compression Using Probabilistic Language Tries and Predictive Delta Coding
Recent work on KV cache quantization, culminating in TurboQuant, has approached the Shannon entropy limit for per-vector compression of tran
arxiv.org·2mo ago
δ-mem: A Compact Online Memory Mechanism for Efficient Long-Context LLM Processing
Large language models increasingly need to accumulate and reuse historical information in long-term assistants and agent systems. Simply exp
arxiv.org·1mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.