Serving DeepSeek-V4: why million-token context is an inference systems problem
DeepSeek-V4 makes million-token context a serving-systems problem. Together AI explores the inference work behind V4 on NVIDIA HGX B200, including compressed KV layouts, prefix caching, kernel…
Read the full articleYou might also wanna read
DeepSeek-V4 Series Preview: Million-Token Context MoE Models with 1.6T Parameters
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
DeepSeek Unveils Highly Anticipated V4 Model With 1M Context Window, Challenging AI Industry Leaders
DeepSeek V4 introduces dual models with 1M context, MoE efficiency, and competitive pricing for scalable AI deployment.
Technical Implementation of DeepSeek LLM Deployment with Expert Parallelism on 96 H100 GPUs
DeepSeek is a popular open-source large language model (LLM) praised for its strong performance. However, its large size and unique architec
lmsys.org·10mo ago
DeepSeek-V4: Hybrid Sparse-Attention Architecture Enables Efficient Million-Token Context Inference
Paper Review DeepSeek-V4 Review: Why Million-Token Context Needs Efficient Attention, Not Just Larger Windows DeepSeek V4 pairs a hybrid spa
artgor.medium.com·1mo agoWe Ran DeepSeek V4 Flash at 1M Context on Two NVIDIA DGX Sparks. Here is What Happened.
Real-world benchmarks running DeepSeek V4 Flash (284B MoE) across two NVIDIA DGX Sparks with tensor parallelism over 200Gbps RoCE. 41 tok/s
vLLM Completes Migration to V1 Engine with DeepSeek Serving at 2.2k Tokens/Second on H200 Hardware
In v0.11.0, the last code from vLLM V0 engine was removed, marking the complete migration to the improved V1 engine architecture. This achie
blog.vllm.ai·5mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.