DeepSeek-V4 Series Preview: Million-Token Context MoE Models with 1.6T Parameters
By
cmrdporcupine
Kettled twice. Extra chewy, extra trustworthy.
Summary
DeepSeek introduces the V4 series of Mixture-of-Experts (MoE) language models, including DeepSeek-V4-Pro (1.6T parameters, 49B activated) and DeepSeek-V4-Flash (284B parameters, 13B activated), both supporting a one-million-token context length. The models feature architectural upgrades including a Hybrid Architecture (Hyb) and optimization improvements, presented as a preview version focused on efficient long-context intelligence.
Key quotes
· 3 pulledWe present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models — DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSeek-V4-Flash with 284B parameters (13B activated) — both supporting a context length of one million tokens.
We're on a journey to advance and democratize artificial intelligence through open source and open science.
DeepSeek-V4 series incorporate several key upgrades in architecture and optimization.
You might also wanna read
DeepSeek-V4: Hybrid Sparse-Attention Architecture Enables Efficient Million-Token Context Inference
DeepSeek-V4 introduces a hybrid sparse-attention architecture combined with on-policy distillation across domain specialists, enabling 1M-to
artgor.medium.com·17h ago
DeepSeek-V3.1: Open-Source Language Model with Hybrid Inference for Advanced Reasoning and Coding
DeepSeek-V3.1 is an open-source large language model that introduces hybrid inference with both 'Think' and 'Non-Think' modes, optimized for
Product Hunt·9mo ago
DeepSeek-V3.1-Terminus: Latest Open-Source LLM with Enhanced Stability and Agent Capabilities
DeepSeek-V3.1-Terminus is the latest open-source large language model from DeepSeek, representing the 7th launch in their series. This refin
Product Hunt·1mo ago
DeepSeek previews V4 AI model, claims competitiveness with US rivals and Huawei compatibility
Chinese AI company DeepSeek has released a preview of its next-generation AI model V4, claiming it can compete with leading closed-source sy
The Verge·1mo ago
StepFun Releases Step 3.5 Flash: 196B Sparse MoE Model for OpenClaw Agents
StepFun has released Step 3.5 Flash, a 196B sparse Mixture of Experts (MoE) model that activates only 11B parameters per token for high effi
Product Hunt·3d ago
DeepSeek's V4 Model Shows Widening Gap with US Frontier AI Despite Being China's Best
DeepSeek's latest V4 model release was met with a muted reaction, as analysis by the US National Institute for Standards and Technology foun
bloomberg.com·4d ago