Xiaomi MiMo Team Details Full-Pipeline Inference Optimization for Hybrid SWA + MoE Multimodal LLM Serving
We present a full-pipeline inference optimization for the MiMo-V2.5 model family, which combines Hybrid Sliding Window Attention (Hybrid SWA), sparse Mixture-of-Experts (MoE), and multimodal…
Read the full articleYou might also wanna read
MiMo v2.5 Hybrid SWA Cuts Model Size 60% While Retaining 98% Accuracy
MiMo v2.5 introduces a hybrid inference optimization technique that combines Stochastic Weight Averaging with dynamic pruning and quantizati
RTP-LLM: Alibaba's High-Performance Inference Engine for Large Language Model Deployment
Large Language Models (LLMs) have revolutionized AI applications, but deploying them at scale presents significant challenges. We present RT
arxiv.org·1mo ago
Building high-performance expert-parallel dispatch and combine kernels for MoE LLM inference
How expert-parallel dispatch and combine kernels work, built up from scratch: the high-throughput shape and the low-latency one.
fergusfinn.com·1mo ago
DigitalOcean Gradient™ AI GPU Droplets Optimized for Inference: Increasing Throughput at Lower the Cost
Production-grade LLM inference demands more than just access to GPUs; it requires deep optimization across the entire serving stack, from qu
Xiaomi MiMo-V2.5-Pro-UltraSpeed achieves 1000+ tokens/s on 1T-parameter model
MiMo, in collaboration with TileRT, releases the UltraSpeed mode of Xiaomi MiMo-V2.5-Pro — breaking 1000 tokens/s generation speed on a 1T-p
mimo.xiaomi.com·1mo ago
Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices
arXiv:2607.08786v1 Announce Type: new Abstract: With the growing deployment of large language models (LLMs), LLM inference cost has become a
Comments
Sign in to join the conversation.
No comments yet. Be the first.