Fireworks AI Achieves 1.9–2.4× Speedup for MiniMax M3 Sparse Attention on NVIDIA Blackwell GPUs
Fireworks built a KV-stationary sparse-attention kernel for MiniMax M3 on NVIDIA Blackwell (SM100), reaching ~980 TFLOP/s: 1.9–2.4× a query-stationary baseline and ~1.6× open-source MSA. The post…
Read the full articleYou might also wanna read
TileMaxSim: IO-Aware GPU Kernels Achieve 80% HBM Bandwidth for Multi-Vector Retrieval Scoring
Multi-vector retrieval models such as ColBERT achieve state-of-the-art accuracy through fine-grained token-level MaxSim scoring, yet existin
arxiv.org·15d agoServing MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets
How Together served MiniMax-M3 efficiently with KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based
Building high-performance expert-parallel dispatch and combine kernels for MoE LLM inference
How expert-parallel dispatch and combine kernels work, built up from scratch: the high-throughput shape and the low-latency one.
fergusfinn.com·1mo ago
MiniCPM 4.0: Ultra-Efficient Open-Source AI Models for On-Device Deployment
MiniCPM 4.0 is a family of ultra-efficient, open-source models for on-device AI. Offers significant speed-ups on edge chips, strong performa
Product Hunt·10mo ago
Rotary GPU: Enabling Large Mixture-of-Experts Models on Consumer Laptop GPUs with Limited Memory
Large language models have achieved remarkable capabilities through scaling, and this paper does not challenge that. It instead investigates
arxiv.org·1mo ago
APEX4: Platform-Dependent W4A4 LLM Inference via Intra-SM Compute Rebalancing
W4A4 quantization promises full utilization of INT4 Tensor Cores, yet group dequantization overhead on CUDA Cores has driven existing system
arxiv.org·1mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.