It Takes a MAESTRO To Prune Bad Experts
arXiv:2607.08601v1 Announce Type: new Abstract: Sparsely-activated Mixture-of-Experts (MoE) language models achieve remarkable inference efficiency by activating only a small fraction of parameters…
Read the full articleYou might also wanna read
Self-Routing: A Parameter-Free Approach to Expert Routing in Mixture-of-Experts Models
Mixture-of-Experts (MoE) layers increase model capacity by activating only a small subset of experts per token, and typically rely on a lear
arxiv.org·1d ago
Building high-performance expert-parallel dispatch and combine kernels for MoE LLM inference
How expert-parallel dispatch and combine kernels work, built up from scratch: the high-throughput shape and the low-latency one.
fergusfinn.com·29d ago
Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM for Efficient Interactive Deployment
We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the mo
arxiv.org·2d agoNemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM for Efficient Interactive Deployment
We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the mo
arxiv.org·2d agoPangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
The surgence of Mixture of Experts (MoE) in Large Language Models promises a small price of execution cost for a much larger model parameter
arxiv.org·1y ago
Geometric and Stochastic Analysis of Discontinuities in Sparse Mixture-of-Experts Architectures
Sparse Mixture-of-Experts (SMoE) architectures are now widely deployed in state-of-the-art language and vision models, where conditional rou
arxiv.org·20d agoJavelinGuard: Low-Cost Transformer Architectures for LLM Security
We present JavelinGuard, a suite of low-cost, high-performance model architectures designed for detecting malicious intent in Large Language
Comments
Sign in to join the conversation.
No comments yet. Be the first.