Rotary GPU: Enabling Large Mixture-of-Experts Models on Consumer Laptop GPUs with Limited Memory
Large language models have achieved remarkable capabilities through scaling, and this paper does not challenge that. It instead investigates a different question: once large models already exist, can…
Read the full articleYou might also wanna read
Researchers Serve 229B-Parameter MoE Model Across Five Consumer GPUs Over Public Internet
We serve MiniMax-M2.5, a 229B-parameter mixture-of-experts model, split across five consumer RTX 5090s in five European countries. The stage
doi.org·14d ago
Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices
arXiv:2607.08786v1 Announce Type: new Abstract: With the growing deployment of large language models (LLMs), LLM inference cost has become a
Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM for Efficient Interactive Deployment
We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the mo
arxiv.org·10d agoNemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM for Efficient Interactive Deployment
We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the mo
arxiv.org·10d agoLong-Context Fine-Tuning with Limited VRAM
arXiv:2607.15105v1 Announce Type: new Abstract: Parameter-efficient fine-tuning reduces model and optimizer memory, but dense attention stil
Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
arXiv:2607.14541v1 Announce Type: new Abstract: Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources th
Guide to Calculating GPU Memory for Self-Hosted LLM Inference
Calculate GPU memory requirements and max concurrent requests for self-hosted LLM inference. Support for Llama, Qwen, DeepSeek, Mistral and
Product Hunt·11mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.