Modular: Modular Platform 25.5: Introducing Large Scale Batch Inference

Modular

Unified Inference Platform for building and scaling AI systems.

MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU

Authors: Zhengqing Yuan, Hanchi Sun, Lichao Sun, Yanfang Ye

The Promise of Distributed AI Inference

Exploring how EXO enables combining multiple devices into a unified computing cluster for accelerated AI inference.

ACM Transactions on Probabilistic Machine Learning (topml) Vol 2, Issue 2

Tokasaurus: An LLM Inference Engine for High-Throughput Workloads

Article URL: Comments URL: Points: 14 # Comments: 0

Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets

How Together served MiniMax-M3 efficiently with KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based