The Promise of Distributed AI Inference

Modular

General Compute Launches ASIC-Based Inference Cloud for Faster AI Agent Performance

General Compute is an inference cloud built on ASICs (purpose-built alternatives to Nvidia GPUs) designed specifically for AI inference, not

OmniRoute: Free AI Gateway With 231+ Providers and One Unified Endpoint

Building high-performance expert-parallel dispatch and combine kernels for MoE LLM inference

This article provides a deep technical deep-dive into the architecture and implementation of high-performance Expert Parallelism (EP) kernel

Arbor: A Multi-Agent Framework Using Tree Search for Autonomous LLM Inference Optimization

Arbor is a multi-agent framework that uses structured tree search as a cognition layer for autonomous agents operating in large, stateful ac

The Evolution of AI: From Static Benchmarks to Inference-Time Search for Autonomous Agents

The article explores the shift from traditional AI benchmarking to inference-time search as the future of AI development. It discusses how c