Ai Inference RSS Feeds

Publishers

0 ai inference feeds · 6 articles

Ai Inference articles

Fireworks AI Achieves 1.9–2.4× Speedup for MiniMax M3 Sparse Attention on NVIDIA Blackwell GPUs

Fireworks built a KV-stationary sparse-attention kernel for MiniMax M3 on NVIDIA Blackwell (SM100), reaching ~980 TFLOP/s: 1.9–2.4× a query-stationary baseline and ~1.6× open-source MSA. The post walks through the Q-outer vs KV-outer design space, an I/O

Insighttechnologyprogrammingai inference

fireworks.ai·@MiniMax_AI·3mo ago·13 min read

Why Unified Memory Lets Mini PCs Run Large AI Models That GPUs Can't Fit

Insighttechnologyprogramming

vettedconsumer.com3mo ago

DFlash Speculative Decoding Boosts NVIDIA Blackwell Inference Performance Up to 15x

Insighttechnologyprogramming

Shared by @PyTorch ↗

developer.nvidia.com3mo ago

How to Run Multi-Token Prediction Models: A Guide to Faster Inference with Gemma 4 and Qwen3.6

technologymachine learning

Shared by @foobar.eagleusb.com ↗

unsloth.ai4mo ago

Kog AI Launches Inference Engine Tech Preview: 3,000 Tokens/s on AMD MI300X GPUs

Today, Kog AI launches a tech preview of the Kog Inference Engine (KIE): 3,000 output tokens/s per request on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). This preview runs a 2B model, with support for large third-party

technologyprogrammingai inference

blog.kog.ai·Hacker News: Front Page·5mo ago·18 min read

ATLAS: Adaptive Learning System for Faster LLM Inference Without Manual Tuning

LLM inference that gets faster as you use it. Our runtime-learning accelerator adapts continuously to your workload, delivering 500 TPS on DeepSeek-V3.1, a 4x speedup over baseline performance without manual tuning.

Newstechnologyprogrammingai inference

together.ai·Hacker News: Front Page·1y ago·10 min read