FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling

TileMaxSim: IO-Aware GPU Kernels Achieve 80% HBM Bandwidth for Multi-Vector Retrieval Scoring

This paper presents TileMaxSim, a family of IO-aware GPU kernels for accelerating MaxSim scoring in multi-vector retrieval models like ColBE

Modular: Software Pipelining for GPU Kernels: Part 1 - The Pipeline Problem

Research Directions for Overcoming Memory and Interconnect Challenges in Large Language Model Inference Hardware

This article discusses the technical challenges of Large Language Model (LLM) inference, highlighting how the autoregressive Decode phase ma

AI-Driven Approach for Portable GPU Kernels in High-Performance Computing

This academic paper from North Carolina State University researchers presents an approach to leveraging AI ecosystems for creating portable

Optimizing LLM Inference: A C++ Backend for VRAM-Aware Sequence Packing

A technical deep-dive into optimizing LLM inference performance by eliminating wasteful padding in sequence batching. The article introduces

APEX4: Platform-Dependent W4A4 LLM Inference via Intra-SM Compute Rebalancing

This paper presents APEX4, a system for efficient W4A4 (4-bit weights, 4-bit activations) LLM inference that addresses the bottleneck of gro