PyTorch Monarch Distributed Training Now Available on AMD GPUs via ROCm
Training state-of-the-art large language models (LLMs) with billions of parameters requires distributed training across hundreds or thousands of GPUs. At this scale, hardware failures are not…
Read the full articleYou might also wanna read
TileMaxSim: IO-Aware GPU Kernels Achieve 80% HBM Bandwidth for Multi-Vector Retrieval Scoring
arxiv.org·15d ago
The Critical Role of GPU Kernel Quality in Machine Learning System Performance
mlc.ai·17d agoThe Critical Role of GPU Kernel Quality in Machine Learning System Performance
mlc.ai·17d ago
APEX4: Platform-Dependent W4A4 LLM Inference via Intra-SM Compute Rebalancing
arxiv.org·1mo ago
Rotary GPU: Enabling Large Mixture-of-Experts Models on Consumer Laptop GPUs with Limited Memory
arxiv.org·1mo ago
Hardware-Aware Dynamic Speculative Decoding: Optimizing LLM Inference Through Adaptive Draft Token Selection
cohere.com·8h ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.