AMD Strix Halo RDMA Cluster Setup Guide for Distributed vLLM Inference
Contribute to kyuz0/amd-strix-halo-vllm-toolboxes development by creating an account on GitHub.
Read the full articleYou might also wanna read
Beyond Porting: How vLLM Orchestrates High-Performance Inference on AMD ROCm
How vLLM orchestrates high-performance inference on AMD ROCm with multiple attention backends, workload-aware prefill, extend, and decode ro
Mesh LLM: Distributed AI Computing on iroh
Mesh LLM surfaced on Hacker News with 179 points and 41 comments , presenting a system for running large language model inference across a p
Setting up the AMD Strix Halo (Ryzen AI Max+ 395) mini-PC for local AI development
Setting up and getting up to speed with an AMD Strix Halo (Ryzen AI Max+ 395) mini-PC for local AI development.
sayak.dev·6d agoBuilding Mixture-of-Models on AMD GPUs with vLLM-SR
How vLLM Semantic Router builds a Mixture-of-Models system on AMD MI300X and MI355X GPUs, routing across specialized models with signals, de
Strix Halo Local AI Workstation Real World Performance Test
I tested the AMD Ryzen AI Max+ 395 (Strix Halo) against Mac Studio and a 4090 box on real local LLM workloads. Honest results, ROCm gaps inc
Guide to Calculating GPU Memory for Self-Hosted LLM Inference
Calculate GPU memory requirements and max concurrent requests for self-hosted LLM inference. Support for Llama, Qwen, DeepSeek, Mistral and
Product Hunt·11mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.