ZML's Steeve Morin releases free LLMD to loosen AI inference from Nvidia
The Paris startup's closed-source inference server runs Llama, Gemma, Qwen and Mistral models across Nvidia, AMD, TPU, Intel and Apple backends.
Read the full articleYou might also wanna read
Hot French startup ZML releases free product to speed inference across lots of AI chips
ZML, a hot French AI startup endorsed by Turing Award winner Yann LeCun, has now released ZML/LLMD, software that could make running AI less
LlamaFactory: Open-Source Framework for Efficient Fine-Tuning of 100+ LLMs and VLMs
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024) - hiyouga/LlamaFactory
github.com·9mo ago
Guide to Calculating GPU Memory for Self-Hosted LLM Inference
Calculate GPU memory requirements and max concurrent requests for self-hosted LLM inference. Support for Llama, Qwen, DeepSeek, Mistral and
Product Hunt·11mo ago
Mesh-LLM: Distributed LLM Inference System Using llama.cpp Across Multiple Machines
reference impl with llama.cpp compiled to distributed inference across machines, with real end to end demo - michaelneale/mesh-llm
github.com·3mo ago
Building a Distributed LLM Inference Cluster with AMD Ryzen AI Max+ Systems
Step-by-step guide to clustering AMD Ryzen™ AI Max+ systems for local one trillion-parameter LLM inference using llama.cpp RPC and ROCm.
amd.com·4mo ago
ZSE: Ultra Memory-Efficient LLM Inference Engine for Running Large Models on Consumer GPUs
The inference engine the open-source world built for itself. - Zyora-Dev/zse
github.com·4mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.