2d agoen
You might also wanna read
ntransformer: C++/CUDA LLM Inference Engine Enables Running Llama 70B on RTX 3090
High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090. - xaskasdf/ntransformer
github.com·4mo ago
Modular: Evaluating Llama Guard with MAX 24.6 and Hugging Face
Evaluating Llama Guard with MAX 24.6 and Hugging Face
modular.com·1y ago
Developer flags harness bug after local 35B model passes task but fails eval on stop token
A developer running agentic evaluations on a local 35-billion-parameter quantized model (ornith-1.0-35b-Q8_0) via llama.cpp found a mismatch
ShortSingh·3d ago
Parakeet.cpp: Fast C++ Implementation of NVIDIA's Speech Recognition Models for On-Device Inference
Ultra fast and portable Parakeet implementation for on-device inference in C++ using Axiom with MPS+Unified Memory - Frikallo/parakeet.cpp
github.com·4mo agoHow to Compile and Cross-Compile llama.cpp for ARM64 on Debian 12/13
llama.cpp is a lightweight C/C++ inference runtime for running large language models locally using the GGUF format on x86_64 servers and ARM
ShortSingh·2d ago
Nibble: A C-like systems programming language that generates LLVM IR without malloc or external dependencies
Generating LLVM IR without malloc or external dependencies - glouw/nibble
github.com·2mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.