Appears on
Articles4
Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic ex
Code
0
github.com2mo ago
Open-Source Compendium for Mathematics, Computer Science, and AI Fundamentals
Code
0
github.com5mo agoH
Cactus: An Open-Source Low-Latency AI Engine for Mobile Devices and Wearables
Code
0
github.com10mo agoComparison of Cactus and Ollama Frameworks for Smartphones
> However, both are platform-specific and only support specific models from the company
News
0
news.ycombinator.com1y ago