Sapient Intelligence Releases HRM-Text-1B: A 1B Parameter Language Model with Hierarchical Reasoning Architecture
Pure flour-power. Hearty enough to carry you through lunch.
Summary
Sapient Intelligence has released HRM-Text-1B, a 1 billion parameter language model built on the Hierarchical Reasoning Model (HRM) architecture. HRM uses a dual-timescale recurrent design with two Transformer modules (high-level/slow and low-level/fast) that iterate over input embeddings for multiple cycles, enabling effectively unbounded compute depth at a bounded parameter count. The model was trained from scratch on structured public datasets and is released as a pre-alignment checkpoint (not a chat or instruction-tuned model). The release is part of an effort to advance and democratize AI through open source and open science.
Key quotes
· 4 pulledHRM is a dual-timescale recurrent architecture: two Transformer modules (H = high-level / slow, L = low-level / fast) iterate over the same input embeddings for H_cycles × (L_cycles + 1) steps, with additive state injection (z_L + z_H).
This gives effectively unbounded compute depth at bounded parameter count.
This is a pre-alignment model checkpoint, not a chat or instruction-
We're on a journey to advance and democratize artificial intelligence through open source and open science.
You might also wanna read
Introducing the Hierarchical Reasoning Model: A Breakthrough in AI Reasoning
The article introduces the Hierarchical Reasoning Model (HRM) as a novel recurrent architecture inspired by the human brain's hierarchical a
arxiv.org·10mo ago
Revolutionary 27M-Parameter AI Model Enhances Sequential Reasoning and Planning
The article introduces a revolutionary 27M-parameter AI model called the Hierarchical Reasoning Model, which performs complex sequential rea
Product Hunt·10mo ago
Falcon-H1: Hybrid-Head Language Models for Efficient and High-Performance AI
The article introduces Falcon-H1, a new series of large language models (LLMs) featuring a hybrid architecture that combines Transformer-bas
arxiv.org·10mo ago
Tiny Recursive Model Outperforms Large Language Models on Complex Reasoning Tasks
Researchers propose Tiny Recursive Model (TRM), a simplified recursive reasoning approach that outperforms both the existing Hierarchical Re
arxiv.org·7mo ago
Sarvam AI Open-Sources 30B and 105B Reasoning Models Trained in India
Sarvam AI is open-sourcing two large language models - Sarvam 30B and Sarvam 105B - which are reasoning models trained entirely in India usi
sarvam.ai·2mo ago
Introducing MiniMax-M1: The World's First Open-Weight Hybrid-Attention Reasoning Model
Introducing MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model powered by a hybrid Mixture-of-Experts a
github.com·11mo ago