History-LLMs Project: Developing Large Language Models for Historical Analysis with Ranke-4B Release
By
iamwil
Pulled from the oven just right. Trustworthy, fact-dense, deeply satisfying.
Summary
The article describes the History-LLMs project, which focuses on training large language models specifically for historical analysis. The project is developing a family of 4 billion parameter models called Ranke-4B, based on the Qwen3 architecture, trained from scratch on 80 billion tokens of historical data with specific knowledge cutoffs (1913, 1929, 1933, 1939, 1946). The project uses a curated dataset of 600 billion tokens of time-stamped text and acknowledges support from Lambda AI research credits. The content serves as an information hub for the project's goal of training the largest possible historical LLMs.
Key quotes
· 4 pulledA family of 4 billion (B) parameter large language models (LLMs) based on the Qwen3 architecture trained from scratch on 80B tokens of historical data up to knowledge-cutoffs $\in {1913, 1929, 1933, 1939, 1946}$
Information hub for our project training the largest possible historical LLMs
We gratefully acknowledge research credits provided by Lambda AI
using a curated dataset of 600B tokens of time-stamped text
You might also wanna read
RTP-LLM: Alibaba's High-Performance Inference Engine for Large Language Model Deployment
This paper presents RTP-LLM, a high-performance inference engine developed by Alibaba for industrial-scale deployment of Large Language Mode
arxiv.org·1d ago
Parametric Memory Law: A Quantitative Framework for Understanding LoRA Memory Capacity in LLMs
This research paper introduces the Parametric Memory Law, a quantitative framework for understanding how Low-Rank Adaptation (LoRA) enables
arxiv.org·1d ago
Monostate: All-in-One AI Training Platform for Fine-Tuning LLMs
Monostate is an all-in-one AI training platform that enables users to fine-tune large language models (LLMs) with their own data using vario
Product Hunt·2mo ago
LLM Stats: Platform for Comparing AI Language Models by Benchmarks, Cost, and Capabilities
LLM Stats is a platform that allows users to compare various AI language models (LLMs) across multiple dimensions including performance benc
Product Hunt·7mo ago