Andrej Karpathy Releases nanochat: Full ChatGPT-Style LLM That Can Be Trained for $100
By
bilsbie
If you only eat one bagel today, this is the bagel.
Summary
Andrej Karpathy has released nanochat, a complete ChatGPT-style large language model implementation that can be trained for as little as $100. The project is a full-stack LLM implementation in a single, clean codebase of about 8,000 lines, mostly in Python with PyTorch and some Rust for tokenizer training. It includes training, inference, and a web UI, with Karpathy suggesting renting an 8XH100 NVIDIA node for approximately $24/hour, making 4 hours of training sufficient for the model.
Key quotes
· 3 pulledThis repo is a full-stack implementation of an LLM like ChatGPT in a single, clean, minimal, hackable, dependency-lite codebase.
It's around 8,000 lines of code, mostly Python (using PyTorch) plus a little bit of Rust for training the tokenizer.
Andrej suggests renting a 8XH100 NVIDA node for around $24/ hour to train the model. 4 hours (~$100) is enough
You might also wanna read
nanochat: Andrej Karpathy's Minimalist ChatGPT Implementation for Educational Development
nanochat is Andrej Karpathy's educational project that implements a full-stack ChatGPT-like LLM in approximately 1000 lines of clean, hackab
Product Hunt·7mo ago
OpenAI's ChatGPT: Free AI Assistant for Answers, Creativity, and Learning
ChatGPT is a free-to-use large language model from OpenAI that provides instant answers, creative inspiration, and learning assistance. It h
Product Hunt·26d ago
OpenAI Launches $100/month ChatGPT Pro Tier with Enhanced Codex Access
OpenAI has launched a new ChatGPT Pro subscription tier priced at $100 per month, offering 5x more usage of its Codex coding tool compared t
The Verge·1mo ago