TRiP: An open-source Transformer AI engine built from scratch in C for educational purposes
By
carlovalenti
Crisp on the outside, thoughtful on the inside. A keeper.
Summary
TRiP (TRansformer in Progress) is an open-source, from-scratch implementation of a Transformer AI engine written entirely in C. Built over 18 months during the developer's personal time, it supports inference, training, tokenizer creation, chat, and vision for models like Gemma 1, Llama 2, PaliGemma, and GPT-2. The project is purely educational, aimed at helping the developer and others understand transformer internals from the ground up.
Key quotes
· 4 pulledTRiP exists just because I wanted to truly understand the transformer internals - from the matrix multiplications up.
TRiP's purpose is purely educational, for me and for anyone willing to learn about transformers.
A few-files, all-in-one C engine for Transformer AI models: inference, training, tokenizer creation, chat, and vision.
Built from scratch over 18 months (from March 2024 to August 2025) during my lunch breaks and weekend nights.
You might also wanna read
Build Your Own LLM From Scratch: A Hands-On GPT Training Workshop
A hands-on workshop and GitHub repository that guides users through building their own GPT training pipeline from scratch, inspired by Andre
github.com·26d ago
Complete Educational Implementations of Ilya Sutskever's 30 Foundational Deep Learning Papers
This repository provides comprehensive educational implementations of the 30 foundational deep learning papers recommended by Ilya Sutskever
github.com·4mo ago
Tutorial: Training a Neural Network to Play Tic-Tac-Toe with Reinforcement Learning in Jax
This article provides a tutorial on training a neural network to play Tic-Tac-Toe using reinforcement learning with Jax. The content is peda
joe-antognini.github.io·4mo ago
Andrej Karpathy's Course: Building Neural Networks from Scratch to GPT
Andrej Karpathy offers a course teaching how to build neural networks from scratch in code, starting with backpropagation basics and progres
Building a Deep Learning Library from Scratch with NumPy
This article introduces a project to build a simple deep learning library from scratch using only NumPy, starting with a blank file and prog
The Importance of Understanding Backpropagation in Deep Learning Education
The article argues for the importance of understanding backpropagation in deep learning, even though modern frameworks automate it. The auth
karpathy.medium.com·7mo ago