GPT-5.6 Built a Tiny iMessage Transformer From One Prompt
Pietro Schirano says Sol generated the code, pipeline, tokenizer, training loop and local reply model in about 30 minutes.
Read the full articleYou might also wanna read
MicroGPT: A Minimalist 200-Line Python Implementation of GPT Architecture
Musings of a Computer Scientist.
Andrej Karpathy Releases nanochat: Full ChatGPT-Style LLM That Can Be Trained for $100
Really interesting new project from Andrej Karpathy, described at length in this discussion post. It provides a full ChatGPT-style LLM, incl
Google's DiffusionGemma achieves 4x faster text generation using diffusion-based approach
An overview of DiffusionGemma, an exceptionally fast text generation model with up to 4x faster speeds.
deepmind.google·25d ago
Research Proves Transformer Language Models Are Injective and Invertible
Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could
arxiv.org·8mo ago
nanochat: Andrej Karpathy's Minimalist ChatGPT Implementation for Educational Development
nanochat is a full-stack implementation of an LLM like ChatGPT in a single, clean, minimal, hackable, dependency-lite codebase. Run tokeniza
Product Hunt·8mo ago
Reverse-Engineering Transformer Attention Heads Using Program Synthesis
A longstanding goal of research on interpretable deep learning is to replace opaque neural computations with human-meaningful symbolic descr
arxiv.org·9d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.