Experimenting with AI-Powered Research Automation: Applying Karpathy's Autoresearch to Legacy eCLIP Code
By
ykumards
Pulled from the oven just right. Trustworthy, fact-dense, deeply satisfying.
Summary
The author describes experimenting with Andrej Karpathy's Autoresearch framework by applying it to their old eCLIP research code. They set up an LLM agent (Claude Code) to iteratively optimize a training script while they performed household chores. The article documents the process of reviving legacy code, implementing the Autoresearch loop, and observing the AI agent's attempts to improve model performance through automated code modifications. The experiment serves as a practical test of AI-assisted research automation on a familiar problem.
Key quotes
· 5 pulledAutoresearch is a simple constrained optimization loop with an LLM agent in the middle.
I picked up my old research code from eCLIP, dusted it off the legacy dependencies and gave it to Claude Code.
The agent iteratively improves some eval metric by modifying a single file (train.py), while reading instructions and constraints.
And just let it cook while I did some chores around the house.
This is my journey of applying Autoresearch to a problem I fully understood.
You might also wanna read
How I Used Coding Agents to Automate My AI Research Work in Copilot Applied Science
An AI researcher shares their experience using coding agents to automate intellectual work, specifically building agents that automate parts
GitHub·2mo ago
Agentipedia: Collaborative AI Agent Platform for Crowdsourced Experimentation and Model Development
Agentipedia is a platform that enables AI agents to collaboratively conduct experiments and share results, building on Andrej Karpathy's Aut
Product Hunt·2mo ago
How AI coding agents are reshaping social science research: Opportunities and concerns
This article examines how AI coding agents are transforming social science research by automating core research tasks traditionally performe
share.google·4d ago
Building a Software Factory with Claude Code: From AI-Assisted Coding to Agentic Development
This article provides a comprehensive guide on building a software factory using Claude Code and other AI coding tools. It covers the evolut
freecodecamp.org·4d ago
Practical Guide to Using AI Coding Tools for Responsible Development
The article provides practical guidance for developers on responsibly integrating AI coding tools into their workflow. Based on two years of
Smashing Magazine·4mo ago
Claude Code Launches Multi-Agent AI Code Review System for Bug Detection
Anthropic's Claude Code now offers a multi-agent AI code review system that analyzes pull requests to catch bugs, security issues, and logic
Product Hunt·3mo ago