Case Study: Physicist-Supervised AI Development of Scientific Software Reveals Supervision Design Gaps
By
[Submitted on 28 May 2026]
Summary
A quantified case study (N=1) examining a physicist supervising an AI coding agent (Claude Code) over 12 work days and 57 sessions to build CLAX-PT, a differentiable perturbation theory module in JAX. The study documented 15 supervision events, finding that the AI agent resolved 10 autonomously via oracle tests, 2 with physicist domain knowledge, and failed on 3 that evaded oracle detection. The critical failures involved the agent treating symptom reduction as root-cause resolution, spending 33 sessions adjusting coefficients within a flawed architecture, and committing a calibrated "fudge factor" that passed all tests but predicted wrong values. The study concludes that supervision design, not model capability, determined trustworthiness, and that current AI agents lack the ability to propose architectural alternatives or distinguish predictive adequacy from explanatory correctness.
Source
bskyCase Study: Physicist-Supervised AI Development of Scientific Software Reveals Supervision Design Gapsarxiv.orgKey quotes
· 5 pulledThe agent treated symptom reduction as root-cause resolution.
The fudge factor was caught and replaced within the same session.
Supervision design, not model capability, determined whether the agent's output was trustworthy.
Closing the gap would require agents that propose architectural alternatives rather than optimize within a given structure, and distinguish predictive adequacy from explanatory correctness.
Three supervision practices proved critical for catching what oracle tests missed: testing at diverse parameter points beyond the fiducial calibration; shared changelogs that surfaced stalled exploration across sessions; and an explicit rule against unphysical numerical patches.
You might also wanna read
The Challenge of Verifying Code Quality from AI Coding Agents
The article discusses the author's experience building AI coding agents that work autonomously while they sleep, and the resulting challenge
claudecodecamp.com·3mo ago
Experimenting with AI-Powered Research Automation: Applying Karpathy's Autoresearch to Legacy eCLIP Code
The author describes experimenting with Andrej Karpathy's Autoresearch framework by applying it to their old eCLIP research code. They set u
ykumar.me·3mo ago
Building a Programming Language with Claude Code: A Case Study in AI-Assisted Development
The author details their experience building a new programming language called Cutlet over four weeks using Claude Code, an AI programming a
ankursethi.com·3mo ago
Jasmine: A Scalable JAX-Based World Modeling Codebase for Efficient AI Training
Researchers introduce Jasmine, a high-performance JAX-based world modeling codebase designed to scale from single hosts to hundreds of accel
arxiv.org·7mo ago
Critique of AI-Generated Code and the Problem of 'Vibe-Coding' in Software Development
The article critiques the problematic use of AI tools like LLMs in software development, particularly focusing on 'vibe-coding' where develo
acko.net·4mo ago
Practical Guide to Using Claude Code: Personal Experience and Professional Insights
The article is a comprehensive personal account of the author's extensive experience using Claude Code, an AI coding assistant. The author d
blog.sshh.io·8mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.