EvoTrainer: A Framework for Co-Evolving LLM Policies and Training Harnesses in Autonomous Agentic Reinforcement Learning

Summary

The article introduces EvoTrainer, an autonomous training framework for LLMs that goes beyond traditional recipe search by co-evolving both LLM policies and training-side harnesses. Unlike static training harnesses that struggle with agentic reinforcement learning (RL) where bottlenecks shift and scalar rewards hide failure modes, EvoTrainer uses empirical feedback to diagnose rollout-level evidence, revise diagnostics, backtest interventions, and accumulate reusable skills. Evaluated on mathematical reasoning, competitive programming code generation, and repository-level software engineering, EvoTrainer matches or exceeds human-engineered RL baselines under identical conditions, with the largest gains in long-horizon agentic software engineering tasks. The paper argues that autonomous LLM RL should evolve both policies and the training harnesses that interpret them.