Verification Becomes the Harder Problem for Advanced Coding Agents

Summary

This paper challenges the classical intuition that verifying a solution is easier than producing one, arguing that for modern coding agents, verification has become the harder problem. The authors characterize verification signal quality along three dimensions—scalability, faithfulness, and robustness—and study four reward constructions (test verifier, rubric verifier, user as verifier, automated agent verifier) across different task types. Experiments show that targeted verification design can suppress reward hacking and improve task completion, but crucially, no fixed reward function remains effective as policy capability grows—verification must co-evolve with the generator.

Source

bskyVerification Becomes the Harder Problem for Advanced Coding Agentsarxiv.org