Autodata: Using AI agents as data scientists to generate high-quality synthetic training data

Summary

This paper introduces Autodata, a method that uses AI agents as data scientists to create high-quality synthetic training and evaluation data. The approach involves training (meta-optimizing) a data scientist agent that learns to produce increasingly better data. The authors present a practical implementation called Agentic Self-Instruct and test it on computer science research, legal reasoning, and mathematical reasoning tasks, achieving improved results over classical synthetic data creation methods. Meta-optimizing the data scientist agent itself yields even larger performance gains. The paper argues that agentic data creation can convert increased inference compute into higher quality model training, potentially changing how AI training data is built.

Source

Twitter / XAutodata: Using AI agents as data scientists to generate high-quality synthetic training dataarxiv.org