TRACE: A New System for Capability-Targeted Agentic Training via Automated Failure Analysis
By
[Submitted on 7 Apr 2026 (v1), last revised 2 Jul 2026 (this version, v2)]
Summary
TRACE (Turning Recurrent Agent failures into Capability-targeted training Environments) is an end-to-end system for improving AI agent performance by automatically identifying missing capabilities from failed trajectories, synthesizing targeted training environments, and training LoRA adapters via reinforcement learning. The system uses a mixture-of-experts approach over capability adapters and demonstrates significant improvements on benchmarks: +15.3 points on τ²-Bench (customer-service) and +15.0 points on SWE-Bench Verified (software engineering), outperforming baselines like GEPA and SWE-RL while being more sample-efficient.
Source
bskyTRACE: A New System for Capability-Targeted Agentic Training via Automated Failure Analysisarxiv.orgKey quotes
· 5 pulledModels often fail to complete agentic tasks because they lack core capabilities required by the target environment.
TRACE contrasts successful and failed trajectories to automatically identify missing capabilities, synthesizes a targeted training environment for each capability that rewards whether the capability is exercised.
TRACE can be effectively applied across different environments, improving over the base agent by +15.3 points on τ²-Bench, a customer-service agent benchmark, and by +15.0 points Pass@1 on SWE-Bench Verified, a software-engineering benchmark.
TRACE outperforms the strongest external baselines, GEPA and SWE-RL, by +8.6 points and +8.4 points, respectively.
Using fewer than one-fourth the number of rollouts, TRACE outperforms the best-performing baselines, GRPO and GEPA, and achieves higher final accuracy by +10.4 and +8.6 points on τ²-Bench.
You might also wanna read
Tracea: Self-hosted monitoring and debugging tool for AI agents with automatic RCA and team memory
Tracea is a self-hosted monitoring and debugging tool for AI agents, similar to Datadog but purpose-built for agent workflows. It captures a
Product Hunt·2mo agoTracing module
Trace-AI: Security Tool for Predicting and Preventing Supply-Chain Attacks in Open-Source Dependencies
Trace-AI is a security tool that predicts and prevents supply-chain attacks by analyzing open-source dependencies, registries, and maintaine
Product Hunt·8mo ago
Retrace: A debugging tool for AI agents that replays and forks executions to identify regressions
Retrace is a debugging tool for AI agents that allows users to record, replay, fork, and share agent executions. It captures every LLM call,
Product Hunt·6d ago
Trails: Automated Analysis Tool for Debugging LLM Agent Performance
Trails is a tool that automatically analyzes LLM (Large Language Model) agent execution traces to surface insights and identify issues. It h
Product Hunt·5mo ago
Trace Launches as a Workflow Automation Platform Routing Tasks Between Humans and AI Agents
Trace (YC S25) is a workflow automation platform that routes tasks to the appropriate agent—whether human or AI. By integrating tools like S
Product Hunt·11mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.