SWE-Marathon: Evaluating AI Coding Agents at Scale
By
StartupHub.ai
1d agoen
Source
StartupHub.aiSWE-Marathon: Evaluating AI Coding Agents at Scalestartuphub.aiRishi Desai from Abundant AI introduces SWE-Marathon, a benchmark evaluating AI coding agents on billion-token scale tasks, revealing current limitations and the need for robust verification.
You might also wanna read
Runner - Coding Agent
runnercode.com·9mo ago
Experiments in Scaling Autonomous Coding Agents at Cursor
Wilson Lin at Cursor conducted experiments with large-scale autonomous coding agents, running hundreds of concurrent agents on a single proj
simonwillison.net·5mo ago
DeepSWE: A New Long-Horizon Benchmark for Evaluating Frontier Coding Agents on Complex Engineering Tasks
DeepSWE is a new long-horizon software engineering benchmark designed to evaluate frontier coding agents on original, complex engineering ta
deepswe.datacurve.ai·1mo ago
Jasmine: A Scalable JAX-Based World Modeling Codebase for Efficient AI Training
Researchers introduce Jasmine, a high-performance JAX-based world modeling codebase designed to scale from single hosts to hundreds of accel
arxiv.org·7mo ago
The Challenge of Verifying Code Quality from AI Coding Agents
The article discusses the author's experience building AI coding agents that work autonomously while they sleep, and the resulting challenge
claudecodecamp.com·3mo ago
Alibaba's Tongyi DeepResearch: Open-Source AI Research Agent Matches OpenAI Performance
Alibaba's Tongyi DeepResearch is presented as the first fully open-source web agent that achieves performance comparable to OpenAI's DeepRes
tongyi-agent.github.io·8mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.