SWE Bench Verified Evaluation Loopholes Allow Agents to Access Future Repository State
We've identified multiple loopholes with SWE Bench Verified where agents may look at future repository state (by querying it directly or through a variety of methods), and cases in which future rep...
Read the full article
Comments
Sign in to join the conversation.
No comments yet. Be the first.