The hidden variables in your agent eval
By
Waldek Mastykarz
Source
MicrosoftThe hidden variables in your agent evalmicrosoft.comYou might also wanna read
The Essential Role of Manual Data Review in AI Agent Evaluation
The article discusses the importance of evaluating AI agents, emphasizing that while automated evaluations (evals) are essential for testing
agent-skills-eval: An open-source test framework for measuring AI agent skill effectiveness
agent-skills-eval is an open-source test runner for evaluating AI agent skills (SKILL.md files) based on the Agent Skills standard from Anth
Research Study: How Experienced Software Developers Use AI Agents in Development Workflows
This research paper investigates how experienced software developers use AI agents in their workflow. Through field observations and surveys
What People Still Get Wrong About AI Agents: Trust, Tools, and Evaluation at Scale
The article discusses persistent misunderstandings about AI agents and tools, particularly in the context of NVIDIA's open-sourcing of their
A Developer's Year-Long Research on AI Agents for Security-Critical Software Development
A software developer and protocol maintainer shares findings from over a year of research into using AI agents for writing high-quality soft
Agent Skills: Making AI Coding Agents Follow Software Engineering Best Practices
The article discusses how AI coding agents default to taking the shortest path to "done," skipping essential software engineering practices

Comments
Sign in to join the conversation.
No comments yet. Be the first.