ITBench-AA Benchmark Launched: Frontier AI Models Score Below 50% on Enterprise IT Tasks
A Blog post by IBM Research on Hugging Face
Read the full articleYou might also wanna read
EdgeBench: A Benchmark for Measuring AI Environment Learning Through Extended Real-World Tasks
EdgeBench studies how agents learn from real-world environments across 134 day-long executable tasks.
edge-bench.org·14d agoDevRev launches Enterprise-Bench for enterprise AI agents
Businesses face a tougher test for AI agents as DevRev's new benchmark measures accuracy, cost and access controls in enterprise settings.
DevRev launches Enterprise-Bench for enterprise AI agents
Businesses face a tougher test for AI agents as DevRev's new benchmark measures accuracy, cost and access controls in enterprise settings.
DeepSWE: A New Long-Horizon Benchmark for Evaluating Frontier Coding Agents on Complex Engineering Tasks
DeepSWE measures frontier coding agents on original, long-horizon software engineering tasks.
deepswe.datacurve.ai·1mo ago
SkillsBench: A Benchmark for Evaluating AI Agent Skills Across Diverse Tasks
Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time. Despite rapid adoption, there is no
arxiv.org·5mo ago
Benchmark Study: AI Models Struggle with OpenTelemetry Instrumentation for Distributed Tracing
A lot of vendors pitch AI SRE. We tested 14 models across 11 programming languages; even the best ones struggle with instrumenting code with
quesma.com·5mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.