M3Bench: A New Standard for Medical AI Model Editing
M3Bench introduces a clinically grounded benchmark for editing medical vision-language models. It challenges current methods, offering insights into their strengths and limitations.
Read the full articleYou might also wanna read
CompileBench: Testing AI Models on Real-World Software Engineering Challenges
We tested 19 LLMs on their ability to handle real-world software engineering tasks like compiling old code and cross-compiling. See how Anth
quesma.com·9mo ago
MedFact: Benchmarking the Fact-Checking Capabilities of Large Language Models on Chinese Medical Texts
Deploying Large Language Models (LLMs) in medical applications requires fact-checking capabilities to ensure patient safety and regulatory c
arxiv.org·1mo ago
ProgramBench: New Benchmark Reveals Language Models Struggle to Build Complete Software Projects From Scratch
Turning ideas into full software projects from scratch has become a popular use case for language models. Agents are being deployed to seed,
arXiv.org·2mo ago
ParseBench — Document Parsing Benchmark for AI Agents
The first benchmark that evaluates what AI agents actually need from document parsing. 2,000 human-verified pages, 169K deterministic test r
parsebench.ai·16d ago
SkillsBench: A Benchmark for Evaluating AI Agent Skills Across Diverse Tasks
Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time. Despite rapid adoption, there is no
arxiv.org·4mo ago
DatBench: A New Framework for More Faithful and Efficient Vision-Language Model Evaluation
Empirical evaluation serves as the primary compass guiding research progress in foundation models. Despite a large body of work focused on t
arxiv.org·6mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.