Moonshot AI PerceptionBench Exposes a Vision Gap Behind Multimodal Model Scores
Moonshot AI released PerceptionBench with 3,000 questions, and every tested multimodal model scored below 60 percent. The Moonshot AI PerceptionBench visual perception benchmark targets a basic…
Read the full articleYou might also wanna read
DatBench: A New Framework for More Faithful and Efficient Vision-Language Model Evaluation
Empirical evaluation serves as the primary compass guiding research progress in foundation models. Despite a large body of work focused on t
arxiv.org·6mo ago
BabyVision Benchmark Reveals MLLMs Fail at Basic Visual Tasks That 3-Year-Olds Can Solve
While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguisti
arxiv.org·24d ago
BilliardPhys-Bench: New Benchmark Reveals Physical Reasoning Gaps in Multimodal AI Models
Current multimodal models handle static image recognition well, but intuitive physical reasoning remains a weakness. Predicting how objects
arxiv.org·1mo ago
ClinHallu: A New Benchmark for Diagnosing Hallucination Sources in Medical AI Reasoning
Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support. Existing medical h
arxiv.org·1mo agoDeveloper Questions Whether Multi-Model AI Systems Can Truly Reduce Hallucinations
A developer building a multi-expert AI system — which routes user queries to several specialized models and aggregates their outputs — has r
Visual Iconicity Challenge: A New Benchmark for Evaluating Vision-Language Models on Sign Language Understanding
Onur Keleş, Asli Ozyurek, Gerardo Ortega, Kadir Gökgöz, Esam Ghaleb. Proceedings of the 64th Annual Meeting of the Association for Computati
doi.org·10d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.