benchmarking

OpenAI releases benchmark to grade AI mental health responses

OpenAI on Wednesday released MentalHealthBench, an open benchmark of 1,215 synthetic mental health conversations built with more than 80 licensed mental health professionals to measure how AI models respond across a range of scenarios from everyday stress to urgent crises.

GPT-6 Astra draws split verdicts from independent labs

OpenAI's GPT-6 Astra, released on September 3, 2026, has produced a rare split among independent evaluators: one ranks it as the top AI model in the world, while another places it level with its predecessor and behind rival Anthropic.

Nvidia agent scores 100% on ARC-AGI-3 reasoning benchmark

Nvidia on Friday published research showing its general-purpose agent system, Agentic Variation Operators (AVO), achieved a perfect 100% score on the ARC-AGI-3 interactive reasoning benchmark — a result that underscores how agent architecture, not just model capability, determines performance on…

OpenAI retracts SWE-Bench Pro recommendation after audit finds 30% of tasks broken

OpenAI announced on Tuesday that it is retracting its recommendation of SWE-Bench Pro as a reliable measure of AI coding capability, citing an internal audit that found roughly 30% of the benchmark's tasks are broken and a noise ceiling of…