Newsletter Subscribe
Enter your email address below and subscribe to our newsletter
All voices matter
benchmarking
OpenAI retracts SWE-Bench Pro recommendation after audit finds 30% of tasks broken
OpenAI announced on Tuesday that it is retracting its recommendation of SWE-Bench Pro as a reliable measure of AI coding capability, citing an internal audit that found roughly 30% of the benchmark's tasks are broken and a noise ceiling of…