OpenAI retracts SWE-Bench Pro: ~30% of tasks broken, audit finds
OpenAI's own datapoint + 5-engineer audit flags 27–34% of SWE-Bench Pro tasks as broken. It just retracted its own recommendation to use the benchmark.
LifeSciBench: GPT-Rosalind 36.1%, artifact gap 17pts
OpenAI published LifeSciBench, a 750-task life-sciences evaluation. GPT-Rosalind hits 36.1% vs GPT-5.5's 25.7%, but drops 17 points on tasks with artifacts.