benchmark

3 articles

← All topics

Mistral opens a public preview of Mistral Large 4, a 1.05T-parameter open-weight model trained in Europe

mistralmistral-large-4open-weightsmoeeurope+5+6

Mistral puts Mistral Large 4 ("le Chonk") into public preview: a 1.05T-parameter open-weight MoE served from EU datacenters, with weights promised by month's end.

OpenAI retracts SWE-Bench Pro: ~30% of tasks broken, audit finds

openaiopenai-blogbenchmarkevaluationai-coding+8+9

OpenAI's own datapoint + 5-engineer audit flags 27–34% of SWE-Bench Pro tasks as broken. It just retracted its own recommendation to use the benchmark.

LifeSciBench: GPT-Rosalind 36.1%, artifact gap 17pts

openailifesci-benchlifescibenchlife-sciencesbenchmark+13+14

OpenAI published LifeSciBench, a 750-task life-sciences evaluation. GPT-Rosalind hits 36.1% vs GPT-5.5's 25.7%, but drops 17 points on tasks with artifacts.