moe

7 articles

← All topics

Strata runs Qwen3.8-Flash-Next on a 12GB gaming PC

strataqwenqwen3-8-flash-nextmoelocal-inference+2+3

Niko1221/Strata streams a ~125B-parameter MoE from RAM and SSD into a 12GB consumer GPU and serves it as a local OpenAI-/Anthropic-compatible API.

Mistral opens a public preview of Mistral Large 4, a 1.05T-parameter open-weight model trained in Europe

mistralmistral-large-4open-weightsmoeeurope+5+6

Mistral puts Mistral Large 4 ("le Chonk") into public preview: a 1.05T-parameter open-weight MoE served from EU datacenters, with weights promised by month's end.

DeepSeek ships V4.1-Flash: asymmetric MoE, 1/4 the KV cache

deepseekdeepseek-v41-flashmoeagent-infrapricing+2+3

552B MoE, native vision, 8B/16B active params, MIT weights. The KV-cache compression DeepSeek wants agents to live with.

Cohere North Small Translate: 218B MoE translation model

coherenorth-small-translatemachine-translationmoeopen-weights+1+2

Cohere's first North-family translation model is a 218B/25B-active MoE under CC BY-NC 4.0. Cohere-run WMT26 tests score it 83.6; the eval method is GPT-5.6-Sol as judge.

Moonshot AI Kimi K3: 2.8T open-weights MoE model

moonshot-aikimikimi-k3open-weightsmoe+3+4

Kimi K3 is a 2.8T-parameter open-weights MoE model built on Kimi Delta Attention and Attention Residuals, with 1M-token context and native vision. Weights to follow by July 27.

Mistral opens Leanstral 1.5: 6B-active Apache-2.0 Lean 4 prover

mistralleanstralproof-engineeringformal-verificationlean-4+15+16

Apache-2.0 Leanstral 1.5 (119B/6B MoE) saturates miniF2F, hits 587/672 PutnamBench, sets SOTA on FATE-H and FATE-X, and finds 5 unknown bugs in 57 Rust repos — at ~$4 per problem.

DiffusionGemma: 1,000+ tokens/sec open-weights text gen

google-deepmindgemmadiffusiongemmatext-diffusionopen-weights+10+11

Google DeepMind's June 10, 2026 release: a 26B/3.8B text-diffusion model denoising 256 tokens in parallel; ~4x faster than AR on a single H100, Apache 2.0, explicitly experimental.