OpenAI's Jalapeño chip posts first measured numbers — 1.5–1.9× more AI work per watt and 1.7–3.6× lower end-to-end latency

OpenAI published the first measured performance numbers for Jalapeño, its custom inference chip, on August 25. On InferenceX — a public benchmark from SemiAnalysis — Jalapeño delivered 1.5–1.9× more AI work per watt and 1.7–3.6× lower end-to-end latency than the NVIDIA systems it was compared against (OpenAI Engineering, 2026-08-25). These are vendor-first-party numbers, not independent measurements.

This follows our June coverage, where the only claim was “substantially better per watt” with no benchmark. Now there are numbers — and the caveats matter.

What happened

On August 25, OpenAI released an engineering deep-dive (jalapeno-first-results) and a strategy essay by CFO Sarah Friar (the-full-stack-behind-abundant-intelligence). The engineering post reports Jalapeño’s InferenceX performance across GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, compared against NVIDIA GB200 and GB300. All numbers are vendor-reported.

Why it matters

• First concrete numbers from custom AI silicon at a frontier lab. Google publishes TPU benchmarks, but this is the first time a model lab previously dependent on external silicon has published measured inference results for its own chip.
• Vendor-selected comparison. OpenAI chose the NVIDIA systems, the benchmark, and the operating points. The framing favors Jalapeño. Independent replication is the missing piece.
• Internal deployment only. No SKU, no API pricing change, no external availability date.

The measured results

All numbers are from OpenAI’s InferenceX testing. Power is normalized to chip TDP: Jalapeño 700 W, GB200 1,200 W, GB300 1,400 W. Jalapeño’s measured sustained power was at or below 550 W on tested workloads.

The “throughput at matched TBT” column shows how much more work Jalapeño does at the same time-to-first-token the comparison system achieves at its best. These are the widest gaps.

How they were measured

InferenceX measures full end-to-end AI serving — prompt ingestion through response generation, not chip-level FLOPS. OpenAI tested at matched user-experience points across high-throughput to low-latency use.

The chip power caveat matters: 700 W vs. 1,200–1,400 W is chip TDP, not system-level TCO. Data-center economics depend on cooling, networking, density, and software overhead that the benchmark does not capture.

AI in the silicon loop

OpenAI used its own models to accelerate chip design, reaching tape-out in nine months. AI-generated kernels for selected GPT-OSS attention and mixture-of-experts blocks ran 1.5–1.8× faster than human-expert implementations — selected blocks only, not the full model. Three open-weight models were brought to high performance on Jalapeño within two months using Codex with GPT-Astra.

The path ahead

OpenAI plans to deploy Jalapeño internally by end of year — not a product launch. Gen 2 is “deep in development,” Gen 3 is “taking shape.” OpenAI will continue deploying NVIDIA and other partners’ accelerators for training and inference.

Practical implications

• No API change today. Jalapeño is internal infrastructure. Your OpenAI API usage is unaffected.
• Watch for Q4 2026 cost reductions. If per-watt efficiency translates to cost-per-token, OpenAI may pass savings through as it did with the GPT-5.6 Sol price cut.
• Full-stack integration is the real signal. The important development is that OpenAI can co-optimize model, chip, memory, network, and serving software in one organization — not any single benchmark number.

Risks and caveats

1. Vendor-first-party benchmarks. No independent replication exists.
2. Chip TDP, not system TCO. The 700 W vs. 1,200–1,400 W comparison is chip power rating, not data-center economics.
3. “By the end of the year” is internal. Not a product launch or API date.
4. Selected-block AI speedup. The 1.5–1.8× applies to selected GPT-OSS attention and MoE blocks, not the full model.
5. No GB300 NVL72, Blackwell Ultra, AMD MI400, or Google TPU v7 numbers. Not in the source.
6. Limited model coverage. Three open-weight models; frontier OpenAI models tested internally but not published.

What to watch

• Independent benchmarks from third-party analysts or researchers.
• OpenAI API pricing or rate-limit changes tied to Jalapeño deployment.
• Microsoft’s capex guidance for any named Jalapeño volume.
• Gen 2 disclosure — timeline, process node, training capability.
• Other frontier labs’ custom inference silicon responses.

Sources