OpenAI Jalapeño Chip Tops Nvidia Blackwell on Efficiency: First Published Results

OpenAI's custom inference chip has crossed from assertion to evidence. On August 25, the company published benchmark results for Jalapeño at the Hot Chips 2026 conference at Stanford University, claiming up to 1.9 times more AI work per watt and up to 3.6 times lower end-to-end latency than Nvidia's current GB200 and GB300 systems across three open-weight models. Semiconductor research firm SemiAnalysis sent engineers to OpenAI's lab to verify the runs — making this the first time an AI company's purpose-built inference accelerator has been independently assessed against Nvidia's production silicon in a public benchmark. What the results establish is significant. What they omit is equally important.

The announcement landed the day before Nvidia reported its second-quarter fiscal 2027 earnings — a $96.2 billion quarter that analysts were already calling the strongest in the company's history. The timing was not accidental. Published benchmark results that survive third-party review function, in infrastructure financing, like an audited cost curve: they are the kind of number that underwriters and counterparties require before committing nine-figure construction loans per gigawatt of planned data center capacity. OpenAI's Stargate infrastructure program and its 10-gigawatt deployment commitment with Broadcom, signed in October 2025, require exactly that kind of credibility.

Jalapeño's Architecture Was Built Around One Problem: Data Movement

Understanding what Jalapeño actually is requires setting aside the GPU frame of reference. Nvidia's GB200 and GB300 accelerators are general-purpose processors — they must handle training frontier models, running inference across thousands of different model architectures, executing scientific simulations, and serving multiple tenants across cloud infrastructure. That versatility comes at a cost: silicon area, power consumption, and memory bandwidth capacity that inference workloads never fully utilize.

OpenAI designed Jalapeño from a different starting question. Richard Ho, the company's VP of hardware, said at Hot Chips that the team started with blank paper and asked what hardware would look like if its only job were serving large language models to millions of concurrent users at the lowest possible power cost. The answer the team arrived at centers on a single design principle: minimize data movement.

In transformer inference, every generated token requires reading the entire key-value (KV) cache — the record of all prior tokens in the conversation — from memory. For long conversations or large batch sizes, this memory read dominates the workload, and latency compounds across every step of a multi-turn agent. Most existing accelerator architectures treat memory as a flat, shared resource accessible by any compute unit — which means data moves across the chip to wherever compute is available. Jalapeño inverts this. The chip is divided into 64 compute-and-memory slices with tight affinity between HBM banks and the compute cores they serve. Software explicitly places model weights and KV cache into specific slices; those data structures stay local unless the workload requires redistribution. The result, according to OpenAI, is that the chip "activates the right combination of compute, memory, and networking for each inference phase" rather than moving data to wherever compute happens to be idle.

This NUMA-style architecture — non-uniform memory access, a concept borrowed from multi-socket server design — is the mechanical reason for Jalapeño's power efficiency advantage. The chip does not run hotter or draw less power through passive choices; it accomplishes more useful work per joule because a larger fraction of its memory bandwidth goes toward productive inference rather than internal data movement overhead.

The physical specifications describe the chip's B0 stepping, which OpenAI presented at Hot Chips alongside benchmark results run on earlier A0 silicon: a single reticle-sized compute die manufactured on TSMC's N3P (3-nanometer) process, paired with an I/O chiplet and six high-bandwidth memory stacks. The package delivers 13.4 petaFLOPs of MXFP4 matrix compute, 3.4 petaFLOPs of MXFP8 compute, and 216 gibibytes of HBM4 memory at 15.4 terabytes per second of bandwidth, at a rated 700-watt package draw. In the InferenceX benchmark runs, measured sustained power stayed at or below 550 watts during active inference workloads — a figure OpenAI attributes to the efficiency of the architecture rather than conservative thermal headroom.

From a Single Chip to a 2,048-ASIC System

No inference accelerator operates in isolation. OpenAI presented the full Jalapeño rack architecture at Hot Chips: a paired system consisting of a CPU host rack (internally called Katsu) and an ASIC accelerator rack (Vindaloo), integrated by Celestica. Each Katsu tray holds two AMD EPYC Turin processors with 1.5 terabytes of system DRAM and 400-gigabit front-end networking. Directly across from each Katsu tray is a matching Vindaloo tray holding eight Jalapeño ASICs, connected via PCIe direct-attach copper cables. The ASIC rack's switch trays carry the name Chana. Network switching across the system uses Broadcom Tomahawk6 Ethernet switches.

A full 128-chip Jalapeño rack delivers 1.7 exaFLOPs of MXFP4 compute and 27.5 terabytes of HBM4 memory. But the more architecturally significant number is the scale-up domain: Jalapeño connects 128 chips within a single rack over a high-bandwidth copper backplane at 600 gigabytes per second, and scales across 16 racks — 2,048 chips — over a global interconnect operating at 200 gigabytes per second per chip. Nvidia's NVL72 scale-up domain connects 72 GPUs. Jalapeño's global domain is 28 times larger.

That interconnect breadth matters for how inference workloads are partitioned. Large models, particularly mixture-of-experts architectures like DeepSeek R1, require expert-parallel communication during the decode phase: tokens route to different expert subnetworks, creating traffic patterns that need both high bandwidth and low latency simultaneously. Jalapeño's network was designed as an integral part of the inference platform rather than an add-on, which is why OpenAI says the entire workload "remains within one connected system" from first token to last.

What InferenceX Measures — and What It Doesn't

The benchmark OpenAI used is InferenceX, a public platform developed by SemiAnalysis that measures the complete AI serving path: prefill (processing the user's input prompt), decode (generating output tokens one at a time), and end-to-end user-facing latency from first request to final token. Its methodology is publicly documented and reproducible via GitHub Actions — every data point is produced by a logged, auditable workflow run.

The three models tested were GPT-OSS 120B (OpenAI's own open-source 120-billion-parameter model), DeepSeek R1 at 670 billion parameters, and Moonshot AI's Kimi K2.5 at one trillion parameters. The latter is the largest model OpenAI has benchmarked on Jalapeño; it was selected partly because InferenceX uses it as a standard test case, not because OpenAI intends to serve it commercially. Across all three, Jalapeño delivered its strongest advantage at low-latency, interactive operating points — single-user, conversational traffic — where efficiency widened to 2.1 to 4.1 times higher performance than comparable Nvidia systems, according to company-provided figures.

One distinction the benchmark methodology requires careful attention to is single-token prediction versus multi-token prediction. Single-token prediction (STP) generates one output token per inference pass — the baseline method. Multi-token prediction (MTP), also called speculative decoding, uses a smaller draft model to propose several tokens at once, which a larger model then verifies in one pass; this technique can inflate throughput figures by a factor of three to five in favorable conditions. OpenAI ran all Jalapeño benchmarks in STP mode. Several of the Nvidia comparison baselines in the InferenceX dataset use MTP, which creates an asymmetry that inflates Nvidia's apparent throughput relative to Jalapeño in some operating point comparisons. Ho said at Hot Chips that Jalapeño's STP performance matches or beats some competitor chips' MTP performance — a meaningful architectural claim about the chip's baseline efficiency.

A critical gap in the published results is SemiAnalysis's AgentX benchmark suite. Where InferenceX measures standard prompt-response inference — 8,000-token inputs generating 1,000-token outputs, single-turn — AgentX tests long-context, multi-turn agentic workloads that stress router behavior, prefix caching mechanisms, and cache management under sustained load. SemiAnalysis explicitly noted that frameworks performing well on standard 8k/1k benchmarks may perform worse on AgentX, because real production loads stress components like routers, prefix cache mechanisms, cache management, and offload infrastructure — none of which are captured by single-turn tests. Those results have not been run on Jalapeño, and the company did not claim they had. How Jalapeño performs on agentic workloads — the segment OpenAI has emphasized as central to its product roadmap — remains unconfirmed.

Read more: Nvidia posts first Vera Rubin benchmark data: 30x efficiency gain on agentic workloads

The Vera Rubin Comparison Is the One That Actually Matters

Every comparison in OpenAI's published results ran against Nvidia's Blackwell generation — the GB200 and GB300 systems. Vera Rubin, the Nvidia platform currently shipping to customers, was not tested. This is the most significant methodological caveat attached to the benchmark. SemiAnalysis described the Blackwell comparison as "somewhat incomplete and unfair" because Jalapeño uses HBM4 memory while Blackwell systems use HBM3e — a newer memory generation that brings higher bandwidth per stack. Vera Rubin also uses HBM4.

When SemiAnalysis compared Jalapeño's published results against Vera Rubin's best figures from July 2026, the gap narrowed considerably. On total cost of ownership per token — the metric that determines whether a data center operator switches suppliers — the two platforms came out approximately tied. The comparison is further complicated by a method asymmetry: Vera Rubin's July results included multi-token prediction, while Jalapeño's InferenceX runs used single-token prediction. Jalapeño holds a narrow lead on throughput per megawatt in STP-to-STP terms; on total cost per output token including capital and power, the two are competitive.

During Nvidia's earnings call that evening, CEO Jensen Huang framed Jalapeño as hardware optimized for a single operator's proprietary workloads rather than a broadly deployable platform. OpenAI's own benchmark results, which included DeepSeek R1 and Kimi K2.5 alongside its own models, argue against this framing: the chip ran non-OpenAI models without model-specific tuning. SemiAnalysis called Jalapeño "a generalized chip for AI inference," noting it excelled "not only in low-latency scenarios but also in high-throughput scenarios" across all three tested architectures. Whether Huang's characterization reflects Nvidia's competitive intelligence or defensive positioning is unclear, but the chip's demonstrated flexibility across three architecturally different models is one of OpenAI's explicit claims about Jalapeño's design.

The competitive picture extends beyond the two-way Jalapeño-Nvidia comparison that most coverage has centered on. AMD's MI355X accelerator appears on the InferenceX benchmark dashboard, and SemiAnalysis described Jalapeño as outperforming it on single-token prediction throughput per watt — though AMD uses HBM3e rather than HBM4, creating the same memory-generation asymmetry as the Blackwell comparison. Google's TPUv7 and v8 platforms are listed as "coming soon" on InferenceX and have not been directly benchmarked. OpenAI declined to ramp Google Cloud TPUs for production use despite conducting trials — an implicit competitive signal that its own hardware strategy took precedence.

The broader significance of Jalapeño's benchmark position is not that it defeats any specific competitor in any specific configuration. It is that a team of roughly 40 engineers, starting from a blank page in mid-2024, built a first-generation inference ASIC that independently sits on the efficiency Pareto frontier alongside hardware from companies with decades of semiconductor design experience. Yole Group technology analyst Adrien Sanchez told CNBC that Jalapeño is "a threat to Nvidia's inference margins, which is the field growing the most at the moment" — not because OpenAI will capture market share directly, but because every data center operator now has evidence that purpose-built inference silicon is viable, which strengthens the negotiating position of anyone building or commissioning such chips.

HBM4 Supply Pressure Is the Constraint the Benchmarks Cannot Capture

Behind every benchmark comparison sits a supply constraint that performance tables do not reflect. High-bandwidth memory is sold out through at least 2027 across all three major manufacturers — Samsung, SK Hynix, and Micron. SK Hynix CEO Kwak Noh-jung publicly warned investors that 2027 will be the worst year of the shortage in the memory industry's history, with demand expected to outpace supply capacity beyond 2030. Micron presented data at Hot Chips 2026 showing HBM4 requires approximately three times the silicon wafer area of DDR5 memory for equivalent capacity — a penalty that compounds with each generation as stacks grow taller and more complex to fabricate. The shortage is severe enough that Nvidia is reportedly testing cut-down Vera Rubin Ultra configurations with as little as 192 gigabytes per package, compared to the standard 288 gigabytes.

Jalapeño's 216 gibibytes per package sits between those figures. In a supply-constrained environment, needing less HBM per unit of inference work delivered is an advantage that compounds with deployment scale. At the same time, OpenAI's 10-gigawatt deployment commitment with Broadcom makes it a major new claimant for HBM4 allocation alongside Nvidia — which currently holds multi-year supply agreements with all three manufacturers. According to TrendForce, Samsung is believed to be supplying HBM4 for Jalapeño, though neither OpenAI nor Samsung has formally confirmed the supply arrangement.

Read more: Nvidia AI server prices rise more than 15% as memory costs hit the AI stack

AI Designed the Chip — and Now Programs It

The nine-month figure OpenAI has emphasized in its Jalapeño communications refers specifically to the period from register-transfer-level (RTL) design freeze to tape-out in November 2025 — not to the full program duration, which was approximately 16 months from the architecture concept that emerged in mid-2024. The compression in the RTL-to-tapeout window came from using OpenAI's own models to accelerate the chip design cycle itself: exploring different implementation approaches, shortening the design-measurement-verification loop, and optimizing arithmetic circuits. OpenAI says AI assistance allowed it to pack more compute into the die while meeting power targets.

The same AI-assisted approach has continued into software optimization after tape-out. Using GPT-Astra and Codex, OpenAI's team ported three open-weight models — GPT-OSS, DeepSeek R1, and Kimi K2.5, none of which were in Jalapeño's original production plan — to run at high performance on the chip within two months. For selected attention-mechanism and mixture-of-experts computation kernels, AI-generated implementations ran 1.5 to 1.8 times faster than the human-expert-written baselines, according to company figures. OpenAI was careful to note this applies to those specific computational modules, not to full-model throughput — the distinction between a fast subroutine and a fast system matters at production scale.

The programmability model Ho described at Hot Chips is designed around predictability rather than flexibility: engineers express work as local tensors with explicit communication and predictable synchronization, and AI systems then optimize how that work maps, places, schedules, and coordinates across the chip cluster. This clear, spatially-explicit programming model — where every data movement is declared rather than inferred by a runtime — is what makes it tractable for AI-generated code to write correct, performant kernels. Ho described it as opening a viable path for AI to tackle problems in parallel programming that have traditionally been very difficult. If accurate, it suggests the AI-chip design loop that compressed Jalapeño's development timeline will compound across generations as the optimization software improves.

Jalapeño is not commercially available and OpenAI has no plans to sell or license it. When asked at Hot Chips whether external customers could access the chip, Ho said OpenAI was struggling to have enough for its own workloads. The chip's competitive pressure on Nvidia is therefore indirect: through inference market pricing signals and through the message it sends to the AI infrastructure financing market that custom silicon can now sit on the performance frontier without waiting for another generation of GPU.

What the Deployment Timeline Actually Commits To

OpenAI has stated it will begin deploying Jalapeño internally by the end of 2026 in small volumes, with more meaningful deployment in 2027. The $96.2 billion Nvidia quarter and the separately reported Anthropic-signed $4.5 billion Vera Rubin deal with Nvidia — reported the same week — make the competitive framing clear: the major AI labs are all simultaneously deploying Nvidia's current hardware at massive scale while accelerating their own custom silicon programs. These are not competing strategies; they are parallel infrastructure bets at different time horizons. OpenAI CFO Sarah Friar confirmed this directly, describing the company's compute portfolio as deliberately diversified across Nvidia, AMD, Broadcom, Cerebras, AWS, CoreWeave, Oracle, and SoftBank — noting that OpenAI "actively manages this portfolio for both capability and economics."

A second-generation Jalapeño is in deep development and a third generation is taking shape, according to OpenAI's Hot Chips presentation. Each generation builds on measured production performance rather than pre-deployment estimates — a compounding advantage that hyperscaler chip programs acquire only after sustained deployment. Gen 2 is expected to target better performance per watt; Gen 3 aims for what OpenAI described as "economical, low-latency serving."

The test that neither benchmark nor roadmap can answer is whether Jalapeño's efficiency advantage holds when exposed to real production traffic at scale — millions of concurrent users, unpredictable request patterns, and the full variety of models OpenAI serves. Custom inference chips have a documented history of performing differently in the lab and in the field. The InferenceX results show Jalapeño on the efficiency Pareto frontier against Blackwell in controlled conditions. Whether it maintains that position against Vera Rubin in production, and whether AgentX results confirm the same architecture efficiency on multi-turn workloads, are the two questions that will determine whether Jalapeño changes OpenAI's infrastructure economics or simply demonstrates what is now technically achievable.

The 16-month development cycle, the AI-accelerated kernel generation, and the verifiable InferenceX positioning together establish something the industry had not seen confirmed before August 25: a full-stack AI company can design a first-generation inference ASIC that sits at the performance frontier without GPU-lineage constraints. Whether it translates to lower token costs for developers, or whether the efficiency gain stays inside OpenAI's data centers as margin, is a business decision that has not yet been made publicly.