Google Launches Gemini Decision 1: Scores 7 of 9 Benchmarks Against OpenAI and TypeSafe

Google launched Gemini Decision 1 on Wednesday, bringing its first dedicated decision model into a category that TypeSafe AI created only 57 days ago. Available immediately through Google AI Studio and the Gemini Decisions API, the model returns structured classification outputs rather than generated text — making binary judgments, ranking multiple-choice options, or applying scoring rubrics across text, audio, video, and PDF inputs in a single API call. On the independent JevBench evaluation, Gemini Decision 1 placed first in 7 of 9 test categories and recorded the lowest Expected Calibration Error among all tested systems, a property that matters substantially for the agentic applications the model is designed to serve.

The launch marks the moment a category first staked out by a startup becomes hyperscaler infrastructure. TypeSafe AI shipped the original Jev model on September 15, 2026, coining the term "decision model" and framing the category as a System One counterpart to generative models. OpenAI followed on September 29 with GPT-6 Luna Decisions, Cloudflare on October 1 with Clef, and now Google. Within eight weeks, three of the largest AI platform providers in the world have shipped competing implementations of an architecture that did not exist as a named product category before mid-September.

A New Model Category Built Around Typed Outputs

Understanding what makes Gemini Decision 1 distinctive requires understanding what a decision model actually is — and how fundamentally it differs from the generative models most developers work with today.

Conventional large language models like GPT-6 or Gemini 3.5 Flash operate autoregressively: they sample one token at a time, feeding each output token back as input to generate the next. For structured classification tasks — is this content policy-compliant? which of these four actions should the agent take next? how would a reviewer score this response on a 1-to-5 rubric? — that generation loop is computational overhead. The model must produce a chain of tokens that arrives at a decision rather than directly scoring the decision itself.

Decision models eliminate the autoregressive loop for classification tasks. As detailed in Google's announcement, Gemini Decision 1 processes all candidate answers in a single prefill pass, scoring each option simultaneously rather than generating text token by token. This architecture is conceptually closer to BERT-style encoder models than to GPT-style decoder models, though Gemini Decision 1 is built on Gemini's native multimodal encoder-decoder architecture rather than a pure encoder stack.

The model exposes three decision types through the Gemini Decisions API. The first, called noul — a portmanteau of "no" and "ul" — returns a binary yes/no judgment with a calibrated probability score. The second, choice, accepts between 2 and 255 labeled options and returns normalized probability scores across all of them, summing to 1.0. The third, score, applies an ordered rubric — for instance, a five-level quality scale — and returns both per-level probabilities and a continuous weighted score derived from those probabilities.

Each request can include up to 1,000 questions against a single input document or media file, allowing a developer to run a full content audit in a single API call rather than issuing separate requests for each policy dimension. At a claimed median compute latency of 50 to 100 milliseconds, the model is positioned for latency-sensitive routing and triage applications where a generative model's typical response time would create unacceptable bottlenecks.

The economic implication at scale is significant. A generative model running inference on a classification task must execute the full autoregressive sequence — attention over the full context, token sampling, the generation loop — to arrive at a label it could in principle produce in a single scoring pass. For a production system handling millions of routing or moderation decisions per day, that compute overhead translates directly into infrastructure cost and latency. Decision models trade generation flexibility for classification efficiency: they cannot produce open-ended text, but for the structured-output subset of AI tasks, they eliminate work that generative models have always been doing at unnecessary cost.

Why Calibration Matters More Than Accuracy Alone

The benchmark conversation around decision models has increasingly centered on a metric called Expected Calibration Error, or ECE, and Gemini Decision 1's headline claim on that metric deserves careful unpacking.

ECE measures how well a model's stated confidence aligns with its actual accuracy. A perfectly calibrated model that says it is 80 percent confident in an answer should be right on that answer type roughly 80 percent of the time. Lower ECE is better: a model with an ECE of 0.03 is better calibrated than one with an ECE of 0.12. For an agentic pipeline that uses a decision model to gate human review — only escalate when confidence is below 0.85, say — a poorly calibrated model systematically produces wrong escalation rates, either over-flagging routine decisions or under-flagging genuinely uncertain ones. The pipeline's reliability depends directly on whether the stated probability tracks the real one.

Google reports Gemini Decision 1's ECE at 0.0250. For comparison, company-submitted scores to the JevBench leaderboard show OpenAI GPT-6 Luna Decisions at 0.0721 and TypeSafe Jev 1.14 at 0.0494. The gap against the leading generative model used as a baseline, Gemini 3.5 Flash-Lite, is substantially larger: its ECE registers at 0.2104, meaning the generative baseline is miscalibrated by roughly eight times as much as the decision model at the top of the leaderboard.

An important qualification applies to all of these numbers: JevBench is an independent benchmark operated by researcher Florian Standhartinger at Benchmark Heaven, but the scores on the leaderboard are company-submitted rather than independently run by a neutral third party. The benchmark's methodology separates 300 public test items from 1,200 sealed evaluation items to reduce contamination risk, but independent reproduction of these ECE figures has not yet occurred at time of publication. Developers evaluating decision models for production use should run their own domain-specific calibration tests before treating the leaderboard scores as a reliable prediction of performance on their workloads.

On the broader JevBench accuracy leaderboard, Gemini Decision 1's company-submitted score of 92.2 percent places it first in 7 of 9 test categories. TypeSafe Jev 1.14 leads in one remaining category, and the benchmark's tool-calling timing test, When2Call — an NVIDIA-published evaluation measuring when a model correctly decides to invoke an external tool rather than answering directly — goes to TypeSafe's Jev 1.14 at 82.0 percent against Gemini Decision 1's 79.9 percent.

The Competitive Landscape Eight Weeks After Category Inception

The three main decision model providers now differ on dimensions that actually matter for different developer use cases, and the differences are substantive enough to affect architectural choices.

TypeSafe Jev 1.14, released October 14, 2026, charges $0.042 per million input tokens — roughly 2.4 times cheaper than both Google and OpenAI at $0.10 per million. For high-volume classification pipelines processing hundreds of millions of queries per day, that pricing difference is a meaningful infrastructure cost. Jev 1.14 supports a 256,000-token context window and leads the When2Call tool-selection benchmark, which matters for agentic systems where the decision model must judge whether to invoke a tool or return an answer directly. TypeSafe was founded by Diogo Almeida, formerly at OpenAI, alongside Erik Gafni and Sasha Sheng. The company raised a $40 million seed round led by DCVC and subsequently closed an $870 million Series A at a $7.5 billion valuation in October 2026. Early adoption data from Vercel suggests the category is gaining traction quickly: Vercel reported that 13 percent of paid AI Gateway teams had adopted Jev within 24 hours of launch, rising to 35 percent within two weeks — the fastest adoption of any model in Vercel AI Gateway history, though the figure was reported by Vercel and not independently confirmed.

OpenAI announced at DevDay on September 29, 2026 the launch of GPT-6 Luna Decisions, built on the GPT-6 Luna base model released September 22. It matches Google's $0.10 per million token price but offers a substantially larger context window of 1,050,000 tokens, compared with 256,000 tokens for both Google and TypeSafe. That context advantage matters for use cases requiring very long document evaluation — full legal contracts, extended transcripts, or lengthy multi-turn conversation histories fed to a classifier in a single call. OpenAI's decision model currently supports only text and image inputs, however, with no audio or video support announced.

Google's differentiation sits in three areas. The multimodal input support is the most distinctive: Gemini Decision 1 handles audio and video natively, which neither TypeSafe nor OpenAI currently offers at the decision model tier. This opens routing and classification use cases involving video content moderation, audio call triage, and document classification pipelines that mix PDF, image, and text. Google also leads on calibration by the ECE metric, which is particularly relevant for production agentic deployments. The architectural bet — building the decision model on Gemini's native multimodal stack rather than a purpose-built encoder — appears to have produced better calibration at the cost of higher per-token pricing than TypeSafe.

Cloudflare Clef, the fourth entrant in the category launched on October 1, 2026, occupies a different position: it runs at the edge rather than in a centralized cloud data center, targeting latency-sensitive classifications that can be executed close to the requesting client. Clef is not yet represented on JevBench at time of publication, making direct accuracy comparisons unavailable. Its edge deployment model is architecturally distinct from the three centralized offerings and targets a different segment — applications where network round-trip time to a central inference endpoint is itself the bottleneck.

What Google has not addressed is the pricing gap. At $0.042 per million tokens, TypeSafe remains the low-cost option by a substantial margin, and for text-only decision tasks where the multimodal capability is irrelevant, the cost difference is a real consideration at scale.

Four Internal Deployments Google Is Already Running

Google did not wait for external developer adoption before deploying Gemini Decision 1 internally. The company disclosed four production use cases at launch, each illustrating a different deployment pattern for the model.

The clearest demonstration of economic logic is in Gemini Notebook, where Google uses a decision model to route incoming queries by complexity before dispatching them to generative models. The company claims this produces a 40 percent reduction in latency for everyday queries — the ones that do not require a full generative response — by allowing simpler requests to be triaged and handled through lower-cost paths before they ever reach a full LLM inference step. The classification step itself is fast and cheap; the savings compound across millions of daily queries.

The second deployment is in Stitch, Google's recently launched design system tool, where Gemini Decision 1 powers the Rapid Canvas feature. The model classifies design intents and routes them to the appropriate generation workflow, allowing the product to distinguish between user requests that require full layout generation and those that require only component substitution or property adjustment.

The third use case is in Google Meet's closed captions feature, where a decision model classifies emoji reaction candidates from audio and transcript context rather than passing the full signal to a generative model for open-ended response generation. The classification step reduces latency for real-time emoji rendering while keeping the decision constrained to a fixed option space.

The fourth is a security application in Google Search's MCP connector layer, where the decision model classifies incoming tool-call requests for prompt injection attempts and policy violations before they reach the execution layer. Using a decision model for this gate rather than a generative classifier keeps the security checkpoint fast and deterministic — a yes/no structured output with calibrated confidence rather than a free-form safety rationale.

Taken together, these four deployments illustrate the core economic argument for the category: in any system where a generative model currently handles tasks that could be expressed as structured classification, a decision model can substitute for that classification step at a fraction of the cost and latency, while freeing the generative model's capacity for the tasks that actually require text generation.

Limitations and Outstanding Questions

Several important limitations apply to Gemini Decision 1 and the decision model category broadly.

The benchmark numbers that anchor most current coverage of this model are company-submitted to an independent benchmark rather than independently reproduced. JevBench's sealed evaluation set reduces gaming risk compared with fully open benchmarks, but it does not substitute for replication by neutral third parties. Developers should weight published ECE and accuracy figures as directional indicators rather than production guarantees.

The model's context window of 256,000 tokens, while large by historical standards, sits below OpenAI's 1,050,000-token limit for applications requiring very long document classification. Google has not announced plans to expand the context limit.

The multimodal advantage is real but conditional on use case. For text-only decision pipelines — the dominant category today — TypeSafe's roughly 2.4x pricing advantage is a substantial counterargument to Google's calibration lead. The case for Gemini Decision 1 over Jev 1.14 on pure text tasks currently rests on calibration quality, which requires validation on each team's specific task distribution before it translates into a production argument.

Decision models also remain new enough that production failure modes are not yet well documented. The question of how decision model outputs degrade under adversarial inputs — crafted text designed to shift calibrated probabilities rather than fool a classifier — has not been extensively studied at this model generation. The security application at Google Search's MCP layer suggests the company believes the model is robust enough for adversarial contexts, but that assessment is not yet backed by independent red-teaming published to the community.

What the Category Looks Like From Here

The speed of hyperscaler entry into the decision model category is itself a signal worth noting. TypeSafe's 57-day window between category creation and Google's launch is unusually short by historical standards for a genuinely new API category to attract hyperscaler competition. It suggests the economic argument was immediately legible to platform teams: any organization running generative models at scale has classification tasks embedded in those pipelines, and a faster, cheaper, better-calibrated alternative addresses a real line item.

The practical question for 2027 is whether decision models expand beyond routing and triage into the broader set of AI-assisted evaluation tasks — automated grading, interview screening, document compliance review, credit decisioning — that currently use generative models for scoring. Those applications share the same structural property: a constrained output space that does not require text generation. The calibration question is especially important here, because human-consequential decisions require probability estimates that can be audited rather than narrative outputs that resist measurement.

TypeSafe's remaining lead on When2Call tool-selection accuracy — the benchmark most relevant for agentic orchestration — and its substantially lower price create a durable niche even after Google's entry. The category founder is not displaced by the first hyperscaler to arrive, but the dynamics shift from a startup defining the space to a multi-vendor market where differentiation becomes more specific. Developers evaluating these systems today are comparing calibration curves, multimodal support matrices, context window limits, and token pricing rather than simply asking whether a decision model exists. That is a sign the category has arrived.