Mistral Large 4 Open Weights Hit Hugging Face as Benchmark Credibility Gap Emerges

Mistral AI released the full weights of Mistral Large 4 — internally dubbed "Le Chonk" — publicly on Hugging Face on October 29, 2026, converting what had been an API-only preview since October 6 into a freely downloadable, self-hostable model. The release is a meaningful moment for the open-weights AI community: at 1.05 trillion total parameters, ML4 is among the largest models anyone can now run without an API dependency. The catch, surfaced by independent evaluator Artificial Analysis within hours of the weights drop, is that ML4's headline coding benchmark — the one Mistral uses to claim the #1 open coding agent rank — appears to measure significantly lower under third-party testing than the company's own figures suggest.

Mistral reports a score of 19.7% on FrontierCode 2.0, a private benchmark developed by Cognition Labs that attempts to measure whether an AI agent's code changes are good enough for a real GitHub maintainer to merge. Artificial Analysis independently measured 15% on the same benchmark — a 31% relative gap. At that lower figure, ML4 would rank below DeepSeek V4.1 Flash (29% on FrontierCode per AA) and GLM-5.3 (22%), not above them. A similar pattern holds for Terminal-Bench 4.0: Mistral claims 41%; AA finds approximately 25%.

None of this makes ML4 a weak model. On benchmarks that don't rely on Mistral's own testing, the picture is more nuanced — and in areas like financial reasoning and agent security, ML4's independent results are genuinely strong. What it does mean is that any engineering team evaluating ML4 as a coding-agent backbone should treat the headline benchmarks as company claims rather than settled facts until reproducible external results emerge.

Weights Drop Completes a 1M-Token, Vision-Capable MoE

The ML4 architecture is a mixture-of-experts (MoE) model with 128 routed expert sub-networks, of which 4 are activated per token along with a single shared expert. This sparse-activation design means that despite having 1.05T total parameters, only approximately 51 billion parameters — 49B in the core transformer blocks plus approximately 1.6B in a Pixtral-style vision encoder — are actually computed during each forward pass. The result is a model with the memorized world-knowledge of a trillion-parameter system at something closer to a 51B-parameter inference cost, though actual throughput depends heavily on hardware configuration.

The attention mechanism borrows Multi-head Latent Attention (MLA) from DeepSeek-V2, which compresses the key-value cache through a low-rank latent space rather than storing full KV pairs for every token. At ML4's 1-million-token context window, this compression matters: naive KV caching at 1M tokens with a full-rank attention would require GPU memory budgets that would make deployment effectively impossible on most hardware.

Three reasoning modes gate how much inference-time compute the model expends: Low (replacing the previous "None" label), Medium, and High. High mode, the default for agent tasks, saw output-token use fall approximately 30% versus the preview during three weeks of additional reinforcement learning post-training that took place between the October 6 API preview and the October 29 weights release.

Benchmark Gap Between Mistral and Artificial Analysis Runs to 31 Percent

The FrontierCode 2.0 discrepancy deserves careful framing. The benchmark is a private Cognition Labs evaluation that tests whether an autonomous coding agent can make changes to a real GitHub repository that a human maintainer would actually accept. It is not reproducible externally — no public test set exists — and both the Mistral-reported 19.7% and the AA-reported 15% were obtained under test conditions that may differ in ways neither organization has fully disclosed.

That said, AA runs the same private evaluation protocol against multiple models and publishes comparative results. Their methodology is consistent across models even if the benchmark itself is opaque. On AA's Coding Agent Index — a composite metric they constructed themselves — ML4 scores 48%, placing it one point behind the leading GLM-5.3 (49%), though Mistral characterizes the gap as "effectively tied" and chose to cite the index in its own release materials. The AA Cyber Index places ML4 at 51%, third among open-weight models at time of testing, behind DeepSeek V4.1 Flash (57%) and MiMo-V2.6-Pro (56%).

The benchmarks where ML4 looks strongest across both Mistral-reported and AA-consistent figures are not coding leaderboards. B3 Agent Security, which measures resistance to adversarial prompt attacks in agent workflows, shows ML4 at 97.1% — up from 93.3% at the preview and, according to Mistral, the highest figure of any tested open model. FinWorkBench (Finch), which evaluates financial reasoning workflows, places ML4 at 73.3%, second only to Kimi K3 among open models. These are areas where the company's sovereign-infrastructure pitch and regulated-industry partnerships make particular sense.

Self-Hosting Requires Eight High-Memory Server GPUs

The publicly available checkpoints ship in two quantization formats: FP8, which preserves near-full numerical precision at approximately one byte per parameter (totaling roughly 1.05TB for the full checkpoint), and NVFP4, which halves the VRAM footprint by reducing parameter resolution to four bits.

Running the FP8 checkpoint requires at minimum either eight NVIDIA H200 SXM5 GPUs with 141GB HBM3e each, or four NVIDIA B300 GPUs with 288GB each. There is no single-GPU path to running this model. The NVFP4 checkpoint reduces these requirements by roughly half, making it possible on configurations like four H200s, but still demands a multi-GPU server setup that places self-hosting firmly in the enterprise or well-resourced research tier. Consumer hardware does not apply.

Supported inference runtimes include vLLM, SGLang, TensorRT-LLM, and NVIDIA NIM containers. Teams already running other large open-weights models on these stacks should find integration reasonably straightforward, though ML4's 1M-token context will stress memory capacity in ways that shorter-context models do not.

Mistral Ecosystem License Permits Deployment but Requires a Deal for Inference Resellers

The weights are released under Mistral's Ecosystem Open-Weight License, a modified MIT license that permits free deployment, fine-tuning, and redistribution with attribution. The modification that matters for startups: organizations that want to resell inference built on ML4 weights — rather than using the model internally — must obtain a separate commercial agreement from Mistral AI. Pricing for that commercial license has not been disclosed. For internal enterprise and research deployment, no royalty or revenue-sharing obligation applies.

European Sovereignty Pitch Attracts Banks and Defense Clients

Mistral trained ML4 on 3,800 NVIDIA Grace Blackwell GPUs housed in the company's own European data centers — a detail that distinguishes it from most comparable open-weights releases, which are trained on US hyperscaler infrastructure. For organizations operating under EU GDPR constraints or national data-sovereignty requirements, the provenance of a model's training compute has become a meaningful procurement criterion.

The company's named deployment partners reflect that framing: Booz Allen Hamilton, the European Union Agency for Cybersecurity (ENISA), La Banque Postale, Belfius, Crédit Agricole, ABN AMRO, and the European Space Agency. The cluster of European financial institutions and government agencies is consistent with a pitch centered on jurisdiction, auditability, and resistance to US cloud provider dependence.

With API pricing at $0.70 per million input tokens at launch (versus a $1.36 list price), ML4 sits above the DeepSeek V4.1 Flash price point of roughly $0.27 per million, though DeepSeek's inferior context window and lack of European training infrastructure are the dimensions Mistral's sales team will stress in competitive evaluations. The open weights give organizations a path off the API entirely once they've invested in appropriate hardware — an option none of the closed-weight alternatives provide.

The immediate question ML4's weights release opens is whether independent testing of FrontierCode and Terminal-Bench will narrow the gap between Mistral's figures and AA's, or widen it. Mistral has not yet published a technical report for ML4, leaving architecture details and benchmark testing conditions in the company's own release materials rather than in peer-reviewed form. How that gap resolves will determine whether ML4's genuine differentiators in security, financial reasoning, and European sovereignty are enough to win deployments against competitors that currently hold a cleaner lead on the benchmarks that agent-infrastructure teams most care about.