Saluki 27B: Conway Research Compresses Qwen3.8-27B to Under 8 GB With 96% Benchmark Retention

Conway Research released Saluki 27B 1.0 on October 9, compressing Alibaba's 54-gigabyte Qwen3.8-27B to a 7.89-gigabyte GGUF file — a 6.84-fold reduction that the company says preserves 96% of the base model's benchmark performance across nine evaluations. The release arrives four days after Conway Research launched its Underdog on-device AI assistant, positioning Saluki 27B as the open-source inference backbone for a commercial product strategy built around consumer hardware.

At 7.89 gigabytes, Saluki 27B fits an 8-gigabyte-VRAM graphics card — an RTX 4060, an Arc A770 — and the base Apple M-series configuration. The Q4_K_M quantization of the same model requires roughly 17.1 gigabytes, demanding a 24-gigabyte GPU or higher-end Apple Silicon. Saluki 27B is the first quantization of Qwen3.8-27B accessible to the hardware class most developers own.

What 2-Bit Quantization Actually Does to a 27B Model

At two bits per parameter, conventional GGUF quantization discards enough numerical precision that the result often becomes functionally incoherent on complex tasks. The widely-used Q4_K_M tier runs around 17 gigabytes for a 27B model; true 2-bit formats typically produce severe quality degradation in exchange for the size reduction.

The explanation for Saluki's claimed retention is importance-matrix (imatrix) calibration — a technique in the llama.cpp ecosystem that runs a calibration dataset through the model before quantization to identify which weights most affect output quality, then rounds those positions more carefully. The released filename, Underdog-Saluki-27B-1.0-IQ2-mix.gguf, confirms the IQ2-mix format from the IQ2 family; the specific calibration dataset used remains undisclosed. IQ2-family GGUF formats using imatrix consistently outperform naive Q2 quantization.

Qwen3.8-27B's architecture may also help. The model uses 64 transformer layers split 48:16 between Gated DeltaNet (linear attention with a fixed-size recurrent state) and conventional quadratic attention. Linear attention layers avoid the expanding key-value cache of full attention; their compact state may also be less sensitive to aggressive weight rounding — a plausible structural reason for better-than-expected 2-bit retention, though not yet confirmed in published quantization research for this model.

Tool-Calling Gains at 2-Bit Raise Benchmark Questions

The most striking claim is not quality preservation but quality improvement on certain tasks. Conway Research reports Saluki 27B scores 88 on its Underdog Bench tool-calling evaluation against 84 for full-precision Qwen3.8-27B, and 42 versus 35 on parallel tool calls. A 2-bit quantization outperforming the base model on any benchmark is counterintuitive and warrants scrutiny.

Underdog Bench is a proprietary evaluation developed and run by Conway Research. Its methodology is not publicly documented and no independent team has reproduced its results. Imatrix calibration against tool-calling-heavy data could systematically improve performance on Underdog Bench's task distribution while degrading performance elsewhere. The independently validated failure mode is the one Conway Research discloses: AIME 2025 and 2026 scores fall by 15 to 18 points — a meaningful regression on benchmarks that test multi-step symbolic reasoning, where weight precision is known to matter most.

Until Underdog Bench methodology is published and reproduced externally, the tool-calling improvement should be treated as company-reported. What the evidence does support is that Saluki 27B preserves general reasoning capability well enough for practical deployment — the AIME caveat defines where not to push it.

The Hardware Tier Saluki 27B Unlocks

Before this release, the best option for an 8-gigabyte GPU was a Q4_K_M of a 7-to-9-billion-parameter model — a Llama 3.1 8B, Qwen2.5 7B, or Gemma 2 9B, each 4-to-6 gigabytes. Saluki 27B reaches 27 billion parameters at a comparable file size, offering meaningfully greater depth for instruction-following and tool-use tasks — provided they do not depend on peak mathematical reasoning.

Read more: Alibaba's Qwen3.8-Omni-Flash API explained

Conway Research's Open-Core Strategy Behind the Release

Saluki 27B is released under the Apache 2.0 license and runs on standard llama.cpp. Any developer can use it with Ollama, LM Studio, or a bare llama.cpp install. That openness is commercially deliberate.

Conway Research launched Underdog four days earlier, on October 5–6. The app runs via the company's proprietary Husky inference engine, optimized for Apple Silicon and separate from llama.cpp. Open-sourcing the Saluki GGUF weights under Apache 2.0 drives adoption across the llama.cpp ecosystem — Ollama, LM Studio, developer testing — while Conway Research keeps its inference advantage proprietary. Conway Research closed its funding round on October 2, backed by Andreessen Horowitz, Khosla Ventures, Hummingbird, SV Angel, and Anthology Fund; angel investors include Patrick Collison and Guillermo Rauch. The sequencing — capital close, Underdog launch, open GGUF release — is a coordinated go-to-market, not incidental research generosity.

Independent Verification Is the Metric That Matters Next

Qwen3.8-27B was released August 14; Saluki 27B followed within eight weeks. Two facts are solid: the 7.89-gigabyte size unlocks 8-gigabyte hardware for a 27B model, and the AIME regression confirms math-heavy reasoning takes a real hit at 2-bit. Everything between those — the 96% aggregate retention, the tool-calling gains, the nine-benchmark summary — is company-reported data from a proprietary evaluation framework. When independent benchmarks arrive, they will determine whether Saluki's retention claim reflects a genuine step in 2-bit quantization quality or favorable benchmark selection — a result worth waiting for before committing Saluki to production workflows that depend on precision reasoning.