Qwenputer Puts a Working RISC-V CPU Inside Qwen3.5-4B: No Training, No Tool Calls

A developer has compiled a functional 32-bit RISC-V processor directly into the weight matrices of Qwen3.5-4B, producing a model that executes arbitrary machine code during its forward pass — without any training, fine-tuning, or external tool calls. The project, called Qwenputer, went viral on October 28 when Divyansh Agrawal, a Qwen ambassador, posted an X thread demonstrating the system; the technical writeup, published October 22 on his blog, reveals a mechanism with direct implications for AI interpretability research and safety oversight.

The development is not weight steganography and does not exploit spare capacity hidden inside overparameterized networks — two framings circulating in early coverage that both mischaracterize what Agrawal built. Qwenputer works by prepending 11 analytically computed transformer layers to Qwen3.5-4B, with every single weight determined by a custom compiler rather than learned through gradient descent. The base model's 32 layers are frozen and unchanged. The result is an 18.8 GB checkpoint — roughly double the base model's footprint due to zero-padded expansions — that runs on a single 24 GB GPU.

How Gated DeltaNet Layers Become Exact Addressable RAM

The mechanism exploits a structural property of Qwen3.5's attention design. Three out of every four attention layers in Qwen3.5 use Gated DeltaNet (GDN), a linear recurrent neural network whose hidden state is a matrix rather than a key-value cache. Under specific conditions — gate set to 1, the update coefficient beta set to 1, and a one-hot vector used as the query key — GDN's state update equation reduces to a precise memory write. A subsequent read with the matching one-hot key retrieves the stored value exactly, with no approximation.

This property makes GDN layers function as deterministic, compiler-addressable RAM. Agrawal's compiler encodes a RISC-V processor — implementing the RV32IM instruction set, the Zba address-generation extension, and a bignum arithmetic coprocessor — into 11 prepended layers whose weights exploit this state-matrix behavior. Execution happens at inference time: a driver called RvChat boots the CPU by loading firmware into the DeltaNet state after prompt submission, then reads ADDR, VAL, and STEP trace tokens generated during forward-pass decoding. These tokens drive the CPU using Qwen's existing tool-call token format, making the processor transparent to surrounding infrastructure. The system runs C and MicroPython programs at firmware level. All layer weights are solved analytically — no gradient is computed, no dataset consumed.

Benchmark Results and What They Actually Measure

On DeepMind's arithmetic evaluation suite, Qwenputer scored 97.5% on div_big, 97.5% on mul_big, 100% on mul_div_multiple_longer, and 97.5% on add_or_sub_big; the Python tool-call reference — Agrawal's baseline using a conventional external calculator — scored 95%, 100%, 97.5%, and 97.5% on the same tasks. On the NUPA long-integer benchmark, Qwenputer reached 100%, compared to 48.6% for the unmodified base model. These figures are author-reported and have not yet been independently reproduced.

The meaningful result is not the arithmetic score itself, but that compiler-generated CPU layers executing inside transformer attention produce the same output as a Python interpreter running outside the model, with no learned approximation involved.

The base model's language generation is essentially unaffected. KL divergence between Qwenputer's text output and Qwen3.5-4B's baseline measures approximately 6×10⁻⁴ — negligibly close to zero. The prepended CPU layers do not interfere with downstream language behavior during normal inference.

Filesystem Extension: 512 Files Inside Model State

A follow-up variant extends Qwenputer into persistent storage. By allocating DeltaNet state capacity more aggressively, the filesystem variant stores 512 discrete files within the model's recurrent state, accessible across approximately 53,000 tokens of context. The architecture is conceptually analogous to embedding a microcontroller with onboard flash into an attention mechanism — a capability with no precedent in deployed transformer systems.

The Safety and Interpretability Implications Agrawal Flags

Agrawal's follow-up research identified two consequences extending beyond the arithmetic demonstration. The first is an evasion risk: because GDN layers maintain computation in an RNN state matrix invisible to the chain-of-thought token stream, a sophisticated adversarial actor could conceal reasoning steps inside the recurrent state. Standard CoT monitoring — a technique safety researchers use to detect deceptive reasoning — would be blind to computation performed inside the DeltaNet matrix. The Qwenputer architecture shows this concealment is architecturally feasible, not merely hypothetical.

The second implication runs in the opposite direction. Because the CPU layers are compiler-generated and analytically verified, their behavior can be formally specified and inspected. Agrawal identifies this as a path toward tamper-proof in-weight monitors — oversight circuits embedded in model weights with verifiable guarantees, rather than external scaffolding that can be stripped away. The same architectural property enabling hidden reasoning also enables hard-to-remove, formally specified oversight.

Read more: Goodfire finds 90–99% of last-layer neurons interpretable

Practical Limits and the Question the Architecture Raises

Qwenputer is a research prototype with real hardware constraints. The 18.8 GB checkpoint and 24 GB GPU requirement are impractical for consumer hardware; execution speed runs at approximately 3,200 CPU tokens per second on an L4 accelerator — sufficient for arithmetic demonstrations but not general-purpose computation. The RvChat driver is required at inference time, ruling out standard deployment endpoints without modification. The CPU runs firmware-level C and MicroPython programs on RISC-V, not a full operating system.

The broader question is whether the technique's underlying logic scales. GDN layers are not unique to Qwen3.5; as linear RNN attention mechanisms spread through the transformer ecosystem, compiler-targeted weight construction may become a generalized tool for embedding verified computation inside language model inference. Whether that path leads first toward more reliable AI agents or toward harder-to-detect hidden reasoning will depend substantially on how safety and interpretability researchers engage with the blueprint Agrawal has now made public.