Global Workspace Survives Recurrence: Interpretability Study Maps How Looped Transformers Differ

A new arXiv preprint posted September 1 by Wenlong Wang and Fergal Reid extends mechanistic interpretability's most powerful workspace analysis tool to a class of AI architectures that the field has largely ignored: looped and depth-recurrent transformers. The results confirm that a global workspace — the compact internal hub of verbalizable, causally active representations identified inside standard transformers by Anthropic researchers in July — does form inside recurrent models too. But recurrence fundamentally rewrites the rules governing how that workspace can be read, modified, and steered. For the growing number of researchers and practitioners building or auditing looped-architecture models, the paper's message is pointed: tools calibrated for standard feed-forward stacks cannot be transplanted to looped systems without architectural adaptation, and in some respects the workspace becomes harder to reach precisely when it is doing the most work.

The Global Workspace, Revisited

To understand what Wang and Reid tested, it helps to recall what the Gurnee et al. paper from Anthropic established in July. Working on Claude Sonnet 4.5 and related models, Anthropic researchers introduced the Jacobian lens — a technique that computes, for each token in the model's vocabulary, the average linearized effect of an intermediate activation on the model's future outputs, averaging that calculation across a large corpus of prompts. Applying the lens layer by layer reveals which concepts a model is "poised to verbalize" at any moment during processing, even when those concepts never appear in what the model actually writes. The full paper, "Verbalizable Representations Form a Global Workspace in Language Models," was published on Anthropic's Transformer Circuits platform.

The result was a richly structured internal space — the J-space — occupying a band of intermediate layers. Content in the J-space was verbalizable on request, responsive to direct instruction, causal for multi-step reasoning, broadly readable by downstream circuits performing different operations on the same representation, and selective: it was absent from purely automatic processing like grammatical parsing but present when the model needed to chain reasoning steps or report on its internal state. These five properties, taken together, match the functional signature of what cognitive scientists call a global workspace — the brain's shared representational hub that makes information available for flexible, deliberate use rather than keeping it locked inside a dedicated processing circuit.

Critically, the J-space was not designed. It emerged from training on next-token prediction, suggesting that some form of compact, shared, verbalizable workspace may be a convergent architectural solution for systems that must chain complex reasoning.

Standard transformers build this workspace by stacking distinct layers, each with its own independently parameterized weights, in a fixed-depth sequence. Looped transformers build effective depth differently: a core set of layers is reused — with shared weights — across multiple iterations before the network produces an output. That architectural difference is not cosmetic. It changes the dynamics of every computation the network performs, and it raises a direct question: does the interpretability toolkit developed for the standard case transfer?

Extending the Lens: The Virtual-Unrolling Adapter

Applying the Jacobian lens to a looped transformer requires solving a non-trivial technical problem. The standard lens computation works by treating each layer as a distinct step in a sequence, averaging the Jacobian of the final-layer activation with respect to each intermediate layer's activation. In a looped architecture, where the same weights are reused across every iteration, there is no well-defined "layer 5 of 64" — there is "iteration 2, block 3," which shares weights with "iteration 1, block 3" and every other instance of that block. The mathematics of the averaged Jacobian needs to be redefined.

Wang and Reid address this with what they call a virtual-unrolling adapter. Rather than treating the network's shared-weight iterations as physically the same set of parameters applied multiple times, the adapter treats each iteration as a conceptually distinct layer for the purposes of the lens computation — "unrolling" the recurrent structure into a virtual depth sequence. This makes each loop pass addressable by the standard Jacobian lens machinery, which can then fit the lens, generate readouts, and run causal experiments across each virtual layer independently. The methodology is detailed in the preprint, which has not yet been peer-reviewed.

The full workspace suite — lens fitting, readout, and eleven families of causal experiments — was then applied to two publicly available looped models: Ouro-2.6B, developed by Rui-Jie Zhu and colleagues at ByteDance Seed, and Huginn-0125, developed by Jonas Geiping and colleagues at the University of Maryland and the ELLIS Institute Tübingen. A standard 64-layer untied transformer, Qwen3.6-27B, served as the feed-forward baseline.

Ouro-2.6B: The Workspace Reconstructs Itself Every Loop

Ouro-2.6B is a 2.6-billion-parameter model whose 48 transformer layers are organized as 12 layers looped 4 times. A distinguishing feature of Ouro's training regimen is what the Ouro paper calls deep supervision: rather than applying a training loss only at the end of the full forward pass, the model is trained to produce meaningful outputs at each loop boundary. This means the model has been explicitly incentivized to generate a readable state at the end of every iteration, not just at the end of the final one.

The workspace study finds that this supervision choice has a direct structural consequence inside the network. The J-space does form in Ouro — Wang and Reid confirm that a mid-depth band of verbalizable, causally active representations emerges in the looped portion of the architecture, satisfying the global workspace properties — but that workspace is reconstructed from scratch on each loop pass rather than carried forward from the previous one. Content that existed in the workspace at the end of iteration one is not simply propagated into iteration two; the network rebuilds it.

The consequence for intervention is significant. In a standard transformer, a researcher can patch a J-lens vector at a given layer and allow the forward pass to continue, trusting that the patched content will be available to downstream circuits throughout the rest of the network's depth. In Ouro, that logic breaks at every loop boundary. Because workspace content is reconstructed rather than transported, linear transport of that content across loop boundaries fails: a patch applied in iteration one does not survive into iteration two in the form the lens would predict. This means that writes and ablations — the core operations by which researchers modify a model's internal workspace — must be applied across every remaining loop iteration rather than at a single point. The intervention surface is multiplied by the number of remaining loops.

This is not merely a practical inconvenience for researchers running causal experiments. It also implies that the efficiency framing often used to justify looped architectures — that a 2.6B parameter model running four loops is computationally equivalent to a roughly 10B parameter standard model — needs to be extended to interpretability. The same multiplied compute that enables a smaller model to approximate larger-model performance also multiplies the cost and complexity of comprehensively auditing the model's internal representations through workspace interventions.

The relationship between deep supervision and verbalizable access is also clarified here: the per-iteration training signal is precisely what makes content injected into the workspace verbalizable at each loop boundary. Architectures trained without this supervision — like Huginn — behave differently in this respect.

Huginn-0125: Content Persists, But Through a Narrow Window

Huginn-0125 presents a structurally different profile. The architecture consists of a 3.5-billion-parameter model organized as a two-layer prelude, a four-layer recurrent core looped 16 times, and a two-layer coda. Unlike Ouro, Huginn was designed and trained specifically for latent reasoning — for building up sophisticated internal representations across many recurrences without necessarily externalizing intermediate states. It was trained without per-iteration deep supervision; the loss is applied at the coda output, after all 16 recurrences complete.

Inside Huginn, the J-space does form and content does persist. Importantly, Huginn carries workspace content forward across all 16 of its recurrences rather than reconstructing it each time — a property consistent with the model converging toward a stable fixed point across iterations, which an April 2026 mechanistic analysis by Blayney, Arroyo, Obando-Ceron, and colleagues had previously confirmed using different methods.

What changes is the effective reach of interventions. When Wang and Reid apply reads, writes, and ablations to Huginn's virtual-unrolled layers, they find that these operations are effective only within a sliding window of roughly two consecutive recurrences. A patch applied at iteration 6 influences processing at iterations 6 and 7 but not at iteration 10. Ablating a workspace concept at iteration 3 removes it from the forward pass at iterations 3 and 4, but by iteration 6, the concept has effectively re-established itself from the persistent hidden state.

This two-recurrence window is not a hard architectural parameter set during design; it is an emergent property of how Huginn learned to use its recurrences during training. Its practical consequence is that the range over which a researcher can steer Huginn's internal representations — the window of influence for any single intervention — is a small fraction of the model's total computational depth. The kind of broad, mid-network workspace modification that produces reliable behavior changes in standard transformers requires repeated interventions applied at multiple recurrence steps to have comparable effect in Huginn.

Notably, whether existing content can be steered in Huginn does not track the presence or absence of deep supervision — this is independent of how the training loss was applied. Both Ouro and Huginn show some form of access constraint on their workspace contents, but through different mechanisms: Ouro's reconstruction-at-every-loop creates a transport barrier; Huginn's persistent-state-with-sliding-window creates a temporal locality barrier.

What This Means for Alignment Monitoring

The Wang and Reid findings arrive at a moment when looped and depth-recurrent transformers are no longer solely academic curiosities. Reports citing anonymous sources suggest that at least one frontier production AI system may incorporate recurrent depth as part of its inference architecture.

The GPTS24 investigation of that development noted a concern raised by AI safety researchers, including Redwood Research chief scientist Ryan Greenblatt: that recurrent depth moves a portion of a model's effective reasoning into hidden activation states that chain-of-thought monitoring cannot read. Greenblatt predicted that mechanistic interpretability — the ability to read and steer internal representations directly — would be the necessary solution to the monitoring gap that recurrent inference creates.

The Wang and Reid paper makes that prediction technically precise, and in doing so reveals that the prediction comes with its own preconditions. Mechanistic interpretability's most powerful tool for workspace-level monitoring, the Jacobian lens, is capable of detecting a global workspace inside recurrent architectures. The workspace is there. But the access rules for modifying what is inside it depend on architecture-specific properties that must be characterized for each new looped design before intervention tools can be usefully applied.

For a model like Ouro, alignment monitoring based on workspace interventions would require patching across every remaining loop — a multi-intervention regime that presupposes knowing exactly how many loops remain at any given point in inference, and applying all patches simultaneously or in sequence. For a model like Huginn, workspace interventions degrade in effectiveness within two recurrences, meaning that any behavioral modification achieved by targeting the workspace at one point in the computation may be partially erased by the model's own subsequent processing before the output is produced.

Neither failure mode makes workspace-based interpretability impossible for looped models. But both mean that the interpretability toolkit requires explicit architectural modeling of the looped structure — the kind of modeling that the virtual-unrolling adapter is designed to support — before intervention-based monitoring can deliver the reliability that alignment auditing requires. A single-point J-lens patch applied as if the architecture were a standard transformer will, in most cases, not behave as expected.

Read more: OpenAI Astra and the monitoring gap in recurrent-depth AI

Deep Supervision Shapes Verbalizable Access

One of the more unexpected findings in the Wang and Reid paper concerns the role of per-iteration training supervision in shaping what the workspace can do. The authors observe a clean asymmetry: whether newly injected content can be verbalized at a given loop boundary tracks whether that boundary received an explicit training signal. Ouro, with its per-iteration deep supervision, allows content injected at loop boundaries to become verbalizable at those boundaries. Huginn, without this signal, does not support the same kind of on-demand verbalization of injected content at intermediate recurrence steps.

This finding has practical implications beyond the two models studied. It suggests that architectural choices made during training — specifically whether and where to apply intermediate loss signals — shape not just model performance on downstream tasks but the internal structure of how the workspace can be accessed and modified. A recurrent model designed with explicit per-iteration supervision may be more amenable to workspace-based interpretability methods. One trained only toward a final-output loss may carry a workspace that is harder to reach from the outside, not because the workspace is absent, but because the mechanisms for injecting content into it were not incentivized during training.

This does not mean that deeply supervised looped models are more interpretable in all respects — Ouro's workspace reconstruction at every loop boundary imposes its own constraints. But it does suggest that interpretability considerations can and should be factored into the design of recurrent training regimes, and that the choice of where to apply the training signal has consequences for the architecture's internal organization that extend well beyond benchmark scores.

Prior Interpretability Tools and the Limits of Transfer

The Wang and Reid study positions itself in relation to earlier interpretability tools that preceded the Jacobian lens. The logit lens, which applies the model's unembedding matrix directly to intermediate residual stream activations, has been widely used on standard transformers and provides a quick readout of what each layer "thinks the next token might be." The tuned lens trains per-block affine probes to match the output distribution more accurately than the direct unembedding. Both provide layer-by-layer readouts but rely on correlational rather than causal relationships between activations and outputs. Neither was designed to handle weight-shared iterated architectures.

Earlier efforts to interpret Huginn's internal representations used these simpler tools with limited success. A 2025 preprint by Wenquan Lu, Yuechuan Yang, and colleagues at Brown University and Harvard attempted to detect latent chain-of-thought inside Huginn using the logit lens and a variant called the coda lens — both correlational approaches that reveal what layers predict but cannot confirm causal responsibility. The coda lens, which reads out the model's representation only after it passes through the final coda blocks, can identify what the model is about to produce but cannot support the kind of midcomputation steering experiments needed for alignment monitoring.

The Jacobian lens improves on both by using the model's own averaged gradient structure to define the mapping from intermediate activations to outputs — a causal relationship that can be tested through intervention. Critically, the Jacobian is computed by averaging over a large corpus of diverse prompts, which isolates representations that are verbalizable in general from those that merely happen to predict the current token. The virtual-unrolling adapter that Wang and Reid introduce extends this causal machinery to looped architectures in a way that neither the logit lens nor the tuned lens was designed to do.

One related prior contribution comes from the April 2026 mechanistic analysis paper by Blayney and colleagues, which used different methods to document stages-of-inference in looped architectures and confirmed that Huginn-0125 converges to a fixed-point representation across recurrences while Ouro does not. That analysis is consistent with what Wang and Reid find through the lens: Huginn's persistent-content profile reflects its fixed-point convergence, while Ouro's reconstructed workspace reflects the absence of that convergence in a deeply supervised design. The two papers, using different methods, produce a coherent picture.

What no prior work had done was apply a causally valid, intervention-capable interpretability framework to these architectures and map the consequences for workspace access. That is the contribution Wang and Reid make, and it advances the conversation from "what does the activation look like" to "what happens when you try to change it."

Interpretability Research Must Adapt as Architectures Diversify

The trajectory of looped transformer development in 2025 and 2026 has been one of growing scale and increasing practical interest. The Huginn-0125 paper from the University of Maryland demonstrated test-time compute scaling through recurrence at research scale. ByteDance's Ouro model pushed that approach to 7.7 trillion training tokens, approaching the data volumes used for production-competitive models. More recently, a family of Looped-MoE architectures has combined recurrent depth with sparse expert routing, finding efficiency gains that suggest recurrent designs may have advantages in certain inference regimes.

The spread of looped architectures across research labs — and their apparent adoption in at least some production contexts — means that the interpretability gap Wang and Reid identify is not a theoretical concern. Alignment auditors, safety researchers, and interpretability practitioners who have developed workflows around standard transformer architectures will need to adapt those workflows to account for loop boundaries, workspace reconstruction, and sliding-window access constraints if they are to provide meaningful oversight of models that use recurrent inference. Practically, this means that any automated alignment auditing pipeline currently built around J-lens readouts — including the "counterfactual reflection training" and evaluation-awareness detection methods demonstrated by Gurnee et al. — cannot be applied to looped models out of the box. Each pipeline component that involves writing to or ablating from the workspace must be reimplemented to account for architecture-specific access patterns. The virtual-unrolling adapter provides the conceptual scaffolding for that reimplementation, but the engineering work remains ahead.

The virtual-unrolling adapter and workspace suite that Wang and Reid have applied to Ouro and Huginn represent the first instance of causally valid, intervention-capable interpretability at this level being brought to looped architectures. The next steps that the paper implicitly points toward include: extension to Looped-MoE models, where the interaction between recurrence and sparse expert routing may create additional access constraints; development of multi-point intervention protocols calibrated to specific loop counts and boundary structures; and investigation of whether training regimes can be modified to make workspace access easier without sacrificing the performance advantages that recurrence provides.

What the paper establishes for now is that the global workspace does not disappear when a transformer loops — it survives recurrence. What changes is the architecture of access, in ways that depend on how the model was trained and how its hidden states evolve across iterations. Figuring out how to read and steer what is inside that workspace reliably is the problem the interpretability community will need to solve before recurrent models can be audited with the same confidence that the Jacobian lens currently provides for their feed-forward counterparts.