Recurrent Looped Transformer | Summary

Contents

Summary
1 Introduction
2 Method
3 Algorithmic Experiments
4 Related Work
5 Conclusion
Appendix
Brief Thoughts

This article explains the key points of Recurrent Looped Transformer.

2026-10-06 (arXiv)
Zhang, Yifan, Feng, Jichen, Qin, Shihan, Kung, Sun-Yuan, Wang, Mengdi.
Princeton University, University of Pennsylvania
Paper
Github
Project Page

Summary

Algorithmic state tracking requires updating an accumulated state after every input, but a standard Transformer applies a fixed number of layers at each position regardless of sequence length. Trained on 3–40-bit inputs, the Recurrent Looped Transformer (RLT) achieves 100±0% accuracy at 256 bits on parity, determining whether the number of ones is odd or even, with its 5+3 and 7+1 splits across three initialization seeds, versus 50.07±1.63% for an eight-layer decoder-only Transformer. In a split a+b, a is the number of encoder layers and b is the number of decoder layers; the reported variation is sample standard deviation across seeds.
RLT combines a causal encoder, which processes known tokens in parallel without reading future positions, with a decoder that receives the previous token’s final output through a learned gate controlling the feedback contribution. The computation path grows across tokens while the number of encoder and decoder blocks evaluated per token remains fixed; attention work still grows with context length.
On swap-based S₅ tracking, composing permutations of five elements using identity and single swaps as inputs, the 4+4 split reaches 97.30±2.76% final-state accuracy at 256 operations after training on 32 operations, versus 0.85±0.30% for the Transformer. After 5,000 training steps, the 6+2 split reaches 93.36±5.69% on length-63 flat mod-5 expressions, arithmetic evaluated modulo five without brackets, versus 33.20±2.33%.
Feedback frequency controls a task-dependent tradeoff between accuracy and parallel processing: at the 4+4 split, updating feedback every four tokens preserves 98.99±1.66% accuracy on 64-bit parity but reduces 64-operation swap-based S₅ accuracy from 100±0% to 19.60±6.54%. Removing feedback leaves both tasks near chance from length 64 at every split.
The evidence comes from supervised synthetic tasks with three initialization seeds and shared data; the eight-layer RLT models contain 26.10–28.73M parameters versus 25.31M for the Transformer. Addition extrapolation remains poor, standard S₅ tracking with all 120 permutations as possible inputs stays near chance, and the paper does not evaluate natural-language modeling or reinforcement learning (RL), learning from rewards.

1 Introduction

Recurrent computation passes the result of one state update directly into the next, creating a dependency path whose depth increases with sequence length. RLT separates parallel contextual encoding from sequential state updates and studies two design choices: encoder–decoder depth allocation and the number of tokens between feedback updates.

The fixed-depth limitation for arbitrarily long S₅ composition is conditional on TC⁰ ≠ NC¹, a separation between circuit-complexity classes used to characterize computational power (Merrill et al., 2024).
Figure 1 shows that prompt and response tokens use the same recurrent transition; the prompt boundary does not reset the accumulated state.

Recurrence across prompt and response, with causal encoder memory and sequential decoder-state updates.

2 Method

The method defines RLT-1 with feedback at every token, then introduces RLT-2 with feedback shared within token chunks and RLT-0 without final-output feedback. All variants retain causal encoding and attention restricted to the available encoder prefix, providing controls for studying feedback frequency.

2.1 Sequence and state

An independent sequence begins with BOS, the beginning-of-sequence token, and uses a learned initial output s⋆. The complete decoder state is Hₜ = (sₜ, Cₜᴰ): the final decoder output and the per-layer key–value (KV) cache, stored attention projections of earlier positions.

Sliding-window attention (SWA), attention restricted to recent decoder positions, uses a window W ≥ 1 including the current token. Each layer therefore retains at most W−1 previous positions after an update.
Encoder and decoder depths are L_E and L_D, and both encoder representations and recurrent outputs have vector dimension d.

2.2 Causal encoder and memory

The encoder computes e₁:ₜ = Eθ(x₁:ₜ) with causal masking, enabling known positions to run together within each encoder layer. Its outputs are normalized and projected into global KV memory, which decoder cross-attention, attention from decoder representations to encoder memory, reads only through the current position.

A single memory group, G = 1, shares projected encoder KV across decoder layers; G = L_D permits separate projections for each decoder layer.
Global encoder memory grows with the prefix, while each decoder layer maintains a separate bounded SWA cache.

2.3 One transition for every token

Every observed or sampled token first merges its encoder representation with the preceding decoder output, then updates both components of the decoder state. The final output predicts the next token through a normalized linear readout and softmax, which converts output scores into probabilities.

The core transition is u_t = Merge(e_t, s_{t−1}), followed by H_t = Dφ(u_t; M_{≤t}, C^D_{t−1}, t). Prompt processing composes these same transitions in token order.
RLT-1 can be interpreted as a recurrent neural network (RNN), a model that repeatedly updates a carried state, whose cell contains the gate and full decoder. S4 and Mamba are state-space models with updates affine in the previous state once layer inputs are fixed; RLT-1 depends nonlinearly on that state, so their affine parallel-scan algorithms do not directly apply.
At fixed width, decoder depth, and SWA window, the decoder state is bounded in size; the complete model also retains growing encoder memory.

2.4 Prompt prefill and incremental decoding

Prompt prefill, processing known prompt tokens before generating a response, first computes encoder features and memory in parallel, then evaluates decoder states in order. Although the full prompt memory is available, the decoder at position t can read only encoder positions up to t.

During generation, each consumed token updates the encoder cache, appends encoder-memory KV, and advances the decoder state and every SWA cache.
Known tokens can be encoded in chunks when the encoder continuation cache is preserved, but RLT-1 still requires a decoder update for every token.

2.5 Merge and decoder blocks

The merge normalizes the preceding output, computes a sigmoid gate, with values between zero and one, from the current encoder representation and normalized state, and adds a scaled projected feedback vector to the encoder output. The main experiments use feedback scale α = 0.1.

The merge equations are r_{t−1} = RMSNorm_s(s_{t−1}), g_t = σ(W_g[e_t; r_{t−1}] + b_g), and u_t = e_t + αg_t ⊙ W_s r_{t−1}; RMSNorm is the state-normalization operator, and ⊙ denotes elementwise multiplication.
Each decoder block applies causal SWA, encoder-memory cross-attention, and a feed-forward network (FFN), a positionwise transformation. Residual additions add each sublayer’s input to its output.
Current-position KV is formed from the layer input before attention, avoiding a circular dependency. Figure 2 distinguishes this layerwise cache from the final output fed into the next token.

RLT-1 architecture for one token update, including the gated merge, local attention, global memory attention, and readout.

2.6 Recurrent depth and execution cost

After a prompt of T tokens and n consumed response tokens, the feedback path traverses (T+n)L_D decoder blocks, while each token evaluates L_E+L_D blocks. This counts a structural dependency path; the influence of earlier states also depends on gates, projections, and products of transition derivatives.

RLT-1 requires T sequential decoder updates during prefill, although independent sequences can batch their next updates together.
Fixed block count per token does not imply constant arithmetic cost: reading encoder memory becomes more expensive as context grows.

2.7 Feedback interval: RLT-1, RLT-2, and RLT-0

RLT-1 uses feedback interval B = 1; RLT-2 holds the feedback register fixed across each B-token chunk and replaces it with the final output only at a complete chunk boundary. RLT-0 removes the learned initial state, gate, state normalization, and feedback projection, directly supplying encoder outputs to the decoder.

Chunk boundaries are anchored at BOS. A partial final chunk preserves the preceding boundary register, and B = 1 reduces RLT-2 exactly to RLT-1.
For T known tokens and L_D ≥ 1, sequential decoder block stages are TL_D for RLT-1, ⌈T/B⌉L_D for RLT-2, and L_D for RLT-0, while all evaluate TL_D decoder blocks. Table 1 distinguishes scheduling depth from arithmetic work.
RLT-2 uses the same parameters for every B. A schedule that starts with large chunks and later reduces B is proposed but untested; every experimental chunk size is trained from scratch.

Execution costs of RLT-1, RLT-2, and RLT-0 at matched stack dimensions.

2.8 Training and state replay

Training uses teacher forcing, supplying correct preceding tokens, and cross-entropy loss on selected next-token targets. Full backpropagation through time (BPTT), differentiation through the recurrent history, follows recurrent outputs, decoder KV, and encoder memory even when a token’s target loss is masked.

After a parameter update, evaluating a sampled response under the current model requires rebuilding its state by replaying its history, unless an equivalent valid current-parameter state is available.
The appendices specify pretraining, supervised fine-tuning on selected assistant targets, and policy-gradient replay for reward-based learning, but the experiments evaluate supervised algorithmic learning only.

3 Algorithmic Experiments

Six tasks test how decoder allocation and feedback frequency affect performance beyond training lengths: addition, parity, flat and bracketed modular arithmetic, and standard and swap-based S₅ tracking. The original complete comparison uses 2,000 training steps; the main modular-arithmetic results use 5,000-step runs.

3.1 Models and evaluation

The eight-layer study compares untied RLT-1 splits 4+4, 5+3, 6+2, 7+1, and 8+0, with separate encoder and decoder weights, against GPT Layer 8, the paper’s decoder-only Transformer baseline. All use width 512, FFN width 1,365, and four attention heads; RLT uses SWA window eight, one shared encoder-memory group, and α = 0.1.

Initialization seeds 42, 43, and 44 share training streams and held-out examples. Global batch size is 512, microbatch size is 32, and execution uses four CPU threads and FP32, 32-bit floating-point arithmetic.
The AdamW optimizer uses (β₁, β₂) = (0.9, 0.95), weight decay 0.1, and gradient clipping at norm 1. Learning rate warms to 10⁻⁴ over 200 steps and decays to 5×10⁻⁶ using a cosine schedule over the run budget.
Parity trains on 3–40 bits; S₅ on 32 operations; flat mod-5 on odd expression lengths 3–39; bracketed mod-5 on lengths 3–40; addition on 1–8-digit operands. Formal tasks use 1,024 shared test examples per length, and addition uses 256 pairs per operand width.
Each run selects the checkpoint with minimum in-distribution (ID) validation example loss, measured within the training-length distribution, breaking exact ties by earliest step; test scores do not select checkpoints. Reported mean ± sample standard deviation describes three initializations on fixed shared data. TBPTT128, truncated BPTT configured with a span of 128, covers every training sequence here, so gradients are not truncated.
Equal layer counts do not match parameter counts or compute. The 8+0 model retains a recurrent gated merge despite having no decoder blocks.

3.2 Length extrapolation and decoder allocation

Length extrapolation depends on the task and depth split: 5+3 and 7+1 retain 100±0% parity accuracy at 256 bits, while swap-based S₅ benefits from more decoder layers and reaches 97.30±2.76% at 256 operations with 4+4. Figure 6 also shows that modular-arithmetic advantages occur at intermediate extrapolation lengths and weaken on longer expressions.

At 512 swap operations, 4+4 reaches 55.70±25.78% final-state accuracy versus 0.85±0.30% for the Transformer; 7+1 and 8+0 are already near chance at 256 operations.
After 5,000 steps, flat mod-5 at length 63 reaches 93.36±5.69% with 6+2 versus 33.20±2.33%; bracketed mod-5 at length 64 reaches 67.97±2.91% with 5+3 versus 46.71±1.21%. Several flat mod-5 configurations show substantial initialization variability.
Figure 7 reports earlier parity learning: at step 500, 6+2 reaches 99.44±0.98% validation accuracy versus 48.48±0.53%. A separate sixteen-layer experiment reaches perfect 256-bit parity for several splits, but uses only seed 42.

Length generalization across five encoder–decoder allocations and the eight-layer Transformer baseline.

Parity validation accuracy at 500 and 2,000 training steps, with individual seeds and mean ± sample standard deviation.

3.3 Feedback frequency and parallelism

At the fixed 4+4 split, Figure 8 shows that four-token feedback chunks preserve most 64-bit parity performance but sharply reduce swap-based S₅ extrapolation. RLT-1 scores 100±0% on both 64-bit parity and 64-operation swaps, while RLT-2 chunk4 scores 98.99±1.66% and 19.60±6.54%, respectively.

RLT-0 reaches 50.23±3.80% on 64-bit parity and remains near the 1/120 uniform-prediction reference on 64-operation swaps. Removing feedback also removes 787,968 parameters per split; RLT-2 retains the RLT-1 parameter count.
Chunking interacts with depth allocation: chunk4 4+4 scores 69.34±24.17% on 256-bit parity, while chunk4 7+1 and 8+0 are near chance already at 64 bits.
For swaps at 64 operations, chunk4 5+3 reaches 50.16±18.77%, compared with 19.60±6.54% for chunk4 4+4. Feedback frequency and decoder allocation therefore need to be assessed together for each task.

Feedback-frequency comparisons at a fixed 4+4 split for parity, swap-based S₅, and bracketed mod-5 arithmetic.

3.4 Addition, standard S₅, and scope

All models reach 100% teacher-forced accuracy on addition within the 1–8-digit training range, but the RLT-1 and Transformer means fall to 14.89–16.84% at 32 digits. Standard S₅ remains near the 1/120 final-state reference across feedback variants.

Addition accuracy measures answer tokens with correct preceding tokens supplied; complete autonomously generated answers are not evaluated.
The experiments measure supervised synthetic-task performance. Reinforcement-learning performance is explicitly unevaluated.

The paper relates RLT to models combining parallel encoders with recurrent decoders, models reusing encoder-derived memory, and models passing hidden states across tokens. RLT combines a separate encoder-memory store with final-output feedback through the full decoder and studies encoder–decoder depth allocation.

Feedback Transformer uses shared memory derived from layer representations, while Recurrent Transformer uses layerwise feedback. RLT feeds the preceding final decoder output into the next decoder input.
Full-bandwidth Transformer, T²MLR, a Transformer with temporal middle-layer recurrence, and Latent Recurrent Transformer also carry information across tokens, but use different feedback locations or parallel training approximations. RLT specifies sequential history replay and full BPTT.
Models with continuous latent computation insert or replace token-level reasoning steps with hidden-state computation; RLT updates once per ordinary token. RLT-2 resembles block-level recurrent architectures but uses the last token’s decoder output as its boundary state without dedicated memory tokens.
Models that loop over depth repeatedly apply computation at one position; RLT advances recurrence across tokens. Encoder–decoder weight sharing is optional.

5 Conclusion

The conclusion identifies decoder allocation and feedback interval as two architectural choices supported by the algorithmic experiments. Per-token feedback enables strong parity and swap-based S₅ extrapolation, while chunking permits known-token parallelism with a task-dependent reduction in accuracy.

The proposed large-to-small chunk training schedule remains untested.
Project page: https://github.com/yifanzhang-pro/recurrent-looped-tranformer

Appendix

A Architecture and Execution Details; A.1 Sharing encoder and decoder weights: The tied reference sets L_E = L_D = L and shares compatible attention projections and corresponding FFN weights between encoder self-attention and decoder SWA. Cross-attention uses separate query/output and memory projections; encoder and decoder normalizations remain separate, and merge and readout are separate modules. Weight sharing retains separate activations and caches; the principal eight-layer experiments use untied weights.
B Computational Properties; B.1 Prompt–response consistency; B.2 Prefill work and sequential depth: Proposition B.1 proves invariance to the prompt–response split under fixed parameters, tokens, positions, and initial state, exact arithmetic, equivalent causal encoder execution, identical SWA windows and cache updates, and deterministic decoder operations. Prefill work sums encoder work, memory projection, and T decoder transitions; dense attention retains quadratic sequence-length terms, while sequential transitions can reduce hardware utilization. Inference storage includes growing encoder caches and memory, bounded decoder SWA history, and the current recurrent output.
C Hardware Execution; C.1 Batching independent sequences; C.2 Memory traffic and parameter reuse; C.3 Training memory and numerical agreement: Independent sequences can batch their next decoder updates while preserving separate states and caches. Shared encoder memory and fewer memory groups reduce projection work and storage; kernel fusion, combining operations in one execution kernel, and keeping weights resident on the accelerator are implementation opportunities. Activation checkpointing trades recomputation for stored intermediate activations, and numerical agreement between sampler and trainer requires aligned precision, positional conventions, cache construction, reductions, and stochastic behavior.
D Training Objectives; D.1 Autoregressive pretraining; D.2 Supervised fine-tuning; D.3 Policy gradients and replay: Pretraining averages next-token loss within each sequence, while supervised fine-tuning selects assistant targets without skipping other tokens’ state updates. Policy replay rebuilds current-parameter states and compares current action probabilities with recorded behavior probabilities, the probabilities used when sampling the actions. Exact importance sampling, correcting for differences between sampling and target policies, requires positive behavior probability wherever the target assigns positive probability; top-k and top-p sampling truncate the candidate distribution and generally violate this condition for an untruncated softmax target, while tokenwise clipping of probability ratios can introduce bias.
E Multi-turn Inference and Cache Reuse: A conversation forms one continuous token history, including user messages, tool results, role delimiters, and assistant tokens. Resuming requires encoder continuation caches, encoder memory, complete decoder state, positions, window conventions, and matching model parameters; identical sampled continuation also requires the sampler’s random state. Editing an earlier token invalidates later cached states, which must be recomputed from a valid preceding checkpoint.
F Execution Procedures; F.1 Recurrent prompt prefill; F.2 Generation and new external inputs; F.3 Pretraining and SFT; F.4 Current-policy RL replay: The procedures specify parallel causal encoding followed by ordered decoder replay, one update per consumed token, and document-isolated states for packed sequences. SFT denotes supervised fine-tuning, and EOS denotes the end-of-sequence token, included in replay when sampled. An emitted but unconsumed final token must be recorded and consumed once before continuation; exact full-BPTT training replay requires gradients through prompt computation and invalidates parameter-dependent caches after each weight update.
G Causality and Gradient Paths; H When Cached States Can Be Reused: Proposition G.1 proves causality under deterministic execution from causal encoding, prefix-restricted memory, and causal SWA. The gradient analysis includes paths through recurrent outputs, decoder KV, and encoder-side tensors carried across a continuation boundary; detaching these tensors from the computation graph can remove gradient terms while preserving forward values. Cached prefixes can replace forward replay only when parameters, tokens, positions, initialization, windows, and execution settings match, and full BPTT additionally requires retaining or recomputing the prefix computation graph, including earlier operations that used subsequently evicted SWA entries.
I RLT-2: Chunk-Parallel Hidden-State Feedback; I.1 Chunk operator and causal masks; I.2 Training and prefill algorithm; I.3 Incremental generation and partial chunks; I.4 Training and inference efficiency: Chunk boundaries remain anchored to sequence start, with BOS counted as position one, and neither positions nor SWA caches reset at boundaries. Known chunk positions run together within each layer; only complete boundaries replace the feedback register, and full BPTT follows boundary states, cross-chunk decoder KV, and encoder memory. A partial chunk preserves its preceding register, changing B or its boundary anchor invalidates continuation state, and generation remains tokenwise; the broader throughput, latency, memory, numerical-equivalence, and gradient benchmark is proposed rather than reported as completed.
J Experimental Details; J.1 Complete six-task study at 2,000 steps; J.1.1 Models, tasks, and evaluation protocol; J.1.2 Parity: learning speed and initialization variability: The original study contains 108 runs, six architectures × six tasks × three seeds, each consuming 1,024,000 training examples, with validation every 100 steps. All three 6+2 parity seeds reach 100% validation accuracy by step 600, while other splits show greater early variability. These runs are distinct from the later 5,000-step modular-arithmetic comparisons.
J.1.3 Modular arithmetic with and without brackets; J.1.4 Addition and permutation state tracking; J.1.5 Length generalization; J.1.6 Training stability and comparison scope: At 2,000 steps, flat mod-5 4+4 validation scores are 17.58%, 19.53%, and 98.96%, illustrating large initialization effects, while bracketed-expression model means are closer. At 512 swap operations, 4+4 obtains 91.16±6.09% prefix-token accuracy, averaging correctness over intermediate states, but 55.70±25.78% final-state accuracy; whole-sequence accuracy requires every intermediate prediction to be correct. A parity 5+3 seed temporarily falls from 100% at step 500 to 48.05% at step 600 before recovering, and the appendix notes that the Transformer comparison changes attention structure, parameters, compute, and feedback together.
J.2 Parameter counts; J.3 Implementation of RLT-0; J.4 Seed aggregation and metric definitions: RLT-1 parameter counts are 28.73M, 28.20M, 27.68M, 27.15M, and 26.10M for splits 4+4 through 8+0, versus 25.31M for GPT Layer 8. The rlt_parallel_tiny 4+4 control has 27,938,944 parameters and was merged in nanogptpro-dev PR #164. Seed aggregation uses the arithmetic mean and sample standard deviation over all three seeds at the same step; addition scoring includes answer digits, spaces, brackets, and EOS, while S₅ distinguishes final-state from prefix-token accuracy.
J.5 Reproducing the completed training snapshot; J.6 Length-generalization protocol and supplementary metrics; J.7 Token-accuracy trajectories at lengths 32 and 33: Formal-task data seeds are 20260914 for training, 20260915 for validation, and 20260916 for testing; the original addition snapshot uses data seed 42. The implementation snapshot is based on commit d1a4516, with histories frozen on September 17, 2026; manifest.json and configs.json in data/experiments-depth8/ record provenance. The extrapolation study contains 47 shared test sets and 846 model–seed–length evaluations, with results and verification records in data/experiments-depth8-generalization/; fixed-length trajectories use length 33 for flat expressions and 32 for the other formal tasks.
J.8 Sixteen-layer parity: best and final checkpoints: Ten seed-42 runs train for 2,000 updates with global batch 1,024 and microbatch 32. Accuracy is unchanged in 74 of 80 model–length comparisons between the best ID-loss and final checkpoints; splits 8+8, 9+7, 11+5, and 16+0 retain 100% at 256 bits in both evaluations, versus 49.41% for GPT Layer 16. The checkpoint comparison is limited to this single initialization seed.
J.9 Three-seed comparison of feedback variants; J.10 Mod-5 feedback variants at 5,000 steps: The expanded study reports end-of-training validation for 288 runs, including 96 modular-arithmetic runs trained for 5,000 steps; addition extrapolation for RLT-0 and RLT-2 is not evaluated. At 4+4 and bracketed length 64, RLT-1 scores 64.10±3.49%, chunk4 62.76±4.42%, RLT-0 39.45±0.61%, and GPT Layer 8 46.71±1.21%. Seed-42 CPU timing over steps 1,001–2,000 finds chunk4 training steps 2.27 times as fast as RLT-1 and RLT-0 4.17–4.30 times as fast, excluding validation, checkpoint writes, and logging.
J.11 Addition feedback scale at a fixed learning-rate schedule: A 45-model comparison tests α = 0.03 at every split and α = 0.01 at every split except 7+1, alongside α = 0.1 and GPT controls. Across 63 matched comparisons with α = 0.1, 37 means rise and 26 fall, with changes from −3.86 to +5.60 percentage points; only three differences exceed both sample standard deviations. No tested reduction improves every operand width for any split, and at 32 digits the RLT means remain 14.44–15.91%, versus 16.84±1.45% for GPT Layer 8.

Brief Thoughts

The feedback controls strengthen the evidence for recurrence: removing final-output feedback eliminates the parity and swap-based S₅ extrapolation gains, while parameter-identical chunking shows that update frequency affects the two tasks differently. Shared data, three-seed reporting, and checkpoint selection independent of test performance support these comparisons, although RLT-0 also removes merge parameters.

The results leave practical scaling unresolved. Sequential decoder dependencies during training and prefill, unequal model compute and parameter counts, single-seed CPU timing, poor addition extrapolation, and failure on standard S₅ limit the evidence to the tested algorithmic settings; large-scale language performance and the proposed chunk-size training schedule remain unevaluated.