Recovering the functions inside a model

[ 01 · a gate is a pole ]

A recurrent cell keeps a state and updates it each step: a fraction of the old state, plus what the input adds. That fraction is the forget gate.¹ Near one, the cell remembers far back. Near zero, it forgets at once. Nothing else sets the timescale.

[ 02 · poles in the weights ]

For a state-space model such as S4 or Mamba, the poles are the eigenvalues of the state matrix, already sitting in the weights.² You can read how far back each layer remembers without running the model.

[ 03 · reading a real model ]

Reading mamba-130m straight from its weights gives a per-layer memory spectrum, from sub-token (instant decay) to near-integrator. The long-context memory is concentrated in a few mid-to-late layers rather than spread evenly, and that location holds from 130m to 1.4b even as the concentration softens with scale.

[ 04 · from reading to causation ]

Perturbing a single layer shows the weight-read timescale predicts how long a signal actually persists. Ablating the slow layers selectively destroys the model's use of far context, while ablating equally-deep fast layers does not.⁴ Each surviving result becomes an edge in a causal graph, scoped to the input class, metric, and intervention it was measured under.

[ 05 · where the lens stops ]

This is a memory lens, not a computation lens. A pole describes how the past decays. Attention does content-addressed selection, which no pole can express.⁵ Knowing where the lens stops is part of using it, and it is what the autoresearch loop turns into the next experiment.