Can an explanation predict an intervention?

The observation

We averaged the change caused by a capital demonstration at the final token, before layer 8, across ten fitting countries. Adding this fixed vector to bare questions improved correct capital answers from 0/10 to 10/10 on a separate replication set, then to 9/10 on ten fresh countries.

Passing the vector directly to the final readout produced no correct answers on either set. Downstream computation matters. But the vector is not yet a clean explanation: unrelated prefixes sometimes supply useful vectors, too.

All five text prefixes have the same token count. Position-only shifts the query’s position embeddings. Output-only bias averages changes to vocabulary logits on the same fitting countries. Vector norms differ; these controls do not isolate a unique semantic direction.

5,120 measured interventions

Try to break the explanation

The eight switches control final-token attention and MLP outputs in layers 8–11. A checked component recomputes normally. An unchecked component receives its output from the same query without the vector. The vector remains present in every condition.

The frozen predictor adds the retained components’ contributions from the fully steered run, then applies exact final LayerNorm and unembedding. The actual model lets retained components respond to the intervention. The response-aware options recompute selected MLPs on the changed residual. All predictors use the same query; this is prediction across interventions, not prediction without access to the original model.

Modelling the response improves prediction

After locating a substantial response at MLP11, we tested predictors that recompute either that MLP or all four downstream MLPs. Attention outputs still use their fully steered values. The predictors use the same donor runs and additional MLP computation, with no parameters fitted to subset outcomes.

The old lattice is retrospective model development. New scope tests contain 210 previously unmeasured nontrivial interventions on thirty previously seen prompts. Their two easy endpoints are excluded. Modelling only MLP11 slightly worsens language predictions. More computation is used; an efficiency advantage has not been established.

Does the effect survive a new question?

We reused the fixed vectors on new prompt forms and primary-language questions for the ten replication countries. No vectors were refitted. These small diagnostic sets test the scope of the interpretation.

Language questions use conventional primary-language labels, without claiming exclusivity. Tokenization exclusions are recorded before inference. Language-query results measure correct languages, not capitals.

What this establishes—and what remains open

This is a reproducible test of one proposed explanation in GPT-2 Small. The subset audit and scope tests were designed after inspecting the first experiment. The validation countries were fresh for the first study; their new subset outcomes were held out for the subsequent audit. They are not a second independent entity holdout.

Clamped states need not be natural model states. Neither a successful steering vector nor an adaptive response proves a mechanism used in ordinary generation. To make this work consequential, the explanation must predict useful interventions across tasks and models, with a clear advantage over simpler behavioral baselines.

Reusable activation vectors and conditional circuit effects have substantial prior work: Function Vectors, MIB, and Conditional Co-Ablation. This study tests a local claim; it does not introduce those ideas.