Overview
This preliminary note studies weight superposition: compressing features into fewer dimensions also entangles the connections between them. Toy experiments reproduce patterns seen in language-model dictionaries and compare possible definitions of interference. These are exploratory findings, not a solved method for globally interpreting full models [1].
Introduction
Even a correct feature dictionary can produce apparent connections that arise from representational overlap. Lifting neuron weights into feature coordinates therefore does not automatically reveal a clean computational circuit.


Should we care about interference weights?
Interference is physically present and can affect robustness, including adversarial inputs. Whether it can be ignored when studying learned objectives is less clear: the authors tentatively distinguish incidental interference from connections favored by training, but explicitly regard that distinction as uncertain. Imperfect features, new learning during optimization, and interactions between weights complicate the claim.
Revisiting Toy Models
The original superposition toy model can be reinterpreted as independent identity computations compressed through a small shared bottleneck [2]. Its effective feature-to-feature weight matrix allows direct study of signal and interference.
Equations

Reproducing Phenomenology from Towards Monosemanticity
Feature-to-logit weights in a one-layer transformer pass through compressed activation spaces. This predicts interference even when individual features are interpretable, offering an explanation for weight distributions reported in Towards Monosemanticity [3].

Toy Model Setup
The initial example compresses 100 features into 20 residual dimensions with feature density 0.02. It is deliberately undertrained because this better resembles the motivating language-model observations; converged cases appear in an appendix.

Interference Weights and Decomposition
Two definitions are compared: deviation from an ideal weight matrix optimized in expanded feature space, or individual weights whose removal does not worsen loss. These are different decompositions. The experiments mainly use each weight’s marginal effect on loss as ground truth.


Weight Histogram
Weight histograms show a large interference mass alongside useful connections, qualitatively resembling real feature-to-logit histograms. Colors and aggregation differ between the toy and language-model plots, so their agreement is qualitative.


Comparing Two Models with a Scatter Plot
Across random seeds, useful identity connections agree while interference varies. Converged models show stronger geometric structure but preserve this separation between stable useful weights and seed-dependent interference.


Getting More Realistic Phenomenology
A harder example is needed because real useful and interference weights overlap in magnitude. A simple small-weight threshold would discard useful connections or retain interference.

A More Sophisticated Toy Model
An untied encoder–decoder approximates a more complex random block-structured circuit. The illustrated case compresses 128 features into 16 dimensions across eight blocks. It reproduces overlap and seed variability, but the chosen regime does not preserve every desired property at full convergence.
Equations



What Should We Do about Interference Weights?
Exact optimization or ablation-based definitions scale quadratically with feature count and are too expensive globally. Cheap candidates include weight magnitude, expected attribution, target-weighted attribution, and activation frequency. Evaluate not only classification precision but how much beneficial loss effect is retained. Summed individual effects are only a proxy for jointly pruning weights.


Conclusion
Interference may be a major obstacle to global circuit analysis, whereas prompt-specific attribution graphs partly avoid it. Toy ground truth and sampled real-model ablations offer ways to develop cheaper filtering rules; no scalable definitive solution is demonstrated.
Author Contributions
Chris Olah led experiments, figures, and writing. Nick Turner investigated detection heuristics in other settings, and Tom Conerly contributed exposition.
Acknowledgments
The report builds on earlier superposition research and feedback from colleagues. Its account credits the prior observations and discussions that motivated these experiments.
Appendix 1: Non-Linear Weight Sharing
Compressing computations need not make model parameters a linear sum of independent components. Two antipodal features can share a single multiplying weight, and partial sharing creates more complicated cases. This challenges a naive compressed-sensing or additive-parameter view [7].

Appendix 2: More Fully Converged Histograms and Isotropic Superposition Geometry
Fully converged isotropic models produce structured weight distributions: density 0.25 yields pentagonal geometry, whereas density 0.001 shows a different apparent upper bound on interference. The main figures’ undertraining is therefore a substantive modeling choice.


Appendix 3: Correlated Feature Toy Models
With two mutually exclusive groups of correlated features, a low-rank component excites the active group and suppresses the other. Removing this component harms loss even though it partly compensates for interference. Useful structure need not reduce to independent pairwise weights.



Appendix 4: Different Definitions of Interference Weights
Individual ablations can be misleading when weights jointly carry a signal, and an optimized ideal model may learn connections absent from the compressed model. Learning sparse masks on existing weights avoids some of this problem [5]. The appendix treats these definitions as competing research choices, with no settled canonical answer.