Literature
Display
← All papers

Sparse Crosscoders for Cross-Layer Features and Model Diffing

Jack Lindsey · Adly Templeton · Jonathan Marcus · Thomas Conerly · Joshua Batson · Christopher Olah

· Transformer Circuits Thread ↗

Interpretability5 min text · CondensedOriginal paper ↗
Contents

Overview

Crosscoders learn sparse features that jointly reconstruct activations from several layers or models. This preliminary report tests whether shared dictionaries reduce duplicate features, track representation changes across layers, and distinguish base models from fine-tuned versions. It finds useful shared structure, while leaving feature quality and mechanistic faithfulness incompletely validated. [1]

(1) Motivating Examples

Two examples motivate jointly modeling layers: features distributed across several computations, and features that simply persist through the residual stream.

(1.1) Cross-Layer Superposition

The residual stream permits computation to be spread across layers that act partly in parallel. If a feature draws contributions from several such layers, fitting each layer separately can miss the shared organization. Joint dictionary learning is a natural candidate for resolving this cross-layer superposition, although this motivating construction is not evidence that every distributed feature has that cause.

(1.2) Persistent Features and Circuit Complexity

A feature can persist across many layers without being recomputed. Separate SAEs represent each occurrence individually, creating long chains of apparently distinct nodes. A crosscoder can assign one feature to those occurrences and potentially simplify the graph. This compression also changes the causal description, so a shorter graph must be checked against the original mechanism.

(2) Crosscoder Basics

The encoder sums learned contributions from the selected activation spaces before a ReLU. Separate decoder vectors then reconstruct each space. Training minimizes reconstruction error plus feature activity weighted by the sum of per-layer decoder-vector norms. This L1-of-norms penalty makes loss comparable with the sum of single-layer SAE losses and encourages layer-specific features. An L2-of-norms alternative favors broadly shared features and can improve the global reconstruction–sparsity frontier; all reported experiments use the L1 version.

Equations

f(xj) = ReLU(∑l∈LWenclal(xj) + benc)f(x_j) ~=~ \text{ReLU}\left(\sum_{l\in L} W^l_{enc} a^l(x_j) ~+~ b_{enc}\right)

al′(xj) = Wdeclf(xj) + bdecla^{l'}(x_j) ~=~ W^l_{dec} f(x_j)~+~b^l_{dec}

L=∑l∈L∣∣al(xj)−al′(xj)∣∣2+∑l∈L∑ifi(xj)∣∣Wdec,il∣∣L = \sum_{l\in L} ||a^{l}(x_j)-a^{l'}(x_j)||^2 + \sum_{l\in L} \sum_i f_i(x_j) ||W^l_{dec, i}||

∑l∈L∑ifi(xj)∣∣Wdec,il∣∣=∑ifi(xj)(∑l∈L∣∣Wdec,il∣∣)\sum_{l\in L} \sum_i f_i(x_j) ||W^l_{dec, i}|| = \sum_i f_i(x_j)\left(\sum_{l\in L} ||W^l_{dec, i}||\right)

(2.1) Crosscoder Variants

Variants differ in which layers or models they connect, whether they reconstruct residual states or layer outputs, and whether later activations may help encode earlier ones. Causal masks restrict allowed connections; strictly causal versions as described do not capture attention computation by themselves. Combining interpretable MLP replacements with separately treated attention is proposed for future circuit analysis.

(3) Cross-Layer Features

The following experiments evaluate cross-layer dictionaries, inspect their feature geometry, and compare masked architectures.

(3.1) Performance and Efficiency of Crosscoders vs. SAEs

An acausal crosscoder jointly reconstructs all layers of an 18-layer model, with each layer normalized separately. At fixed total dictionary size it achieves lower reconstruction-plus-sparsity loss than separate SAEs. At large matched training budgets, however, reaching the same loss costs about twice the compute. Counting active features globally rather than separately at every layer reveals a much more compact description: one shared feature replaces repeated layer-wise occurrences. These are distinct notions of efficiency.

(3.2) Analysis of Crosscoder Features

Most sampled features peak at particular layers and decay across neighboring ones, but some persist widely. Their decoder directions are more stable than random directions yet often drift substantially. Unlike a shared-direction dictionary, crosscoders align features by coactivation while allowing layer-dependent directions. Gradual strength changes are consistent with cross-layer superposition but could also reflect amplification; circuit analysis is needed to distinguish them. Qualitative interpretability is reported only anecdotally.

(3.3) Masked Crosscoders

Local and causal masks restrict which layers each feature may read and reconstruct.

(3.3.1) Locality experiments

Restricting each feature to a local layer window trades performance for cost smoothly. The experiments find no especially efficient intermediate window: locally masked crosscoders perform similarly to smaller global crosscoders at matched compute.

(3.3.2) Causality experiments

A weakly causal crosscoder encodes at one layer and decodes that layer and later ones. Its compute efficiency lies between independent SAEs and global acausal crosscoders, while its dictionary-size efficiency trails the global version by a factor of three to four. Sparse decoder kernels could alter this comparison but were not tested here. Strictly causal MLP-output predictors show a mixture of local, global, and intermediate feature spans.

(3.3.3) Pre/post MLP crosscoders

A crosscoder that reads the pre-MLP residual state and reconstructs both that state and the MLP output separates input-only, shared, and newly computed features. Encoder–decoder products describe candidate inputs to the new features; illustrative cases combine specific words into a broader concept such as uniqueness. Shared features often change direction across the MLP, potentially explaining some cross-layer drift.

(4) Model Diffing

The same joint-dictionary idea applies to activations from different models. Shared features offer correspondence, while asymmetric decoder norms suggest model-specific features. The initial examples are promising but many purportedly exclusive features remain difficult to interpret.

(4.1) Model Comparison and Diffing Background

Earlier comparisons study whole representations through distance geometry, stitching, and canonical correlation, or compare individual neurons and dictionary features. Crosscoders target a fine-grained shared vocabulary that could support comparisons of features and their downstream roles. Prior evidence that base-model SAEs often transfer to fine-tuned models motivates searching for a smaller changed subset. [41]

(4.2) What Kind of Comparisons are Possible?

Potential comparisons include training checkpoints, fine-tunes, independent runs, model sizes, architectures, and transformed inputs. Shared features could expose changes in feature availability, geometry, or downstream use. This report actually tests only Sonnet fine-tuning and a three-size cross-layer comparison; the broader applications are proposals.

(4.3) Model Diffing Sonnet Finetuning

A one-million-feature crosscoder compares the middle residual layer of Claude 3 Sonnet with its base model. Decoder norms form shared, base-specific, and fine-tune-specific clusters, with about 4,000–5,000 features exclusive to each model. Selected examples concern refusal, code review, personal questions, and roleplay, but most exclusive features are not immediately understandable. Most shared decoder directions align strongly; several thousand do not, an unexplained pattern rather than confirmed new functions.

(4.4) Model Diffing Over (Layer, Scale)

A crosscoder spans ten evenly spaced layers from each of three model sizes. Factoring each feature’s thirty decoder norms into four nonnegative components groups early layers, late layers, and two different middle-layer regimes. This suggests changes in middle-layer representations with scale, but the features responsible have not yet been examined in detail.

(5) Discussion

Crosscoders open new comparisons while raising questions about the relation between compact explanations and the model’s actual computation.

(5.1) Questions with Fresh Traction

Open questions include when features form, whether their directions drift during training, how reproducible they are, and whether scaling adds new concepts or repacks existing ones. Crosscoder comparisons provide possible tools for these questions, contingent on resolving the mixed interpretability results.

(5.2) Literal vs Isomorphic Models

Adding residual error terms can reproduce the original activations exactly, but does not establish that the crosscoder’s causal graph matches the network’s mechanism. Correlated signals may be represented as one shared feature even when the model computes or copies them separately. Error terms can also remain large and opaque. Interventions and out-of-distribution tests are needed before treating a simplified representation as mechanistically faithful.

Acknowledgments

The authors acknowledge colleagues and external researchers for discussions, technical feedback, and ideas that shaped the cross-layer and model-comparison experiments.

Concept demonstration

Open image ↗