At a glance
Negative bias initialization can steer small networks toward more interpretable neurons without worsening loss in several controlled sparse-feature settings. Wider hidden layers also help, at higher compute cost. The result is task-dependent: absolute-value models need a slower bias-decay schedule, and the study does not establish the same tradeoff for language models. [2]
1 Introduction
The paper separates the motivation for interpretable neurons from experiments in which useful input features are known exactly.
1.1 Motivation
A neuron is easier to interpret when it responds to one useful feature rather than an unrelated mixture. These experiments ask whether training choices can favor that organization without sacrificing task performance.
1.2 Overview
Two learned linear maps surround a nonlinear hidden layer. Three synthetic tasks separate input dimension, number of possible features, and number of nonlinear units. The study compares loss basins, tests negative biases and width, then analyzes the most monosemantic feature-decoder models.
2 Setup
The synthetic data, three target tasks, and adjustable-width architecture define the controlled setting used throughout.
2.1 Features
Each of 512 possible features is usually zero and otherwise uniform on [0,1]. Feature occurrence is either equally frequent or follows a power law proportional to rank raised to −1.1. A fixed random projection compresses these sparse feature vectors into fewer dimensions.
2.2 Tasks
The tasks are feature recovery, conversion between random projections, and recovery of absolute feature values. Knowing the generating features makes neuron selectivity measurable.
2.2.1 Feature Decoder
The decoder reconstructs the original feature vector from its fixed lower-dimensional random projection using squared-error loss. Recovery depends on the input’s sparsity and projection, rather than being possible for arbitrary dense vectors.
Equation 1
| (1) |
2.2.2 Re-Projector
The re-projector predicts a second fixed random projection from the first. This changes the output basis while retaining the same underlying sparse-feature structure.
Equation 2
| (2) |
2.2.3 Absolute Value Calculator
The absolute-value task projects the difference of two independently sampled feature vectors, then predicts its elementwise absolute value. It requires both recovering information and applying a nonlinear transformation.
Equation 3
| (3) |
2.3 Architecture
A biased linear encoder feeds ReLU or GeLU units, followed by an unbiased linear output layer. Hidden width varies independently of input and feature dimensions, unlike a tied-width reconstruction model.
Equations 4–6
| (4) | ||||
| (5) | ||||
| (6) |
3 Measuring Monosemanticity
The selectivity score divides a neuron’s largest positive response to any unit-strength single-feature input by its summed positive responses. The paper labels scores above 0.999 monosemantic, adapting the test for signed inputs in the absolute-value task. This diagnostic examines controlled single-feature probes, not all possible feature mixtures.
Equation 7
| (7) |
4 Results
Default experiments use 512 features, 64 input dimensions, feature frequency 1/64, and thus eight active features on average. Many decoder runs achieve similar loss with substantially different selectivity. Negative biases and sufficient width favor interpretable units, but all three tasks do not share an identical loss–interpretability tradeoff.
4.1 Task: Feature Decoder
The feature-decoder experiments examine optimization basins, bias initialization, activation regularization, hidden width, and feature sparsity separately.
4.1.1 Monosemantic and Polysemantic Basins
At hidden width 1,024, learning-rate changes produce markedly different neuron selectivity with nearly identical final decoder loss. Negative biases characterize selective neurons, while positive biases characterize many mixed units. Similar behavior occurs with power-law feature frequencies. Distinct basins are a plausible interpretation, not a demonstrated absence of connecting low-loss paths.
4.1.2 Negative Bias and Training Protocol
Biases start near −1 and decay until units activate; decay is disabled for the final half of training. This steers ReLU models toward selective solutions across tested learning rates. Effective starting scales depend on feature frequency, and GeLU has a preferred scale near −1. A persistent group of roughly input-dimension-many positive-bias neurons remains polysemantic.
4.1.3 Comparison with L1 Regularization
Negative-bias initialization obtains performance and selectivity comparable to the best tested L1 activation penalty. Bias decay changes the optimization path; L1 changes the objective. Both introduce task-dependent tradeoffs and may be combined rather than treated as universally competing methods.
4.1.4 Varying Nonlinear Dimension
Increasing hidden width generally raises the number of selective neurons and lowers loss until width exceeds the feature count, after which loss gains diminish. At some intermediate widths the more monosemantic solution is slightly worse, while sufficiently wide models can improve both. Fixed random sparse connectivity substantially worsens loss at matched compute in the attempted efficiency experiment.
4.1.5 Feature Sparsity
As more features co-occur, selectivity decreases smoothly around the regime where active-feature count approaches input dimension. Very rare features require slower bias decay and longer training; otherwise the bias disappears before they are learned. The observed behavior is consistent with compressed-recovery limits but does not constitute a general recovery theorem.
4.2 Task: Random Re-Projector
The random re-projection task also yields roughly one selective neuron per feature under negative-bias initialization across the tested learning rates. This suggests the effect is not solely a consequence of an explicitly feature-aligned output basis.
4.3 Task: Absolute Value
On absolute values, more monosemantic models achieve lower loss, and negative initialization alone is insufficient. Lower bias decay or a higher learning rate helps. The authors hypothesize that this harder task needs more time within a useful bias range; preliminary negative-bias regularization is left for further study.
5 Mechanistic Interpretability
The mechanistic analysis focuses on highly monosemantic feature decoders, splitting units approximately into negative-bias selective and positive-bias mixed groups. It does not explain the most polysemantic models or every task.
5.1 Monosemantic Neurons
A single selective ReLU suppresses weak apparent features that may be projection interference, then estimates stronger features. Two selective neurons provide an additional bend, improving the transition to near-faithful recovery at high amplitude. Lower feature frequency shifts the thresholds downward, consistent with reduced interference rather than redundant duplicate encoding.
5.2 Polysemantic Neurons
The remaining positive-bias units approximately supply a low-rank linear correction to all outputs. In the inspected model, its nonzero singular values lie around 0.99–1.03, and bias contributions nearly cancel. This helps explain why about one input-dimension’s worth of mixed units remains, though the exact excess count is unexplained.
5.3 Summary
The interpreted decoder combines thresholded feature-specific estimates with a shared approximately linear correction. This accounts for selected high-selectivity models; the algorithm of highly polysemantic solutions remains an open question.
6 Conclusions
The findings motivate optimization strategies that favor interpretable representations, but the current bias schedule is fragile and width costs extra computation. The equal-loss result depends on the synthetic task and available capacity. Better schedules, efficient wide layers, and interpretation of mixed solutions remain future work.
Author Contributions
Adam S. Jermyn designs and runs the experiments and leads writing. Nicholas Schiefer contributes training and bias-method ideas; Evan Hubinger contributes interpretation and activation-function analysis.
Acknowledgments
The authors acknowledge discussions, suggested tasks and comparisons, and computational support from colleagues and Hofvarpnir Studios.
Appendix A Training Details
Training uses LAMB with batch size 2²³ divided by hidden width and a cosine learning-rate schedule. The retained table specifies all run families, feature distributions, dimensions, learning rates, decay rates, and bias choices. Training examples are newly generated each batch, so expected training and test losses coincide under the same synthetic distribution.
| Batch | Task | Activation | Feature Dist. | Learning Rate | Decay Rate | Bias Offset | L1 Reg. | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LR1 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | 1/64 | Variable | 0 | 0 | 0 |
| LR2 | Decoder | ReLU | Power-law | 512 | 64 | 1024 | 1/64 | Variable | 0 | 0 | 0 |
| LR3 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | 1/64 | Variable | 0.03 | -1 | 0 |
| B1 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | 1/16 | 0.003 | 0.03 | Variable | 0 |
| B2 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | 1/32 | 0.003 | 0.003 | Variable | 0 |
| B3 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | 1/64 | 0.003 | 0.003 | Variable | 0 |
| B4 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | 1/128 | 0.003 | 0.003 | Variable | 0 |
| B5 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | 1/256 | 0.003 | 0.003 | Variable | 0 |
| LR4 | Decoder | ReLU | Power-law | 512 | 64 | 1024 | 1/64 | Variable | 0.03 | -1 | 0 |
| B3 | Decoder | GeLU | Uniform | 512 | 64 | 1024 | 1/64 | Variable | 0.03 | Variable | 0 |
| E1 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | Variable | 0.003 | 0.03 | -1 | 0 |
| E2 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | Variable | 0.003 | 0.01 | -1 | 0 |
| E3 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | Variable | 0.003 | 0.003 | -1 | 0 |
| E4 | Decoder | ReLU | Uniform | 512 | 64 | 1024 | Variable | 0.003 | 0.001 | -1 | 0 |
| K0 | Decoder | ReLU | Uniform | 512 | 64 | Variable | 1/64 | 0.007 | 0 | 0 | 0 |
| K1 | Decoder | ReLU | Uniform | 512 | 64 | Variable | 1/64 | 0.007 | 0.03 | -1 | 0 |
| K2 | Decoder | ReLU | Power-law | 512 | 64 | Variable | 1/64 | 0.007 | 0.03 | -1 | 0 |
| RG1 | Decoder | ReLU | Uniform | 512 | 64 | 1/64 | 0.005 | 0.03 | -1 | Variable | |
| RP1 | Re-Projector | ReLU | Uniform | 512 | 64 | 1/64 | Variable | 0.03 | -1 | 0 | |
| LR5 | Abs. | ReLU | Uniform | 512 | 64 | 2048 | 1/64 | Variable | 0.03 | -1 | 0 |
| D1 | Abs. | ReLU | Uniform | 512 | 64 | 2048 | 1/64 | 0.007 | Variable | -1 | 0 |