1. Introduction
Replacing the transformer MLP activation with a softmax linear unit, followed by LayerNorm, makes more neurons appear understandable on brief inspection while preserving the tested language-model performance. The study reaches 50-billion-parameter models. It also finds that normalization can restore information hidden in small, less interpretable activations; clearer top activations therefore do not mean that the model has eliminated superposition. [7]
2. Key Results
In one- through forty-layer models, the fraction of neurons receiving a plausible quick interpretation rises from roughly 35% to 60%; at sixty-four layers the improvement is smaller. Test loss and downstream evaluations remain broadly comparable to the baseline. Qualitative examples suggest early token combination, middle-layer concepts, and late token production. These findings motivate architectural interpretability research, subject to the later caveat about the evaluation metric.
3. Background
The background explains why understanding nonlinear MLPs requires interpretable activation components, and why polysemantic neurons may reflect a useful representation strategy rather than a removable accident.
3.1 The Importance of Understanding Activations
Weights describe operations on activations, so interpreting their computation generally requires knowing what those activations represent. Attention-only models sometimes permit algebraic shortcuts that bypass intermediate representations; nonlinear MLPs make that approach much less available.
3.2 Decomposing Activations and the Role of Bases
A representation can be easier to understand as a collection of features than as an undifferentiated high-dimensional space. Purely linear surroundings allow changes of basis without changing the computation. Coordinatewise nonlinearities instead privilege the neuron basis, making alignment with features plausible but not guaranteed.

3.3 Neurons and Polysemanticity
Some neurons consistently respond to recognizable properties; others activate for several apparently unrelated ones. The latter are called polysemantic. An opaque neuron might instead represent a coherent but unfamiliar concept, so top examples alone cannot settle its interpretation.
3.4 The Superposition Hypothesis
The superposition hypothesis proposes that models encode more features than there are dimensions using nonorthogonal directions. Sparse activity limits how often those directions interfere, making the tradeoff worthwhile. If this occurs, simply finding a different basis cannot separate every feature. Connections to compressed sensing and almost-orthogonal vectors motivate the hypothesis rather than prove it for these models.


Update
A later author update points to Toy Models of Superposition as the clearer, more developed account of this hypothesis.
3.5 What Can We Do About Superposition?
Two broad responses are to build models with less superposition or to recover useful feature dictionaries from superposed representations. This paper investigates the architectural route.
4. SoLU: Designing for Interpretability
Architectural design can change how hard a network is to reverse engineer. The aim here is to encourage features to align with individual MLP neurons while retaining useful predictive performance.
4.1 Properties that May Reduce Polysemanticity
Candidate mechanisms include sparse activations, competition between simultaneously active neurons, superlinear responses that penalize distributed features, and architectures with more neurons per parameter or unit of compute. Sparse weights may help when both adjacent spaces have meaningful coordinate bases, but are less natural for residual-stream connections. These are design motivations, not independently demonstrated remedies.
4.2 The SoLU Activation Function
SoLU multiplies each activation by its softmax weight across the activation vector. Large entries suppress smaller competitors, and a concentrated feature can survive more strongly than one spread over many coordinates. The original formulas and numerical examples are retained.
Equations
4.3 LayerNorm
SoLU alone substantially hurts performance, behaving like a model about 30–50% smaller. Adding LayerNorm after it restores competitive loss while retaining clearer pre-normalization examples. Because normalization removes overall scale, it can amplify distributed features suppressed by SoLU; the softmax denominator also cancels in the idealized composition. This is a central limitation of the interpretability claim.
Equation

Equation
4.4 Parallelism Implementation Details
Large-model tensor parallelism applies softmax and LayerNorm within local activation blocks so each accelerator can operate without extra communication. Informal comparisons find no large effect from this blocking, but it is part of the implemented architecture.
5. Results on Performance
Matched baseline and SoLU-plus-LayerNorm models range from roughly ten million to fifty billion parameters. Loss differences stay within approximately a 0.95–1.05 equivalent model-size multiplier, and downstream tasks show mixed small differences rather than a systematic winner. No separate hyperparameter search is performed. Suitable accelerator kernels keep measured training-speed differences below 1%.


6. Results on Interpretability
The interpretability evaluation combines blinded quick judgments with more open-ended neuron exploration. Its outcome measures ease of proposing an explanation, not a complete verified account of every neuron’s behavior.
6.1 Setup of Experiments
A rigorous neuron interpretation can require extensive counterexamples and interventions. To cover several model sizes, the study instead asks whether brief inspection produces one plausible explanation. This practical proxy can make both false-positive and false-negative judgments.
Caveat
The later author caveat is substantial: clean strongest activations need not imply monosemantic behavior across the full activation range, especially when the architecture directly optimizes that appearance. The authors consequently became less confident in this metric for architectural selection.
In the original blinded experiment, evaluators spend one or two minutes on about twenty highly activating snippets and ask whether one rule explains at least 80% of the strongest firings. Sixty neurons from each of two architectures at five sizes, plus sixty from a normalization-only control, total 660 judgments.

6.2 Quantitative Results
SoLU increases quick interpretability by about twenty-five percentage points through forty layers. At sixty-four layers, rates are 42% versus 33%. A normalization-only sixteen-layer control lies between the two architectures. Differences across annotators and the omission of systematic negative examples constrain the conclusion: the experiment supports a change in first-glance legibility, not established neuron purity.

6.3 Qualitative Exploration of SoLU Models
The following examples are exploratory selections rather than a systematic comparison against baseline models. They illustrate what the authors could investigate more easily after introducing SoLU.
6.3.1 One-Layer Model Neurons
In a one-layer model, a neuron’s output vector can be multiplied into the unembedding to inspect its direct logit effects, modulo normalization. A base64-responsive neuron promotes mixed-case strings and suppresses ordinary English words, providing an additional check on its proposed role. Other examples concern capitalized text and number punctuation.

6.3.2 Early Layer Neurons in Larger Models ("de-tokenization")
Early-layer examples combine token fragments into words, names, and compound expressions, or distinguish the same token in different languages and programming contexts. The proposed role is to translate arbitrary tokenization into more contextual representations; it is a qualitative pattern, not an exhaustive description of early layers.

6.3.3 Late Layer Neurons in Larger Models ("re-tokenization")
Late-layer examples help produce the successive tokens of a word already represented internally. Their activations and output weights support a tentative reverse mapping from contextual concepts back to the tokenizer’s vocabulary.

6.3.4 Middle Layer Neurons in Larger Models
Middle-layer examples concern more abstract contexts, such as numbers counting people, descriptive clauses, hedging, dates, quantities in recipes, and table structure. Together with the early and late examples, these suggest a useful orientation for further study, without establishing a universal layer-by-layer pipeline.

6.3.5 Abstract Patterns
Additional observations include apparent feature splitting in larger models, families indexed by token and language, similarities between early recognition and late production, and concepts resembling those seen in multimodal models. These possible organizing principles remain untested in depth.
6.3.6 Partial Mitigation of Interpretability Illusions
Interpretation can be misleading when only favorable examples are shown. The exploratory work partly mitigates this by sampling the model’s broad training distribution, editing examples interactively, sometimes inspecting lower activations, and occasionally testing precise hypotheses such as a regular expression. Most neuron explanations still lack extensive validation. [38]
6.4 Implications of LayerNorm
Comparing pre- and post-LayerNorm examples reveals more activations inconsistent with a neuron’s apparent theme after normalization, especially at low magnitudes. An informal, unblinded study of about a thousand examples for a left/right neuron supports that impression. The evidence is consistent with normalization recovering less visible superposed features, and warns against treating a clean top-activation display as a complete explanation.

7. Related Work
The related literature covers transformer MLPs, neuron interpretation, sparse representations, and architectural approaches to interpretability.
7.1 Understanding Transformer MLPs
Prior transformer work analyzes MLPs as key–value memories, localizes factual information, edits knowledge, and studies individual neurons. Other researchers report both interpretable examples and persistent difficulty, with differences potentially arising from models, methodology, and definitions of interpretability.
7.2 Analyzing Individual Neurons and Features
Related feature analysis spans word embeddings, recurrent networks, and convolutional vision models. Strong interpretations in those settings motivate analogous work on transformer activations.
7.3 Polysemanticity and Superposition
Polysemanticity describes apparently unrelated responses in a single neuron; superposition is one proposed explanation. Related ideas arise in multiple word senses, sparse coding, dictionary learning, neural coding, and vector-symbolic representations.
Follow-Up
The later Toy Models of Superposition paper provides a broader account of these related research traditions.
7.4 Transformer Architectural Variants
Many transformer modifications target training stability or predictive performance. SoLU instead asks whether an activation change can improve the accessibility of internal representations while retaining performance.
7.5 Sparsity
Nonnegative matrix factorization and sparse coding historically produced useful interpretable factors. Later work applies sparsity to neural activations, weights, and word embeddings for several goals, including efficiency and interpretability. Sparsity alone is not equivalent to a verified semantic decomposition.
7.6 Designing Models for Interpretability
Other interpretable-by-design approaches guarantee properties such as monotonicity or build readable rules. This paper’s goal is more modest: make a still-complex neural network easier to reverse engineer, rather than immediately understandable in full.
8. Discussion
The results show that architecture can change the legibility of selected internal features at comparable loss. Limitations include one model family and training distribution, a much smaller effect at fifty billion parameters, the weak quick-inspection metric, and hidden features recovered by normalization. Even genuinely interpretable individual neurons would require additional structure to explain the global behavior of a network with millions of them.
Acknowledgments
The authors thank colleagues and external researchers for discussion, feedback, and support.
Author contributions
Nelson Elhage implemented and trained the models; several authors conducted blinded neuron evaluation and qualitative analysis. The wider team supplied distributed training infrastructure, baseline models, visualization tools, cluster operation, and writing.
Citation Information
The source’s citation identifies Nelson Elhage and colleagues, Softmax Linear Units, Transformer Circuits Thread, 2022. Its original citation listing is retained below.
Elhage, et al., "Softmax Linear Units", Transformer Circuits Thread, 2022.
@article{elhage2022solu,
title={Softmax Linear Units},
author={Elhage, Nelson and Hume, Tristan and Olsson, Catherine and Nanda, Neel and Henighan, Tom and Johnston, Scott and ElShowk, Sheer and Joseph, Nicholas and DasSarma, Nova and Mann, Ben and Hernandez, Danny and Askell, Amanda and Ndousse, Kamal and Jones, Andy and Drain, Dawn and Chen, Anna and Bai, Yuntao and Ganguli, Deep and Lovitt, Liane and Hatfield-Dodds, Zac and Kernion, Jackson and Conerly, Tom and Kravec, Shauna and Fort, Stanislav and Kadavath, Saurav and Jacobson, Josh and Tran-Johnson, Eli and Kaplan, Jared and Clark, Jack and Brown, Tom and McCandlish, Sam and Amodei, Dario and Olah, Christopher},
year={2022},
journal={Transformer Circuits Thread},
note={https://transformer-circuits.pub/2022/solu/index.html}
}