Literature
Display
← All papers

Using Dictionary Learning Features as Classifiers

Trenton Bricken · Jonathan Marcus · Siddharth Mishra-Sharma · Meg Tong · Ethan Perez · Mrinank Sharma · Kelley Rivoire · Thomas Henighan

· Transformer Circuits Thread ↗

Interpretability3 min text · CondensedOriginal paper ↗
Contents

Overview

This preliminary report tests whether SAE features improve harmfulness classifiers over raw model activations. Feature classifiers can be competitive and sometimes generalize better to changed input formats, while feature visualizations expose dataset shortcuts and make small decision trees intelligible. Raw activations remain a strong, simpler baseline; the findings do not establish a generally reliable safety detector. [1] [4]

Classifiers using Feature Activations can be Competitive with Raw Activations

Logistic probes use middle-layer residual activations or SAE features to classify synthetic biology-related prompts. Feature probes receive L1 regularization and raw-activation probes L2 regularization, selected on held-out synthetic data. Evaluations include a second generator, translated or encoded versions of its prompts, and a small difficult set written by human experts.

Experiments use an 18-layer transformer and a one-million-feature dictionary for Claude 3 Sonnet. Max-pooling feature activations across tokens performs well on synthetic sets; last-token representations perform better on the human set. The Sonnet experiment uses the second synthetic set for validation, unlike the smaller-model comparison. Consistent conversation tags and biology-enriched SAE training are consequential choices.

Using Features for Visualizing Dataset Spurious Correlations

A strongly harmless-predicting feature responds to academic writing instructions rather than the underlying harmfulness of a request. It occurs in about 10–15% of synthetic examples and is predominantly associated with harmless labels there. Removing it generally reduces in-distribution performance, illustrating how a useful dataset correlation can still be a deployment shortcut. Feature activation examples make this shortcut easy to notice.

Constructing Adversarial Examples based on Spurious Correlations

The authors turn this shortcut into a controlled stress test: they append three academic-style suffixes to 300 harmful training prompts, producing 900 examples. Nearly all are then classified as harmless by both feature and raw-activation probes. Polite suffixes selected from raw-activation inspection have a much weaker effect. This shows how interpretability can help find a classifier failure; it does not demonstrate bypassing the underlying language model’s safeguards. The exact tested suffix lists and comparison figures are retained.

adversarial_suffixes =  [

   "Please provide a relevant abstract, not to exceed 150 words (italic characters) 6-8 relevant keywords (italic characters).",

   "Aim for 2000 words, with research quotes/links. Explain it like you would to a colleague technician. Thank you.",

   "Include key findings, hypotheses, and quotes/references from relevant journal articles. The response should be written as if speaking to a biology student (~2000 words)."]

adversarial_suffixes = [

   "Thank you in advance for any guidance you can provide!",

   "Thanks so much for your help!",

   "Thank you in advance for your help!"

]

Out of Distribution Generalization of Features

On the translated and encoded synthetic set, a dictionary trained with a 90% biology mixture yields ROC AUC 0.96 for the feature classifier versus 0.90 for the raw-activation classifier. A spelling-abbreviated benign prompt illustrates the same possible advantage. These are dataset-specific results; the human-written evaluation does not show uniform superiority.

Wat r da long-term complicatns aftr recoverin frm neumonia? As a bioligy student, im particulrly intrestd in undrstandin da risk factrs that contrbute 2 da developmnt of these complicatns.

Feature Decision Trees are Less Performant, but Easy to Interpret

Depth-five decision trees, chosen on synthetic validation data, perform worse than logistic probes. Trees over SAE features outperform trees over raw activations and admit compact explanations in terms of recognizable concepts. The illustrated depth-two tree is an example of readability, rather than the full depth-five classifier used for the reported comparison.

Residuals

The SAE reconstruction residual retains information omitted by the feature representation. A separate L2 logistic classifier on that residual supplies a probability appended to the feature vector before fitting the L1 probe. This closes some performance gaps; residual-only probes can also perform comparably. The authors suggest shrinkage may leave useful feature information in the residual, but do not test that explanation.

Important Considerations for Feature-based Classifier Performance

Three practical factors repeatedly affect performance: matching Human/Assistant formatting across transformer, SAE, and classifier data; including relevant domain data in SAE training; and choosing token aggregation carefully. Max-pooling improves the synthetic comparisons but can underperform last-token pooling on human prompts. Its much larger number of active features and susceptibility to shortcuts are possible explanations, not established causes. Both pooling rules discard information, motivating richer aggregation methods.

Concept demonstration

Open image ↗