At a glance
Feature steering changes a model’s internal activations along an interpretable direction. This study tests 29 socially relevant features in Claude 3 Sonnet against knowledge, social-bias, and political-choice evaluations. Moderate steering can alter answers while largely preserving the measured capabilities, but effects often extend beyond the topic suggested by a feature’s name. Two features reduce measured bias across nine BBQ categories. The results are preliminary and do not establish reliable control of behavior in deployment.

Methods
The experiment crosses every selected feature with every evaluation, testing both intended changes and effects on other domains.
How we picked features and implemented feature steering
The features come from the dictionary learned for Claude 3 Sonnet in Scaling Monosemanticity. Steering adds a constant multiple of a feature direction to the residual stream. The 29 chosen directions cover social biases, political ideologies, neutrality, agreement, safety, and related concepts. Their labels describe activating contexts; they do not determine whether increasing the feature will increase or reduce a corresponding bias.
How we picked and implemented evaluations
MMLU and PubMedQA serve as capability checks. BBQ measures nine categories of social bias, while the model-written evaluations ask multiple-choice questions about abortion and immigration. Each question is sampled ten times to estimate answer probabilities, introducing sampling noise. These narrow benchmarks enable many comparisons but do not measure overall helpfulness or real-world fairness. [5], [6], [8], [9]
Results
The study first bounds useful steering strengths, then examines intended effects, cross-topic effects, and two promising bias-reduction directions.
Finding the feature steering sweet spot
Across nine tested strengths between −20 and 20, MMLU performance is relatively well preserved between approximately −5 and +5 for all 29 features. PubMedQA shows a similar pattern. Stronger interventions sharply reduce accuracy and can derail the response entirely, as Table A1 illustrates. This shared range is an empirical result for these features and evaluations, rather than a generally safe steering scale.
Measuring social biases with BBQ
Within the moderate range, steering changes both targeted and unrelated BBQ scores. Increasing the “Multiple perspectives and balance” feature reduces several measured biases. Increasing “Gender bias awareness” instead increases both gender and age bias scores. A direction activated by discussion of a bias therefore need not counteract that bias, and the effects of intervention cannot be inferred from its label alone. Figures A2–A4 retain results for the broader feature set.

Measuring political biases
Political-choice evaluations respond to steering in some expected ways: strengthening the pro-life direction increases anti-abortion selections, while strengthening a left-wing direction reduces them. However, the pro-life direction also changes anti-immigration selections more than the feature specifically associated with immigrants taking jobs. These are shifts in answers to constructed multiple-choice items, not measurements of an intrinsic political position. Figures A5–A6 show the wider set of directions.


The “Neutrality” and “Multiple Perspectives” features
Positive steering of “Neutrality and impartiality” and “Multiple perspectives and balance” reduces BBQ bias scores across all nine categories. The first direction also slightly reduces BBQ accuracy; the second largely maintains it over the tested range. This is a useful candidate intervention, but the limited evaluations do not establish a universal reduction in harmful bias.

Limitations
The study examines 29 features out of millions using static multiple-choice evaluations. Ten samples per item make probability estimates noisy, and only two capability benchmarks constrain possible damage. Results therefore do not characterize the typical feature, broader conversation quality, or robustness across settings.
Lessons learned
A feature’s activation pattern and its causal effect on generation can differ. Steering may change several domains at once, and extreme strengths impair relevance and quality. Reliable interventions require direct evaluation of resulting behavior and interactions between features.
Future work
The authors propose testing larger sparse autoencoders, broader evaluations, and steering only generated tokens instead of the entire prompt. They also suggest comparing activation steering and prompt-based methods, studying feature interactions in circuits, and trying multiplicative, projective, or conditional interventions. The appendix’s limited prompt comparison finds both targeted and cross-topic changes, without establishing that either approach is generally superior. [10], [11]
Conclusion
Feature steering can change socially relevant outputs with modest measured capability costs, but meaningful side effects and unreliable feature labels limit precision. The two bias-reducing directions motivate further study; deployment would require much broader validation.
Bibtex
This is an Anthropic research report by Esin Durmus and collaborators, published on 25 October 2024. The original page provides the complete author list and citation record.
Acknowledgements
Esin Durmus and Deep Ganguli designed the study and wrote the report; Durmus ran the experiments. Jonathan Marcus and Oliver Rausch built the steering interface, with code, advice, and review from the listed collaborators.
Appendices
The accompanying appendix retains the example generations, all 29 feature identifiers, additional evaluation plots, and the complete prompting comparison. Its tables and figures appear below.
Appendix 1: Example generations
Table A1 varies the gender-bias-awareness feature while completing a sentence about a nurse. At strength 0 the model offers several relevant continuations; at 5 it discusses gender conventions; at 12 it drifts to elementary education. This illustrates how strong steering can sacrifice relevance.

Appendix 2: All 29 features
Table A2 lists all 29 investigated feature IDs, grouped by their interpreted topics. The descriptions summarize activating contexts and should not be read as guarantees about the effect of steering.


Appendix 3.1: PubMedQA
Figure A1 repeats the capability check with PubMedQA and shows a broadly similar moderate-steering range to MMLU.

Appendix 3.2: Additional BBQ results
Figures A2 and A3 show aggregate and category-specific BBQ bias scores for the selected features. Figure A4 checks accuracy: the neutrality direction reduces bias but also lowers BBQ accuracy, demonstrating why a bias score alone is insufficient.



Appendix 3.3: Political-choice results
Figures A5 and A6 retain all plotted feature effects on anti-abortion and anti-immigration selections. Several directions affect both topics, illustrating the limited specificity of these interventions.


Appendix 4: Comparison with prompting
Five prompts per topic range from strongly supportive to strongly opposed positions. They change selections on their own topic and on the other topic. The abortion prompts produce anti-abortion selection rates from 12.8% to 83.0%; immigration prompts produce anti-immigration rates from 4.3% to 55.2%. Effects need not be monotonic: the strongest anti-abortion prompt produces a lower selection rate than the generally anti-abortion prompt. The comparison remains too limited to rank prompting against feature steering.
The published captions of Tables A3 and A4 are swapped relative to their headers and contents. Here, A3 is read as abortion prompts → anti-abortion choices and A4 as immigration prompts → anti-immigration choices; their original printed captions remain visible.




Appendix 4: Complete prompt sets
Tables A7 and A8 provide the full five-prompt sets for abortion and immigration, including the intermediate positions used in the comparisons.

