Literature
Display
← All papers

Evaluating feature steering: A case study in mitigating social biases

Esin Durmus · Alex Tamkin · Jack Clark · Jerry Wei · Jonathan Marcus · Joshua Batson · Kunal Handa · Liane Lovitt · Meg Tong · Miles McCain · Oliver Rausch · Saffron Huang · Sam Bowman · Stuart Ritchie · Tom Henighan · Deep Ganguli

· Anthropic research report ↗

Interpretability5 min text · CondensedOriginal paper ↗
Contents

At a glance

Feature steering changes a model’s internal activations along an interpretable direction. This study tests 29 socially relevant features in Claude 3 Sonnet against knowledge, social-bias, and political-choice evaluations. Moderate steering can alter answers while largely preserving the measured capabilities, but effects often extend beyond the topic suggested by a feature’s name. Two features reduce measured bias across nine BBQ categories. The results are preliminary and do not establish reliable control of behavior in deployment.

Figure 1. We identify a feature steering “sweet spot” (x-axis, a steering factor between -5 and 5) where feature steering does not significantly impact model capabilities (y-axis, we use MMLU accuracy as a proxy for model capabilities). Surprisingly, this “sweet spot” is shared across all 29 features (colored lines, see legend for short description of the features) that we tested for.
Figure 1. We identify a feature steering “sweet spot” (x-axis, a steering factor between -5 and 5) where feature steering does not significantly impact model capabilities (y-axis, we use MMLU accuracy as a proxy for model capabilities). Surprisingly, this “sweet spot” is shared across all 29 features (colored lines, see legend for short description of the features) that we tested for.

Methods

The experiment crosses every selected feature with every evaluation, testing both intended changes and effects on other domains.

How we picked features and implemented feature steering

The features come from the dictionary learned for Claude 3 Sonnet in Scaling Monosemanticity. Steering adds a constant multiple of a feature direction to the residual stream. The 29 chosen directions cover social biases, political ideologies, neutrality, agreement, safety, and related concepts. Their labels describe activating contexts; they do not determine whether increasing the feature will increase or reduce a corresponding bias.

How we picked and implemented evaluations

MMLU and PubMedQA serve as capability checks. BBQ measures nine categories of social bias, while the model-written evaluations ask multiple-choice questions about abortion and immigration. Each question is sampled ten times to estimate answer probabilities, introducing sampling noise. These narrow benchmarks enable many comparisons but do not measure overall helpfulness or real-world fairness. [5], [6], [8], [9]

Results

The study first bounds useful steering strengths, then examines intended effects, cross-topic effects, and two promising bias-reduction directions.

Finding the feature steering sweet spot

Across nine tested strengths between −20 and 20, MMLU performance is relatively well preserved between approximately −5 and +5 for all 29 features. PubMedQA shows a similar pattern. Stronger interventions sharply reduce accuracy and can derail the response entirely, as Table A1 illustrates. This shared range is an empirical result for these features and evaluations, rather than a generally safe steering scale.

Measuring social biases with BBQ

Within the moderate range, steering changes both targeted and unrelated BBQ scores. Increasing the “Multiple perspectives and balance” feature reduces several measured biases. Increasing “Gender bias awareness” instead increases both gender and age bias scores. A direction activated by discussion of a bias therefore need not counteract that bias, and the effects of intervention cannot be inferred from its label alone. Figures A2–A4 retain results for the broader feature set.

Figure 2. The gender bias awareness feature (purple) exhibits both on-target effects (Left panel, increasing the steering factor increases gender bias) and off-target effects (Right panel, increasing the steering factor also increases age bias). We measure the bias scores (y-axes) using the BBQ benchmark . We observe these effects within the feature steering sweet spot (x-axes, (-5, 5)).
Figure 2. The gender bias awareness feature (purple) exhibits both on-target effects (Left panel, increasing the steering factor increases gender bias) and off-target effects (Right panel, increasing the steering factor also increases age bias). We measure the bias scores (y-axes) using the BBQ benchmark. We observe these effects within the feature steering sweet spot (x-axes, (-5, 5)).

Measuring political biases

Political-choice evaluations respond to steering in some expected ways: strengthening the pro-life direction increases anti-abortion selections, while strengthening a left-wing direction reduces them. However, the pro-life direction also changes anti-immigration selections more than the feature specifically associated with immigrants taking jobs. These are shifts in answers to constructed multiple-choice items, not measurements of an intrinsic political position. Figures A5–A6 show the wider set of directions.

Figure 3. A variety of political ideology features (colored lines, see legend for a short description of each feature) exhibit on-target effects in how they change the model’s selection rate for anti-abortion responses (y-axis). For example, increasing the pro-life and anti-abortion stance feature (blue) increases the percentage of anti-abortion selections by 50%.
Figure 3. A variety of political ideology features (colored lines, see legend for a short description of each feature) exhibit on-target effects in how they change the model’s selection rate for anti-abortion responses (y-axis). For example, increasing the pro-life and anti-abortion stance feature (blue) increases the percentage of anti-abortion selections by 50%.
Figure 4. A variety of political ideology features (Colored lines, see legend for a short description of each feature) also exhibit on-target effects in how they change the model’s selection rate for anti-immigration responses (y-axis). For example, increasing the Left-wing political ideologies feature (orange) decreases the percentage of anti-immigration selections by 25%. However, we also see an off-target effect: steering the pro-life and anti-abortion stance (purple) has the largest impact on changing the %anti-immigration responses, even though this feature does not appear to fire in contexts regarding immigration.
Figure 4. A variety of political ideology features (Colored lines, see legend for a short description of each feature) also exhibit on-target effects in how they change the model’s selection rate for anti-immigration responses (y-axis). For example, increasing the Left-wing political ideologies feature (orange) decreases the percentage of anti-immigration selections by 25%. However, we also see an off-target effect: steering the pro-life and anti-abortion stance (purple) has the largest impact on changing the %anti-immigration responses, even though this feature does not appear to fire in contexts regarding immigration.

The “Neutrality” and “Multiple Perspectives” features

Positive steering of “Neutrality and impartiality” and “Multiple perspectives and balance” reduces BBQ bias scores across all nine categories. The first direction also slightly reduces BBQ accuracy; the second largely maintains it over the tested range. This is a useful candidate intervention, but the limited evaluations do not establish a universal reduction in harmful bias.

Figure 5. Steering the “Neutrality and Impartiality” (blue) and “Multiple perspectives and balance” (orange) features reduces BBQ bias scores (y-axes) across nine different categories (panels).
Figure 5. Steering the “Neutrality and Impartiality” (blue) and “Multiple perspectives and balance” (orange) features reduces BBQ bias scores (y-axes) across nine different categories (panels).

Limitations

The study examines 29 features out of millions using static multiple-choice evaluations. Ten samples per item make probability estimates noisy, and only two capability benchmarks constrain possible damage. Results therefore do not characterize the typical feature, broader conversation quality, or robustness across settings.

Lessons learned

A feature’s activation pattern and its causal effect on generation can differ. Steering may change several domains at once, and extreme strengths impair relevance and quality. Reliable interventions require direct evaluation of resulting behavior and interactions between features.

Future work

The authors propose testing larger sparse autoencoders, broader evaluations, and steering only generated tokens instead of the entire prompt. They also suggest comparing activation steering and prompt-based methods, studying feature interactions in circuits, and trying multiplicative, projective, or conditional interventions. The appendix’s limited prompt comparison finds both targeted and cross-topic changes, without establishing that either approach is generally superior. [10], [11]

Conclusion

Feature steering can change socially relevant outputs with modest measured capability costs, but meaningful side effects and unreliable feature labels limit precision. The two bias-reducing directions motivate further study; deployment would require much broader validation.

Bibtex

This is an Anthropic research report by Esin Durmus and collaborators, published on 25 October 2024. The original page provides the complete author list and citation record.

Acknowledgements

Esin Durmus and Deep Ganguli designed the study and wrote the report; Durmus ran the experiments. Jonathan Marcus and Oliver Rausch built the steering interface, with code, advice, and review from the listed collaborators.

Appendices

The accompanying appendix retains the example generations, all 29 feature identifiers, additional evaluation plots, and the complete prompting comparison. Its tables and figures appear below.

Appendix 1: Example generations

Table A1 varies the gender-bias-awareness feature while completing a sentence about a nurse. At strength 0 the model offers several relevant continuations; at 5 it discusses gender conventions; at 12 it drifts to elementary education. This illustrates how strong steering can sacrifice relevance.

Table A1: While steering can influence the generated content, it may also compromise response quality and relevance if the steering factor is too high. This table demonstrates how model outputs change as the steering factor increases. At baseline (factor 0), the model used both 'he' and 'she' pronouns. As the steering factor increased, it shifted to discussing gender dynamics in nursing (factor 5), then to gender representation in education (factor 12).
Table A1: While steering can influence the generated content, it may also compromise response quality and relevance if the steering factor is too high. This table demonstrates how model outputs change as the steering factor increases. At baseline (factor 0), the model used both 'he' and 'she' pronouns. As the steering factor increased, it shifted to discussing gender dynamics in nursing (factor 5), then to gender representation in education (factor 12).

Appendix 2: All 29 features

Table A2 lists all 29 investigated feature IDs, grouped by their interpreted topics. The descriptions summarize activating contexts and should not be read as guarantees about the effect of steering.

Table A2, first part: 14 feature IDs and descriptions.
Table A2, first part: 14 feature IDs and descriptions.
Table A2: Full list of features with corresponding categories, IDs, and descriptions. Continued: remaining 15 features.
Table A2: Full list of features with corresponding categories, IDs, and descriptions. Continued: remaining 15 features.

Appendix 3.1: PubMedQA

Figure A1 repeats the capability check with PubMedQA and shows a broadly similar moderate-steering range to MMLU.

Figure A1: We observe a similar "sweet spot" (x-axis, a steering factor between -5 and 5) for PubMedQA, where feature steering does not significantly impact model capabilities (y-axis, we use PubMedQA accuracy as another proxy for model capabilities). Similar to the MMLU case, this "sweet spot" is mostly shared across all 29 features (colored lines, see legend for a short description of the features) that we tested for.
Figure A1: We observe a similar "sweet spot" (x-axis, a steering factor between -5 and 5) for PubMedQA, where feature steering does not significantly impact model capabilities (y-axis, we use PubMedQA accuracy as another proxy for model capabilities). Similar to the MMLU case, this "sweet spot" is mostly shared across all 29 features (colored lines, see legend for a short description of the features) that we tested for.

Appendix 3.2: Additional BBQ results

Figures A2 and A3 show aggregate and category-specific BBQ bias scores for the selected features. Figure A4 checks accuracy: the neutrality direction reduces bias but also lowers BBQ accuracy, demonstrating why a bias score alone is insufficient.

Figure A2: The features "Neutrality and impartiality" (green line) and "Gender bias awareness" (purple line) demonstrate significant impacts on overall BBQ bias scores. As the steering factor increases, the bias score decreases for "Neutrality and impartiality" and increases for "Gender bias awareness". The colored lines represent different features related to bias and discrimination, such as age, gender, political, and ethnic factors, and their effects on the overall BBQ bias score as the steering factor varies from -5 to 5.
Figure A2: The features "Neutrality and impartiality" (green line) and "Gender bias awareness" (purple line) demonstrate significant impacts on overall BBQ bias scores. As the steering factor increases, the bias score decreases for "Neutrality and impartiality" and increases for "Gender bias awareness". The colored lines represent different features related to bias and discrimination, such as age, gender, political, and ethnic factors, and their effects on the overall BBQ bias score as the steering factor varies from -5 to 5.
Figure A3: Feature steering demonstrates varied and sometimes unexpected effects on bias scores across different categories (Age, Disability Status, Gender Identity, Nationality, Physical Appearance, Race/Ethnicity, Religion, Socioeconomic Status, and Sexual Orientation). The x-axis represents steering factors (-5 to 5), and the y-axis shows bias scores (0% to 40%). Colored lines denote various features related to bias and discrimination, such as gender roles, inclusivity, political neutrality, and stereotypes.
Figure A3: Feature steering demonstrates varied and sometimes unexpected effects on bias scores across different categories (Age, Disability Status, Gender Identity, Nationality, Physical Appearance, Race/Ethnicity, Religion, Socioeconomic Status, and Sexual Orientation). The x-axis represents steering factors (-5 to 5), and the y-axis shows bias scores (0% to 40%). Colored lines denote various features related to bias and discrimination, such as gender roles, inclusivity, political neutrality, and stereotypes.
Figure A4: The "Neutrality and impartiality" feature decreases overall bias but also reduces BBQ Accuracy. The graph shows the impact of steering on BBQ accuracy, with some decrease for "Neutrality and impartiality" and "Gender norms and discrimination" features, while others remain largely unchanged.
Figure A4: The "Neutrality and impartiality" feature decreases overall bias but also reduces BBQ Accuracy. The graph shows the impact of steering on BBQ accuracy, with some decrease for "Neutrality and impartiality" and "Gender norms and discrimination" features, while others remain largely unchanged.

Appendix 3.3: Political-choice results

Figures A5 and A6 retain all plotted feature effects on anti-abortion and anti-immigration selections. Several directions affect both topics, illustrating the limited specificity of these interventions.

Figure A5: Feature steering can significantly influence anti-abortion selection percentages. This graph illustrates how different steering factors (-5.0 to 5.0) affect the percentage of anti-abortion selections in multiple-choice questions across various ideological features.
Figure A5: Feature steering can significantly influence anti-abortion selection percentages. This graph illustrates how different steering factors (-5.0 to 5.0) affect the percentage of anti-abortion selections in multiple-choice questions across various ideological features.
Figure A6: Feature steering shows varied and sometimes unexpected effects on anti-immigration selection percentages across various ideological features. This graph illustrates how different steering factors (-5.0 to 5.0) influence the percentage of anti-immigration selections in multiple-choice questions.
Figure A6: Feature steering shows varied and sometimes unexpected effects on anti-immigration selection percentages across various ideological features. This graph illustrates how different steering factors (-5.0 to 5.0) influence the percentage of anti-immigration selections in multiple-choice questions.

Appendix 4: Comparison with prompting

Five prompts per topic range from strongly supportive to strongly opposed positions. They change selections on their own topic and on the other topic. The abortion prompts produce anti-abortion selection rates from 12.8% to 83.0%; immigration prompts produce anti-immigration rates from 4.3% to 55.2%. Effects need not be monotonic: the strongest anti-abortion prompt produces a lower selection rate than the generally anti-abortion prompt. The comparison remains too limited to rank prompting against feature steering.

The published captions of Tables A3 and A4 are swapped relative to their headers and contents. Here, A3 is read as abortion prompts → anti-abortion choices and A4 as immigration prompts → anti-immigration choices; their original printed captions remain visible.

Table A3: Effect of Different Immigration Prompts on Anti-Immigration Selection Percentage.
Table A3: Effect of Different Immigration Prompts on Anti-Immigration Selection Percentage.
Table A4: Effect of Different Abortion Prompts on Anti-Abortion Selection Percentage.
Table A4: Effect of Different Abortion Prompts on Anti-Abortion Selection Percentage.
Table A5: Effect of Different Abortion Prompts on Anti-Immigration Selection Percentage.
Table A5: Effect of Different Abortion Prompts on Anti-Immigration Selection Percentage.
Table A6: Effect of Immigration Prompts on Anti-Abortion Selection Percentage.
Table A6: Effect of Immigration Prompts on Anti-Abortion Selection Percentage.

Appendix 4: Complete prompt sets

Tables A7 and A8 provide the full five-prompt sets for abortion and immigration, including the intermediate positions used in the comparisons.

Table A7: Abortion Prompts.
Table A7: Abortion Prompts.
Table A8: Immigration Prompts.
Table A8: Immigration Prompts.
Concept demonstration

Open image ↗