Mechanistic interpretability2026ICML Mechanistic Interpretability Workshop

Mixture-of-Steering Vectors

Prompt-conditioned activation steering for compositional hallucination mitigation

MoSV learns specialized activation-steering directions and routes each prompt to a tailored mixture at inference time. The method improved factual accuracy where a single global steering direction did not.

Daniel Lee, Feolu Kolawole, Vedant Srinivas

Stanford University

Activation steering
Representation geometry
Language models

Outputs

Presentation

Mixture-of-Steering Vectors

ICML Mechanistic Interpretability Workshop, 2026

Workshop presentation on prompt-conditioned routing over specialized residual-stream steering directions.

OpenReview

Code

MoSV implementation

Open source

Training, routing, evaluation, and analysis code for the reported experiments.

GitHub

MoSV pipeline from contrastive activation differences through clustering and prompt-conditioned routing. — Contrastive activation differences form specialized directions. At inference, a lightweight router composes directions for each prompt.