Steering Truth in LLMs
Feature-engineering truth and other modality directions in LLM activations, then steering with XGBoost to beat linear-probe and contrastive-mean-difference baselines.
Researcher, Trustworthy Robotics Lab · 2026 · Trustworthy Robotics Lab
Demo walkthrough coming soon
Why I built it
draftTruth, and other high-level concepts, show up as directions in a model's activations — but the usual ways to recover them (a linear probe, or the contrastive mean difference between true and false statements) are blunt. I treated it as a feature-engineering problem: find richer truth and modality directions, then steer with XGBoost on top of them to move the model's behavior further than the linear baselines could. The harder lesson on gemma-2-2b was that a direction you can decode at ~99% accuracy isn't necessarily one you can push — reading a concept and steering it are not the same axis.
Stack
- Python
- PyTorch
- gemma-2-2b
- XGBoost
- NCSA DeltaAI (GH200)