Paper proposes better Activation Oracles for LLM interpretability

Arxiv CS.LG (machine learning)

Activation Oracles (AOs) for interpreting residual stream activations suffer from hallucinations and vagueness. The paper also identifies text-inversion confounds that complicate evaluation.

interpretability

Arxiv CS.LG (machine learning)·Jan BauerCeleste De SchamphelaereAdam KarvonenNiclas LuickNeel Nanda

8 days ago