Paper reveals CoRe heads drive functional sparsity in MLLMs

Arxiv CS.CL (NLP/language models)

New research identifies 'CoRe' (Concentrated Response) heads in multimodal LLMs that enforce query-relevant visual feature extraction, explaining functional sparsity. The authors show these heads can be manipulated to improve task performance and interpretability.

multimodalinterpretability

Arxiv CS.CL (NLP/language models)·Ruoxi SunQuantong QiuJuntao LiZecheng TangYihang LouMin Zhang

6 days ago