BYORn method defends LVLMs against backdoor attacks

Arxiv CS.LG (machine learning)

The paper introduces BYORn (Bootstrap Your Own Responses), a defense against backdoor attacks in large vision-language models during supervised fine-tuning. Unlike existing defenses that require clean reference data, BYORn uses the model's own responses to detect and mitigate attacks. Experiments show it effectively neutralizes various backdoor triggers while maintaining model performance.

vision-language-models

Arxiv CS.LG (machine learning)·Ivan Saboli\'cMarin Or\v{s}i\'cJosip \v{S}ari\'cSven Lon\v{c}ari\'c

8 days ago