Our paper Adversarial Preference Learning for Robust LLM Alignment is accepted by ACL2025. Arxiv Link