{"type":"ScholarlyArticle","title":"Cross-Relational Preference Learning for Better LLM Instruction Following","authors":[{"name":"Runsheng Li","slug":"runsheng-li","given":"Runsheng","family":"Li","affiliation":""},{"name":"Kai Sun","slug":"kai-sun","given":"Kai","family":"Sun","affiliation":""},{"name":"Bin Shi","slug":"bin-shi","given":"Bin","family":"Shi","affiliation":""},{"name":"Bo Dong","slug":"bo-dong","given":"Bo","family":"Dong","affiliation":""}],"abstract":"Large Language Models (LLMs) still exhibit limited capability in following complex instructions. While existing approaches often rely on preference learning to enhance this ability, they typically overlook the relationships between the permissible response spaces of different instructions, which restricts a model to align with subtle and diverse constraint variations. To address this, we propose Cross-Relational Preference Learning (CRPL), a novel framework for constructing preference data that explicitly models inter-instruction relationships through two key techniques: Cross-Relationship Perturbation and Cross-Region Pair Sampling. This enables the generation of more diverse preference data that captures a wide spectrum of constraint variations. Additionally, we introduce an atomic constraint-based verification mechanism to rigorously assess response satisfaction, ensuring high-quality preference pair construction. Extensive experiments across multiple preference learning methods (e.g., DPO, KTO), LLM backbones and four instruction-following benchmarks demonstrate that our approach achieves substantial improvements over prior baselines and exhibits strong generalization.","doi":"10.24963/ijcai.2026/439","url":"https://ijcai.org/proceedings/2026/439","pdf":"https://ijcai.org/proceedings/2026/0439.pdf","markdown":"https://ijcai.org/proceedings/2026/439.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/439","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":439,"pages":{"first":3943,"last":3951},"keywords":["Knowledge Representation and Reasoning: Preference modelling and preference-based reasoning","Natural Language Processing: Language models","Natural Language Processing: Language generation"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}