Proceedings

Two-stage Risk Control with Application to Ranked Retrieval

Yunpeng Xu, Mufang Ying, Wenge Guo, Zhi Wei

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 9104-9111.

https://doi.org/10.24963/ijcai.2025/1012

Keywords

  • Machine Learning: ML: Trustworthy machine learning
  • Uncertainty in AI: UAI: Inference
  • Uncertainty in AI: UAI: Other
  • Data Mining: DM: Information retrieval

Abstract

Practical machine learning systems often operate in multiple sequential stages, as seen in ranking and recommendation systems, which typically include a retrieval phase followed by a ranking phase. Effectively assessing prediction uncertainty and ensuring effective risk control in such systems pose significant challenges due to their inherent complexity. To address these challenges, we developed two-stage risk control methods based on the recently proposed learn-then-test (LTT) and conformal risk control (CRC) frameworks. Unlike the methods in prior work that address multiple risks, our approach leverages the sequential nature of the problem, resulting in reduced computational burden. We provide theoretical guarantees for our proposed methods and design novel loss functions tailored for ranked retrieval tasks. The effectiveness of our approach is validated through experiments on two large-scale, widely-used datasets: MSLR- Web and Yahoo LTRC.

Yunpeng Xu, Mufang Ying, Wenge Guo, and Zhi Wei. 2025. Two-stage Risk Control with Application to Ranked Retrieval. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 9104–9111. https://doi.org/10.24963/ijcai.2025/1012
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards