Proceedings

Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterion

Xian-Jin Gui, Wei Wang, Zhang-Hao Tian

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 2469-2475.

https://doi.org/10.24963/ijcai.2021/340

Keywords

  • Machine Learning: Learning Theory
  • Machine Learning: Deep Learning
  • Machine Learning: Classification
  • Machine Learning: Weakly Supervised Learning

Abstract

Deep neural networks need large amounts of labeled data to achieve good performance. In real-world applications, labels are usually collected from non-experts such as crowdsourcing to save cost and thus are noisy. In the past few years, deep learning methods for dealing with noisy labels have been developed, many of which are based on the small-loss criterion. However, there are few theoretical analyses to explain why these methods could learn well from noisy labels. In this paper, we theoretically explain why the widely-used small-loss criterion works. Based on the explanation, we reformalize the vanilla small-loss criterion to better tackle noisy labels. The experimental results verify our theoretical explanation and also demonstrate the effectiveness of the reformalization.

Xian-Jin Gui, Wei Wang, and Zhang-Hao Tian. 2021. Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterion. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 2469–2475. https://doi.org/10.24963/ijcai.2021/340
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards