{"type":"ScholarlyArticle","title":"Information-Theoretic Methods in Deep Neural Networks: Recent Advances and Emerging Opportunities","authors":[{"name":"Shujian Yu","slug":"shujian-yu","given":"Shujian","family":"Yu","affiliation":""},{"name":"Luis Sanchez Giraldo","slug":"luis-sanchez-giraldo","given":"Luis","family":"Sanchez Giraldo","affiliation":""},{"name":"Jose Principe","slug":"jose-principe","given":"Jose","family":"Principe","affiliation":""}],"abstract":"We present a review on the recent advances and emerging opportunities around the theme of analyzing deep neural networks (DNNs) with information-theoretic methods. We first discuss popular information-theoretic quantities and their estimators. We then introduce recent developments on information-theoretic learning principles (e.g., loss functions, regularizers and objectives) and their parameterization with DNNs. We finally briefly review current usages of information-theoretic concepts in a few modern machine learning problems and list a few emerging opportunities.","doi":"10.24963/ijcai.2021/633","url":"https://ijcai.org/proceedings/2021/633","pdf":"https://ijcai.org/proceedings/2021/0633.pdf","markdown":"https://ijcai.org/proceedings/2021/633.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/633","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Survey Track","track":"Survey Track","number":633,"pages":{"first":4669,"last":4678},"keywords":["Machine learning: General"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}