{"type":"ScholarlyArticle","title":"Recognizable Information Bottleneck","authors":[{"name":"Yilin Lyu","slug":"yilin-lyu","given":"Yilin","family":"Lyu","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining , Beijing Jiaotong University"},{"name":"Xin Liu","slug":"xin-liu","given":"Xin","family":"Liu","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining , Beijing Jiaotong University"},{"name":"Mingyang Song","slug":"mingyang-song","given":"Mingyang","family":"Song","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining , Beijing Jiaotong University"},{"name":"Xinyue Wang","slug":"xinyue-wang","given":"Xinyue","family":"Wang","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining , Beijing Jiaotong University"},{"name":"Yaxin Peng","slug":"yaxin-peng","given":"Yaxin","family":"Peng","affiliation":"Department of Mathematics , School of Science , Shanghai University"},{"name":"Tieyong Zeng","slug":"tieyong-zeng","given":"Tieyong","family":"Zeng","affiliation":"The Chinese University of Hong Kong"},{"name":"Liping Jing","slug":"liping-jing","given":"Liping","family":"Jing","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining , Beijing Jiaotong University"}],"abstract":"Information Bottlenecks (IBs) learn representations that generalize to unseen data by information compression. However, existing IBs are practically unable to guarantee generalization in real-world scenarios due to the vacuous generalization bound. The recent PAC-Bayes IB uses information complexity instead of information compression to establish a connection with the mutual information generalization bound. However, it requires the computation of expensive second-order curvature, which hinders its practical application. In this paper, we establish the connection between the recognizability of representations and the recent functional conditional mutual information (f-CMI) generalization bound, which is significantly easier to estimate. On this basis we propose a Recognizable Information Bottleneck (RIB) which regularizes the recognizability of representations through a recognizability critic optimized by density ratio matching under the Bregman divergence. Extensive experiments on several commonly used datasets demonstrate the effectiveness of the proposed method in regularizing the model and estimating the generalization gap.","doi":"10.24963/ijcai.2023/448","url":"https://ijcai.org/proceedings/2023/448","pdf":"https://ijcai.org/proceedings/2023/0448.pdf","markdown":"https://ijcai.org/proceedings/2023/448.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/448","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":448,"pages":{"first":4028,"last":4036},"keywords":["Machine Learning: ML: Classification","Machine Learning: ML: Representation learning"],"videos":[],"references":[{"position":1,"raw":"[Alemi et al., 2017] Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open- Review.net, 2017.","title":"Deep variational information bottleneck","authors":"Alexander A Alemi, Ian Fischer, Joshua V Dillon, Kevin Murphy","year":"2017","venue":"5th International Conference on Learning Representations, ICLR 2017","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banerjee and Montúfar, 2021] Pradeep Kr. Banerjee and Guido Montúfar. Information complexity and generaliza- tion bounds. In IEEE International Symposium on Infor- mation Theory, ISIT 2021, Melbourne, Australia, July 12- 20, 2021, pages 676-681. IEEE, 2021.","title":"Information complexity and generalization bounds","authors":"Pradeep Kr, Guido Banerjee, Montúfar","year":"2021","venue":"IEEE International Symposium on Information Theory, ISIT 2021","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bu et al., 2020] Yuheng Bu, Shaofeng Zou, and Venu- gopal V. Veeravalli. Tightening mutual information-based bounds on generalization error. IEEE J. Sel. Areas Inf. Theory, 1(1):121-130, 2020.","title":"Tightening mutual information-based bounds on generalization error","authors":"Yuheng Bu, Shaofeng Zou, Venugopal V Veeravalli","year":"2020","venue":"IEEE J. Sel. Areas Inf. Theory","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Catoni, 2007] Olivier Catoni. Pac-bayesian supervised clas- sification: The thermodynamics of statistical learning. IMS Lecture Notes Monograph Series, 56:1-163, 2007. arXiv:0712.0248 [stat].","title":"Olivier Catoni. Pac-bayesian supervised classification: The thermodynamics of statistical learning","authors":"Catoni","year":"2007","venue":"IMS Lecture Notes Monograph Series","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Coates et al., 2011] Adam Coates, Andrew Y. Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Geoffrey J. Gordon, David B. Dunson, and Miroslav Dudík, editors, Proceed- ings of the Fourteenth International Conference on Artifi- cial Intelligence and Statistics, AISTATS 2011, Fort Laud- erdale, USA, April 11-13, 2011, volume 15, pages 215- 223. JMLR.org, 2011.","title":"An analysis of single-layer networks in unsupervised feature learning","authors":"Adam Coates, Andrew Y Ng, Honglak Lee","year":"2011","venue":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2011","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Du et al., 2020] Ying-Jun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees G. M. Snoek, and Ling Shao. Learning to learn with variational information bottleneck for domain generalization. In Computer Vision -ECCV 2020 -16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part X, volume 12355, pages 200-216. Springer, 2020.","title":"Learning to learn with variational information bottleneck for domain generalization","authors":"Ying-Jun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, G M Cees, Ling Snoek, Shao","year":"2020","venue":"Computer Vision -ECCV 2020 -16th European Conference","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dubois et al., 2020] Yann Dubois, Douwe Kiela, David J. Schwab, and Ramakrishna Vedantam. Learning optimal representations with the decodable information bottleneck. In NeurIPS 2020, December 6-12, 2020, virtual, 2020.","title":"Learning optimal representations with the decodable information bottleneck","authors":"Yann Dubois, Douwe Kiela, David J Schwab, Ramakrishna Vedantam","year":"2020","venue":"NeurIPS 2020","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dziugaite and Roy, 2017] Gintare Karolina Dziugaite and Daniel M. Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intel- ligence, UAI 2017, Sydney, Australia, August 11-15, 2017. AUAI Press, 2017.","title":"Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data","authors":"Karolina Gintare, Daniel M Dziugaite, Roy","year":"2017","venue":"Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI 2017","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gálvez et al., 2020] Borja Rodríguez Gálvez, Ragnar Thob- aben, and Mikael Skoglund. The convex information bot- tleneck lagrangian. Entropy, 22(1):98, 2020.","title":"The convex information bottleneck lagrangian","authors":"Rodríguez Borja, Ragnar Gálvez, Mikael Thobaben, Skoglund","year":"2020","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C Courville, Yoshua Bengio","year":"2014","venue":"NeurIPS 2014","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Haghifam et al., 2020] Mahdi Haghifam, Jeffrey Negrea, Ashish Khisti, Daniel M. Roy, and Gintare Karolina Dz- iugaite. Sharpened generalization bounds based on condi- tional mutual information and an application to noisy, iter- ative algorithms. In NeurIPS 2020, December 6-12, 2020, virtual, 2020.","title":"Sharpened generalization bounds based on conditional mutual information and an application to noisy, iterative algorithms","authors":"Mahdi Haghifam, Jeffrey Negrea, Ashish Khisti, Daniel M Roy, Gintare Karolina, Dziugaite","year":"2020","venue":"NeurIPS 2020","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Harutyunyan et al., 2021] Hrayr Harutyunyan, Maxim Ra- ginsky, Greg Ver Steeg, and Aram Galstyan. Information- theoretic generalization bounds for black-box learning al- gorithms. In NeurIPS 2021, December 6-14, 2021, virtual, pages 24670-24682, 2021.","title":"Informationtheoretic generalization bounds for black-box learning algorithms","authors":"Hrayr Harutyunyan, Maxim Raginsky, Greg Ver Steeg, Aram Galstyan","year":"2021","venue":"NeurIPS 2021","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hjelm et al., 2019] R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning deep rep- resentations by mutual information estimation and max- imization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.","title":"Learning deep representations by mutual information estimation and maximization","authors":"R, Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, Yoshua Bengio","year":"2019","venue":"7th International Conference on Learning Representations, ICLR 2019","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"3rd International Conference on Learning Representations, ICLR 2015","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kolchinsky et al., 2019] Artemy Kolchinsky, Brendan D. Tracey, and David H. Wolpert. Nonlinear information bot- tleneck. Entropy, 21(12):1181, 2019.","title":"Nonlinear information bottleneck","authors":"Artemy Kolchinsky, Brendan D Tracey, David H Wolpert","year":"2019","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krizhevsky and Hinton, 2009] Alex Krizhevsky and Geof- frey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lai et al., 2021] Qiuxia Lai, Yu Li, Ailing Zeng, Minhao Liu, Hanqiu Sun, and Qiang Xu. Information bottleneck approach to spatial attention learning. In Proceedings of the Thirtieth International Joint Conference on Artifi- cial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pages 779-785. ijcai.org, 2021.","title":"Information bottleneck approach to spatial attention learning","authors":"Qiuxia Lai, Yu Li, Ailing Zeng, Minhao Liu, Hanqiu Sun, Qiang Xu","year":"2021","venue":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event","doi":null,"url":null,"internal_url":"/proceedings/2021/108"},{"position":18,"raw":"[Levy, 2008] Bernard C. Levy. Binary and M-ary Hypothesis Testing, page 1-57. Springer US, Boston, MA, 2008.","title":"Binary and M-ary Hypothesis Testing","authors":"C Bernard, Levy","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2022] Bo Li, Yifei Shen, Yezhen Wang, Wenzhen Zhu, Colorado Reed, Dongsheng Li, Kurt Keutzer, and Han Zhao. Invariant information bottleneck for domain generalization. In Thirty-Sixth AAAI Conference on Arti- ficial Intelligence, Virtual Event, February 22 -March 1, 2022, pages 7399-7407. AAAI Press, 2022.","title":"Invariant information bottleneck for domain generalization","authors":"Bo Li, Yifei Shen, Yezhen Wang, Wenzhen Zhu, Colorado Reed, Dongsheng Li, Kurt Keutzer, Han Zhao","year":"2022","venue":"Thirty-Sixth AAAI Conference on Artificial Intelligence, Virtual Event","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Negrea et al., 2019] Jeffrey Negrea, Mahdi Haghifam, Gintare Karolina Dziugaite, Ashish Khisti, and Daniel M. Roy. Information-theoretic generalization bounds for SGLD via data-dependent estimates. In NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 11013-11023, 2019.","title":"Information-theoretic generalization bounds for SGLD via data-dependent estimates","authors":"Jeffrey Negrea, Mahdi Haghifam, Gintare Karolina Dziugaite, Ashish Khisti, Daniel M Roy","year":"2019","venue":"NeurIPS 2019","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learn- ing. In NIPS Workshop on Deep Learning and Unsuper- vised Feature Learning 2011, 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":"NIPS Workshop on Deep Learning and Unsupervised Feature Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Neyman et al., 1933] Jerzy Neyman, Egon Sharpe Pearson, and Karl Pearson. On the problem of the most effi- cient tests of statistical hypotheses. Philosophical Trans- actions of the Royal Society of London. Series A, Con- taining Papers of a Mathematical or Physical Character, 231(694-706):289-337, Feb 1933.","title":"On the problem of the most efficient tests of statistical hypotheses","authors":"Neyman","year":"1933","venue":"Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nowozin et al., 2016] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In NeurIPS 2016, December 5-10, 2016, Barcelona, Spain, pages 271-279, 2016.","title":"f-gan: Training generative neural samplers using variational divergence minimization","authors":"Sebastian Nowozin, Botond Cseke, Ryota Tomioka","year":"2016","venue":"NeurIPS 2016","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pan et al., 2021] Ziqi Pan, Li Niu, Jianfu Zhang, and Liqing Zhang. Disentangled information bottleneck. In Thirty- Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Event, February 2-9, 2021, pages 9285- 9293. AAAI Press, 2021.","title":"Disentangled information bottleneck","authors":"Ziqi Pan, Li Niu, Jianfu Zhang, Liqing Zhang","year":"2021","venue":"Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Event","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pensia et al., 2018] Ankit Pensia, Varun S. Jog, and Po-Ling Loh. Generalization error bounds for noisy, iterative al- gorithms. In 2018 IEEE International Symposium on In- formation Theory, ISIT 2018, Vail, CO, USA, June 17-22, 2018, pages 546-550. IEEE, 2018.","title":"Generalization error bounds for noisy, iterative algorithms","authors":"Ankit Pensia, S Varun, Po-Ling Jog, Loh","year":"2018","venue":"IEEE International Symposium on Information Theory, ISIT 2018","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Poole et al., 2019] Ben Poole, Sherjil Ozair, Aäron van den Oord, Alexander A. Alemi, and George Tucker. On vari- ational bounds of mutual information. In Proceedings of the 36th International Conference on Machine Learn- ing, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97, pages 5171-5180. PMLR, 2019.","title":"On variational bounds of mutual information","authors":"Ben Poole, Sherjil Ozair, Aäron Van Den, Alexander A Oord, George Alemi, Tucker","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning, ICML 2019","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rodriguez Galvez, 2019] Borja Rodriguez Galvez. The in- formation bottleneck : Connections to other problems, learning and exploration of the ib curve. Master's thesis, KTH, School of Electrical Engineering and Computer Sci- ence (EECS), 2019.","title":"The information bottleneck : Connections to other problems, learning and exploration of the ib curve","authors":"Rodriguez Galvez, ; Borja, Rodriguez Galvez","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Saxe et al., 2018] Andrew M. Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan D. Tracey, and David D. Cox. On the information bottleneck theory of deep learning. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 -May 3, 2018, Conference Track Pro- ceedings. OpenReview.net, 2018.","title":"On the information bottleneck theory of deep learning","authors":"Andrew M Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan D Tracey, David D Cox","year":"2018","venue":"6th International Conference on Learning Representations, ICLR 2018","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shamir et al., 2010] Ohad Shamir, Sivan Sabato, and Naf- tali Tishby. Learning and generalization with the informa- tion bottleneck. Theor. Comput. Sci., 411(29-30):2696- 2711, 2010.","title":"Learning and generalization with the informa-tion bottleneck","authors":"Ohad Shamir, Sivan Sabato, Naftali Tishby","year":"2010","venue":"Theor. Comput. Sci","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E. Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neu- ral networks from overfitting. J. Mach. Learn. Res., 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Steinke and Zakynthinou, 2020] Thomas Steinke and Lydia Zakynthinou. Reasoning about generalization via condi- tional mutual information. In Conference on Learning Theory, COLT 2020, 9-12 July 2020, Virtual Event [Graz, Austria], volume 125, pages 3437-3452. PMLR, 2020.","title":"Reasoning about generalization via conditional mutual information","authors":"Thomas Steinke, Lydia Zakynthinou","year":"2020","venue":"Conference on Learning Theory, COLT 2020","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sugiyama et al., 2012] Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the bregman divergence: a unified framework of density-ratio estimation. Annals of the Institute of Statistical Mathemat- ics, 64(5):1009-1044, Oct 2012.","title":"Density-ratio matching under the bregman divergence: a unified framework of density-ratio estimation","authors":"Sugiyama","year":"2012","venue":"Annals of the Institute of Statistical Mathematics","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Tishby et al., 2000] Naftali Tishby, Fernando C. N. Pereira, and William Bialek. The information bottleneck method. CoRR, physics/0004057, 2000.","title":"The information bottleneck method","authors":"Naftali Tishby, C N Fernando, William Pereira, Bialek","year":"2000","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Vera et al., 2018] Matías Vera, Pablo Piantanida, and Leonardo Rey Vega. The role of the information bot- tleneck in representation learning. In 2018 IEEE Inter- national Symposium on Information Theory, ISIT 2018, Vail, CO, USA, June 17-22, 2018, pages 1580-1584. IEEE, 2018.","title":"The role of the information bottleneck in representation learning","authors":"Matías Vera, Pablo Piantanida, Leonardo Rey Vega","year":"2018","venue":"IEEE International Symposium on Information Theory, ISIT 2018","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2022] Zifeng Wang, Shao-Lun Huang, Er- can Engin Kuruoglu, Jimeng Sun, Xi Chen, and Yefeng Zheng. PAC-bayes information bottleneck. In Interna- tional Conference on Learning Representations, 2022.","title":"PAC-bayes information bottleneck","authors":"Wang","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. CoRR, abs/1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu and Raginsky, 2017] Aolin Xu and Maxim Raginsky. Information-theoretic analysis of generalization capability of learning algorithms. In NeurIPS 2017, December 4-9, 2017, Long Beach, CA, USA, pages 2524-2533, 2017.","title":"Aolin Xu and Maxim Raginsky. Information-theoretic analysis of generalization capability of learning algorithms","authors":"Raginsky Xu","year":"2017","venue":"NeurIPS 2017","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yu et al., 2021] Xi Yu, Shujian Yu, and José C. Príncipe. Deep deterministic information bottleneck with matrix- based entropy functional. In IEEE International Confer- ence on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, pages 3160- 3164. IEEE, 2021.","title":"Príncipe. Deep deterministic information bottleneck with matrixbased entropy functional","authors":"Xi Yu, Shujian Yu, C José","year":"2021","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhou et al., 2022] Ruida Zhou, Chao Tian, and Tie Liu. Individually conditional individual mutual information bound on generalization error. IEEE Trans. Inf. Theory, 68(5):3304-3316, 2022.","title":"Individually conditional individual mutual information bound on generalization error","authors":"Zhou","year":"2022","venue":"IEEE Trans. Inf. Theory","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}