{"type":"ScholarlyArticle","title":"On the Complexity of Learning from Label Proportions","authors":[{"name":"Benjamin Fish","slug":"benjamin-fish","given":"Benjamin","family":"Fish","affiliation":"Department of Mathematics, Statistics, and Computer Science University of Illinois at Chicago , Chicago , IL 60022"},{"name":"Lev Reyzin","slug":"lev-reyzin","given":"Lev","family":"Reyzin","affiliation":"Department of Mathematics, Statistics, and Computer Science University of Illinois at Chicago , Chicago , IL 60022"}],"abstract":"In the problem of learning with label proportions (also known as the problem of estimating class ratios), the training data is unlabeled, and only the proportions of examples receiving each label are given.  The goal is to learn a hypothesis that predicts the proportions of labels on the distribution underlying the sample.  This model of learning is useful in a wide variety of settings, including predicting the number of votes for candidates in political elections from polls.  In this paper, we resolve foundational questions regarding the computational complexity of learning in this setting.  We formalize a simple version of the setting, and we compare the computational complexity of learning in this model to classical PAC learning. Perhaps surprisingly, we show that what can be learned efficiently in this model is a strict subset of what may be leaned efficiently in PAC, under standard complexity assumptions.  We give a characterization in terms of VC dimension, and we show that there are non-trivial problems in this model that can be efficiently learned. We also give an algorithm that demonstrates the feasibility of learning under well-behaved distributions.","doi":"10.24963/ijcai.2017/232","url":"https://ijcai.org/proceedings/2017/232","pdf":"https://ijcai.org/proceedings/2017/0232.pdf","markdown":"https://ijcai.org/proceedings/2017/232.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/232","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":232,"pages":{"first":1675,"last":1681},"keywords":["Machine Learning: Learning Theory","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Ailon and Chazelle, 2005] Nir Ailon and Bernard Chazelle. Lower bounds for linear degeneracy testing. J. ACM, 52(2):157-171, 2005.","title":"Lower bounds for linear degeneracy testing","authors":"Ailon, Chazelle, Nir Ailon, Bernard Chazelle","year":"2005","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blum et al., 2003] Avrim Blum, Adam Kalai, and Hal Wasserman. Noise-tolerant learning, the parity problem, and the statistical query model. J. ACM, 50(4):506-519, 2003.","title":"Noise-tolerant learning, the parity problem, and the statistical query model","authors":"Avrim Blum, Adam Kalai, Hal Wasserman","year":"2003","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2006] Bee-Chung Chen, Lei Chen, Raghu Ra- makrishnan, and David R. Musicant. Learning from ag- gregate views. In Proceedings of the 22nd International Conference on Data Engineering, ICDE 2006, 3-8 April 2006, Atlanta, GA, USA, page 3, 2006.","title":"Learning from aggregate views","authors":"Chen","year":"2006","venue":"Proceedings of the 22nd International Conference on Data Engineering, ICDE 2006","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[de Freitas and Kück, 2005] Nando de Freitas and Hendrik Kück. Learning about individuals from group statistics. In Proceedings of the 21st Conference in Uncertainty in Artificial Intelligence, UAI '05, Edinburgh, Scotland, July 26-29, 2005, pages 332-339, 2005.","title":"Learning about individuals from group statistics","authors":"Kück ; Nando De De Freitas, Hendrik Freitas, Kück","year":"2005","venue":"Proceedings of the 21st Conference in Uncertainty in Artificial Intelligence, UAI '05","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dietterich et al., 1997] Thomas G. Dietterich, Richard H. Lathrop, and Tomás Lozano-Pérez. Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1-2):31-71, 1997.","title":"Solving the multiple instance problem with axis-parallel rectangles","authors":"G Thomas, Richard H Dietterich, Tomás Lathrop, Lozano-Pérez","year":"1997","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fish et al., 2016] Benjamin Fish, Yi Huang, and Lev Reyzin. Recovering social networks by observing votes. In Proceedings of the 2016 International Conference on Au- tonomous Agents & Multiagent Systems, Singapore, May 9-13, 2016, pages 376-384, 2016.","title":"Recovering social networks by observing votes","authors":"Benjamin Fish, Yi Huang, Lev Reyzin","year":"2016","venue":"Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hernández-González et al., 2013] Jerónimo Hernández- González, Iñaki Inza, and José Antonio Lozano. Learning bayesian network classifiers from label proportions. Pattern Recognition, 46(12):3425-3440, 2013.","title":"Learning bayesian network classifiers from label proportions","authors":"Jerónimo Hernández-González, Iñaki Inza, José Antonio Lozano","year":"2013","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Iyer et al., 2014] Arun Shankar Iyer, J. Saketha Nath, and Sunita Sarawagi. Maximum mean discrepancy for class ratio estimation: Convergence bounds and kernel selec- tion. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014, pages 530-538, 2014.","title":"Maximum mean discrepancy for class ratio estimation: Convergence bounds and kernel selection","authors":"Arun Shankar Iyer, J Saketha Nath, Sunita Sarawagi","year":"2014","venue":"Proceedings of the 31th International Conference on Machine Learning, ICML 2014","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Iyer et al., 2016] Arun Shankar Iyer, J. Saketha Nath, and Sunita Sarawagi. Privacy-preserving class ratio estima- tion. In Proceedings of the 22nd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Min- ing, San Francisco, CA, USA, August 13-17, 2016, pages 925-934, 2016.","title":"Privacy-preserving class ratio estimation","authors":"Arun Shankar Iyer, J Saketha Nath, Sunita Sarawagi","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Mohri et al., 2012] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. Adaptive computation and machine learning. MIT Press, 2012.","title":"Foundations of Machine Learning. Adaptive computation and machine learning","authors":"Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Musicant et al., 2007] David R. Musicant, Janara M. Chris- tensen, and Jamie F. Olson. Supervised learning by train- ing on aggregate outputs. In Proceedings of the 7th IEEE International Conference on Data Mining (ICDM 2007), October 28-31, 2007, Omaha, Nebraska, USA, pages 252- 261, 2007.","title":"Supervised learning by training on aggregate outputs","authors":"David R Musicant, M Janara, Jamie F Christensen, Olson","year":"2007","venue":"Proceedings of the 7th IEEE International Conference on Data Mining (ICDM 2007)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Pȃtras ¸cu and Williams, 2010] Mihai Pȃtras ¸cu and Ryan Williams. On the possibility of faster SAT algorithms. In Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010, Austin, Texas, USA, January 17-19, 2010, pages 1065-1075, 2010.","title":"On the possibility of faster SAT algorithms","authors":"Mihai Pȃtras, Ryan Williams","year":"2010","venue":"Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Patrini et al., 2014] Giorgio Patrini, Richard Nock, Tiberio Caetano, and Paul Rivera. (almost) no label no cry. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 190-198, 2014.","title":"almost) no label no cry","authors":"Giorgio Patrini, Richard Nock, Tiberio Caetano, Paul Rivera","year":"2014","venue":"Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Quadrianto et al., 2009] Novi Quadrianto, Alexander J. Smola, Tibério S. Caetano, and Quoc V. Le. Estimating labels from label proportions. Journal of Machine Learn- ing Research, 10:2349-2374, 2009.","title":"Estimating labels from label proportions","authors":"Quadrianto","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rüping, 2010] Stefan Rüping. SVM classifier estimation from group probabilities. In Proceedings of the 27th In- ternational Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel, pages 911-918, 2010.","title":"SVM classifier estimation from group probabilities","authors":"Stefan Rüping","year":"2010","venue":"Proceedings of the 27th International Conference on Machine Learning (ICML-10)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shalev-Shwartz and Ben-David, 2014] Shai Shalev- Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge University Press, 2014.","title":"Understanding machine learning: From theory to algorithms","authors":"Shai Shalev, -Shwartz, Shai Ben-David","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Stolpe and Morik, 2011] Marco Stolpe and Katharina Morik. Learning from label proportions by optimizing cluster model selection. In Proceedings of the Joint Eu- ropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML-PKDD 2011, Athens, Greece, September 5-9, 2011, pages 349-364, 2011.","title":"Learning from label proportions by optimizing cluster model selection","authors":"Morik Stolpe, Marco Stolpe, Katharina Morik","year":"2011","venue":"Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, ECML-PKDD 2011","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wojtusiak et al., 2011] Janusz Wojtusiak, Katherine Irvin, Aybike Birerdinc, and Ancha V. Baranova. Using pub- lished medical results and non-homogenous data in rule learning. In 10th International Conference on Machine Learning and Applications and Workshops, ICMLA 2011, Honolulu, Hawaii, USA, December 18-21, 2011. Volume 2: Special Sessions and Workshop, pages 84-89, 2011.","title":"Using published medical results and non-homogenous data in rule learning","authors":"Janusz Wojtusiak, Katherine Irvin, Aybike Birerdinc, Ancha V Baranova","year":"2011","venue":"10th International Conference on Machine Learning and Applications and Workshops, ICMLA 2011","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yu et al., 2013] Felix X. Yu, Dong Liu, Sanjiv Kumar, Tony Jebara, and Shih-Fu Chang. ∝-SVM for learning with la- bel proportions. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, pages 504-512, 2013.","title":"∝-SVM for learning with label proportions","authors":"Felix X Yu, Dong Liu, Sanjiv Kumar, Tony Jebara, Shih-Fu Chang","year":"2013","venue":"Proceedings of the 30th International Conference on Machine Learning, ICML 2013","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yu et al., 2014] Felix X. Yu, Krzysztof Choromanski, San- jiv Kumar, Tony Jebara, and Shih-Fu Chang. On learning from label proportions. arXiv preprint arXiv:1402.5902, 2014.","title":"On learning from label proportions","authors":"Felix X Yu, Krzysztof Choromanski, Sanjiv Kumar, Tony Jebara, Shih-Fu Chang","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}