{"type":"ScholarlyArticle","title":"No Learner Left Behind: On the Complexity of Teaching Multiple Learners Simultaneously","authors":[{"name":"Xiaojin Zhu","slug":"xiaojin-zhu","given":"Xiaojin","family":"Zhu","affiliation":"University of Wisconsin-Madison"},{"name":"Ji Liu","slug":"ji-liu","given":"Ji","family":"Liu","affiliation":"University of Rochester"},{"name":"Manuel Lopes","slug":"manuel-lopes","given":"Manuel","family":"Lopes","affiliation":"Instituto Superior Técnico / INESC-ID"}],"abstract":"We present a theoretical study of algorithmic teaching in the setting where the teacher must use the same training set to teach multiple learners.  This problem is a theoretical abstraction of the real-world classroom setting in which the teacher delivers the same lecture to academically diverse students.  We define a minimax teaching criterion to guarantee the performance of the worst learner in the class.  We prove that the teaching dimension increases with class diversity in general.  For the classes of conjugate Bayesian learners and linear regression learners, respectively, we exhibit corresponding minimax teaching set.  We then propose a method to enhance teaching by partitioning the class into sections.  We present cases where the optimal partition minimizes overall teaching dimension while maintaining the guarantee on all learners.  Interestingly, we show personalized education (one learner per section) is not necessarily the optimal partition.  Our results generalize algorithmic teaching to multiple learners and offer insight on how to teach large classes.","doi":"10.24963/ijcai.2017/502","url":"https://ijcai.org/proceedings/2017/502","pdf":"https://ijcai.org/proceedings/2017/0502.pdf","markdown":"https://ijcai.org/proceedings/2017/502.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/502","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":502,"pages":{"first":3588,"last":3594},"keywords":["Machine Learning: Learning Theory","Machine Learning: New Problems"],"videos":[],"references":[{"position":1,"raw":"[Alfeld et al., 2016] S. Alfeld, X. Zhu, and P. Barford. Data poisoning attacks against autoregressive models. AAAI, 2016.","title":"Data poisoning attacks against autoregressive models","authors":"S Alfeld, X Zhu, P Barford","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alfeld et al., 2017] S. Alfeld, X. Zhu, and P. Barford. Ex- plicit defense actions against test-set attacks. In AAAI, 2017.","title":"Explicit defense actions against test-set attacks","authors":"S Alfeld, X Zhu, P Barford","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Angluin and Krikis, 1997] D. Angluin and M. Krikis. Teachers, learners and black boxes. COLT, pages 285- 297, 1997.","title":"Teachers, learners and black boxes","authors":"D Angluin, M Krikis","year":"1997","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Angluin and Krikis, 2003] D. Angluin and M. Krikis. Learning from different teachers. Machine Learning, 51(2):137-163, 2003.","title":"Learning from different teachers","authors":"D Angluin, M Krikis","year":"2003","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Angluin, 2004] D. Angluin. Queries revisited. Theoretical Computer Science, 313(2):175-194, 2004.","title":"Queries revisited","authors":"D Angluin","year":"2004","venue":"Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Balbach and Zeugmann, 2006] F. J. Balbach and T. Zeug- mann. Teaching randomized learners. COLT, pages 229- 243, 2006.","title":"Teaching randomized learners","authors":"F J Balbach, T Zeugmann","year":"2006","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Balbach and Zeugmann, 2009] F. J. Balbach and T. Zeug- mann. Recent developments in algorithmic teaching. In Proceedings of the 3rd International Conference on Lan- guage and Automata Theory and Applications, pages 1- 18, 2009.","title":"Recent developments in algorithmic teaching","authors":"F J Balbach, T Zeugmann","year":"2009","venue":"Proceedings of the 3rd International Conference on Language and Automata Theory and Applications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Balbach, 2008] F. J. Balbach. Measuring teachability using variants of the teaching dimension. Theor. Comput. Sci., 397(1-3):94-113, 2008.","title":"Measuring teachability using variants of the teaching dimension","authors":"F J Balbach","year":"2008","venue":"Theor. Comput. Sci","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ben-David and Eiron, 1998] S. Ben-David and Nadav Eiron. Self-directed learning and its relation to the VC-dimension and to teacher-directed learning. Machine Learning, 33(1):87-104, 1998.","title":"Self-directed learning and its relation to the VC-dimension and to teacher-directed learning","authors":"S Ben-David, Nadav Eiron","year":"1998","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Cakmak and Thomaz, 2011] M. Cakmak and A. Thomaz. Mixed-initiative active learning. ICML Workshop on Com- bining Learning Strategies to Reduce Label Cost, 2011.","title":"Mixed-initiative active learning","authors":"M Cakmak, A Thomaz","year":"2011","venue":"ICML Workshop on Combining Learning Strategies to Reduce Label Cost","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Clement et al., 2016] B. Clement, P.-Y. Oudeyer, and M. Lopes. A comparison of automatic teaching strategies for heterogeneous student populations. In Educational Data Mining (EDM), 2016.","title":"A comparison of automatic teaching strategies for heterogeneous student populations","authors":"B Clement, P.-Y Oudeyer, M Lopes","year":"2016","venue":"Educational Data Mining (EDM)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Doliwa et al., 2014] T. Doliwa, G. Fan, H. U. Simon, and S. Zilles. Recursive teaching dimension, VC-dimension and sample compression. Journal of Machine Learning Research, 15:3107-3131, 2014.","title":"Recursive teaching dimension, VC-dimension and sample compression","authors":"T Doliwa, G Fan, H U Simon, S Zilles","year":"2014","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Goldman and Kearns, 1995] S. Goldman and M. Kearns. On the complexity of teaching. Journal of Computer and Systems Sciences, 50(1):20-31, 1995.","title":"On the complexity of teaching","authors":"S Goldman, M Kearns","year":"1995","venue":"Journal of Computer and Systems Sciences","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Goldman and Mathias, 1996] S. A. Goldman and H. D. Mathias. Teaching a smarter learner. Journal of Computer and Systems Sciences, 52(2):255267, 1996.","title":"Teaching a smarter learner","authors":"S A Goldman, H D Mathias","year":"1996","venue":"Journal of Computer and Systems Sciences","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kobayashi and Shinohara, 2009] H. Kobayashi and A. Shi- nohara. Complexity of teaching by a restricted number of examples. In COLT, pages 293-302, 2009.","title":"Complexity of teaching by a restricted number of examples","authors":"H Kobayashi, A Shinohara","year":"2009","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lee and Brunskill, 2012] J.I. Lee and E. Brunskill. The im- pact on individualizing student models on necessary prac- tice opportunities. In EDM, 2012.","title":"The impact on individualizing student models on necessary practice opportunities","authors":"J I Lee, E Brunskill","year":"2012","venue":"EDM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lindsey et al., 2013] R. Lindsey, Michael Mozer, William J Huggins, and Harold Pashler. Optimizing instructional policies. In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 2778- 2786. 2013.","title":"Optimizing instructional policies","authors":"R Lindsey, Michael Mozer, William J Huggins, Harold Pashler","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2016] J. Liu, X. Zhu, and H. G. Ohannessian. The teaching dimension of linear learners. In ICML, 2016.","title":"The teaching dimension of linear learners","authors":"J Liu, X Zhu, H G Ohannessian","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mathias, 1997] H. David Mathias. A model of interactive teaching. J. Comput. Syst. Sci., 54(3):487-501, 1997.","title":"A model of interactive teaching","authors":"H, David Mathias","year":"1997","venue":"J. Comput. Syst. Sci","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mei and Zhu, 2015] S. Mei and X. Zhu. Using machine teaching to identify optimal training-set attacks on ma- chine learners. AAAI, 2015.","title":"Using machine teaching to identify optimal training-set attacks on machine learners","authors":"S Mei, X Zhu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Patil et al., 2014] K. Patil, X. Zhu, L. Kopec, and B. C. Love. Optimal teaching for limited-capacity human learn- ers. NIPS, pages 2465-2473, 2014.","title":"Optimal teaching for limited-capacity human learners","authors":"K Patil, X Zhu, L Kopec, B C Love","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rivest and Yin, 1995] R. L. Rivest and Y. L. Yin. Being taught can be faster than asking questions. COLT, pages 144-151, 1995.","title":"Being taught can be faster than asking questions","authors":"R L Rivest, Y L Yin","year":"1995","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shinohara and Miyano, 1991] A. Shinohara and S. Miyano. Teachability in computational learning. New Generation Computing, 8(4):337-348, 1991.","title":"Teachability in computational learning","authors":"A Shinohara, S Miyano","year":"1991","venue":"New Generation Computing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Singla et al., 2014] A. Singla, I. Bogunovic, G. Bartok, A. Karbasi, and A. Krause. Near-optimally teaching the crowd to classify. In ICML, pages 154-162, 2014.","title":"Near-optimally teaching the crowd to classify","authors":"A Singla, I Bogunovic, G Bartok, A Karbasi, A Krause","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Stigler and Hiebert, 1998] J. W Stigler and J. Hiebert. Teaching is a cultural activity. American Educator, 22(4):4-11, 1998.","title":"Teaching is a cultural activity","authors":"J W Stigler, J Hiebert","year":"1998","venue":"American Educator","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Suh et al., 2016] J. Suh, X. Zhu, and S. Amershi. The label complexity of mixed-initiative classifier training. ICML, 2016.","title":"The label complexity of mixed-initiative classifier training","authors":"J Suh, X Zhu, S Amershi","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Valiant, 1984] L. G. Valiant. A theory of the learnable. In STOC '84: Proceedings of the sixteenth annual ACM symposium on Theory of computing, pages 436-445, New York, NY, USA, 1984. ACM.","title":"A theory of the learnable","authors":"L G Valiant","year":"1984","venue":"STOC '84: Proceedings of the sixteenth annual ACM symposium on Theory of computing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vapnik, 1995] Vladimir Vapnik. The Nature of Statistical Learning Theory. Springer, second edition, 1995.","title":"The Nature of Statistical Learning Theory","authors":"Vladimir Vapnik","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhu, 2013] X. Zhu. Machine teaching for Bayesian learners in the exponential family. NIPS, pages 1905-1913, 2013.","title":"Machine teaching for Bayesian learners in the exponential family","authors":"X Zhu","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhu, 2015] X. Zhu. Machine teaching: an inverse problem to machine learning and an approach toward optimal edu- cation. AAAI, 2015.","title":"Machine teaching: an inverse problem to machine learning and an approach toward optimal education","authors":"X Zhu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zilles et al., 2011] S. Zilles, S. Lange, R. Holte, and M. Zinkevich. Models of cooperative teaching and learn- ing. Journal of Machine Learning Research, 12:349-384, 2011.","title":"Models of cooperative teaching and learning","authors":"S Zilles, S Lange, R Holte, M Zinkevich","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}