{"type":"ScholarlyArticle","title":"Neural Networks for Predicting Human Interactions in Repeated Games","authors":[{"name":"Yoav Kolumbus","slug":"yoav-kolumbus","given":"Yoav","family":"Kolumbus","affiliation":"The School of Computer Science and Engineering , Hebrew University of Jerusalem , Israel.; Racah Institute of Physics , Hebrew University of Jerusalem , Israel."},{"name":"Gali Noti","slug":"gali-noti","given":"Gali","family":"Noti","affiliation":"The School of Computer Science and Engineering , Hebrew University of Jerusalem , Israel.; Federmann Center for the Study of Rationality , Hebrew University of Jerusalem , Israel."}],"abstract":"We consider the problem of predicting human players' actions in repeated strategic interactions. Our goal is to predict the dynamic step-by-step behavior of individual players in previously unseen games. We study the ability of neural networks to perform such predictions and the information that they require. We show on a dataset of normal-form games from experiments with human participants that standard neural networks are able to learn functions that provide more accurate predictions of the players' actions than established models from behavioral economics. The networks outperform the other models in terms of prediction accuracy and cross-entropy, and yield higher economic value. We show that if the available input is only of a short sequence of play, economic information about the game is important for predicting behavior of human agents. However, interestingly, we find that when the networks are trained with long enough sequences of history of play, action-based networks do well and additional economic details about the game do not improve their performance, indicating that the sequence of actions encode sufficient information for the success in the prediction task.","doi":"10.24963/ijcai.2019/56","url":"https://ijcai.org/proceedings/2019/56","pdf":"https://ijcai.org/proceedings/2019/0056.pdf","markdown":"https://ijcai.org/proceedings/2019/56.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/56","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Agent-based and Multi-agent Systems","track":"Main track","number":56,"pages":{"first":392,"last":399},"keywords":["Machine Learning Applications: Applications of Supervised Learning","Humans and AI: Cognitive Modeling","Agent-based and Multi-agent Systems: Algorithmic Game Theory"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2015] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Joze- fowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Lev- enberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. Ten- sorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.","title":"Ten-sorFlow: Large-scale machine learning on heterogeneous systems","authors":"Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, Xiaoqiang Zheng","year":"2015","venue":"Software available from tensorflow","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Abdel-Hamid et al., 2014] Ossama Abdel-Hamid, Abdel- rahman Mohamed, Hui Jiang, Li Deng, Gerald Penn, and Dong Yu. Convolutional neural networks for speech recog- nition. IEEE/ACM Transactions on audio, speech, and language processing, 22(10):1533-1545, 2014.","title":"Convolutional neural networks for speech recognition","authors":"Ossama Abdel-Hamid, Abdelrahman Mohamed, Hui Jiang, Li Deng, Gerald Penn, Dong Yu","year":"2014","venue":"IEEE/ACM Transactions on audio, speech, and language processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2003] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilis- tic language model. Journal of machine learning research, 3(Feb):1137-1155, 2003.","title":"A neural probabilistic language model","authors":"Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin","year":"2003","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Camerer, 2011] Colin F Camerer. Behavioral game theory: Experiments in strategic interaction. Princeton University Press, 2011.","title":"Behavioral game theory: Experiments in strategic interaction","authors":"Colin F Camerer","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chollet, 2015] Franc ¸ois et al. Chollet. Keras. https://keras. io, 2015.","title":"Keras","authors":"Chollet ; Franc ¸ois","year":"2015","venue":null,"doi":null,"url":"https://keras.io","internal_url":null},{"position":6,"raw":"[Erev and Haruvy, 2013] Ido Erev and Ernan Haruvy. Learn- ing and the economics of small decisions. The handbook of experimental economics, 2:9781400883172-011, 2013.","title":"Learning and the economics of small decisions","authors":"Haruvy Erev, Ido Erev, Ernan Haruvy","year":"2013","venue":"The handbook of experimental economics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Erev and Roth, 1998] Ido Erev and Alvin E Roth. Predict- ing how people play games: Reinforcement learning in ex- perimental games with unique, mixed strategy equilibria. American economic review, pages 848-881, 1998.","title":"Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria","authors":"Roth ; ] Ido Erev, Alvin E Erev, Roth","year":"1998","venue":"American economic review","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Erev et al., 2007] Ido Erev, Alvin E Roth, Robert L Slonim, and Greg Barron. Learning and equilibrium as useful ap- proximations: Accuracy of prediction on randomly se- lected constant sum games. Economic Theory, 33(1):29- 51, 2007.","title":"Learning and equilibrium as useful approximations: Accuracy of prediction on randomly selected constant sum games","authors":"Ido Erev, Alvin E Roth, Robert L Slonim, Greg Barron","year":"2007","venue":"Economic Theory","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Erev et al., 2010] Ido Erev, Eyal Ert, and Alvin Roth. A choice prediction competition for market entry games: An introduction. Games, 1(2):117-136, 2010.","title":"A choice prediction competition for market entry games: An introduction","authors":"Ido Erev, Eyal Ert, Alvin Roth","year":"2010","venue":"Games","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ert and Erev, 2007] Eyal Ert and Ido Erev. Replicated al- ternatives and the role of confusion, chasing, and regret in decisions from experience. Journal of Behavioral Deci- sion Making, 20(3):305-322, 2007.","title":"Replicated alternatives and the role of confusion, chasing, and regret in decisions from experience","authors":"Erev Ert, Eyal Ert, Ido Erev","year":"2007","venue":"Journal of Behavioral Decision Making","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gehring et al., 2017] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convo- lutional sequence to sequence learning. arXiv preprint arXiv:1705.03122, 2017.","title":"Convolutional sequence to sequence learning","authors":"Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N Dauphin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gers et al., 1999] Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. Learning to forget: Continual prediction with lstm. 1999.","title":"Learning to forget: Continual prediction with lstm","authors":"Jürgen Felix A Gers, Fred Schmidhuber, Cummins","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Greff et al., 2017] Klaus Greff, Rupesh K Srivastava, Jan Koutník, Bas R Steunebrink, and Jürgen Schmidhuber. Lstm: A search space odyssey. IEEE transactions on neural networks and learning systems, 28(10):2222-2232, 2017.","title":"Lstm: A search space odyssey","authors":"Klaus Greff, K Rupesh, Jan Srivastava, Koutník, Jürgen Bas R Steunebrink, Schmidhuber","year":"2017","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hartford et al., 2016] Jason S Hartford, James R Wright, and Kevin Leyton-Brown. Deep learning for predicting human strategic behavior. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Ad- vances in Neural Information Processing Systems 29, pages 2424-2432.","title":"Deep learning for predicting human strategic behavior","authors":"Jason S Hartford, James R Wright, Kevin Leyton-Brown","year":null,"venue":"Advances in Neural Information Processing Systems 29","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Curran Associates, Inc., 2016.","title":null,"authors":"Curran Associates, Inc","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kagel and Levin, 2010] John H. Kagel and Dan Levin. The Handbook of Experimental Economics, Volume II, chap- ter Auctions: A Survey of Experimental Research, 1995 - 2008. Princeton: Princeton University Press, 2010.","title":"The Handbook of Experimental Economics, Volume II, chapter Auctions: A Survey of Experimental Research","authors":"John H Kagel, Dan Levin","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kagel and Roth, 1995] John H. Kagel and Alvin E. Roth. The Handbook of Experimental Economics. Princeton University Press, Princeton, N.J., 1995.","title":"The Handbook of Experimental Economics","authors":"John H Kagel, Alvin E Roth","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[McKelvey and Palfrey, 1995] Richard D McKelvey and Thomas R Palfrey. Quantal response equilibria for normal form games. Games and economic behavior, 10(1):6-38, 1995.","title":"Quantal response equilibria for normal form games","authors":"D Richard, Thomas R Mckelvey, Palfrey","year":"1995","venue":"Games and economic behavior","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Nisan and Noti, 2017] Noam Nisan and Gali Noti. A \"quan- tal regret\" method for structural econometrics in repeated games. In Proceedings of the 2017 ACM Conference on Economics and Computation, EC '17, pages 123-123, New York, NY, USA, 2017. ACM.","title":"Noam Nisan and Gali Noti. A \"quantal regret\" method for structural econometrics in repeated games","authors":"Noti Nisan","year":"2017","venue":"Proceedings of the 2017 ACM Conference on Economics and Computation, EC '17","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Osborne and Rubinstein, 1998] Martin J. Osborne and Ariel Rubinstein. Games with procedurally rational players. The American Economic Review, 88(4):834-847, 1998.","title":"Games with procedurally rational players","authors":"Martin J Osborne, Ariel Rubinstein","year":"1998","venue":"The American Economic Review","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rosenfeld and Kraus, 2018] Ariel Rosenfeld and Sarit Kraus. Predicting human decision-making: From predic- tion to action. Synthesis Lectures on Artificial Intelligence and Machine Learning, 12(1):1-150, 2018.","title":"Predicting human decision-making: From prediction to action","authors":"Kraus Rosenfeld, Ariel Rosenfeld, Sarit Kraus","year":"2018","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rosenfeld, 2000] Ronald Rosenfeld. Two decades of sta- tistical language modeling: Where do we go from here? Proceedings of the IEEE, 88(8):1270-1278, 2000.","title":"Two decades of statistical language modeling: Where do we go from here","authors":"Ronald Rosenfeld","year":"2000","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Selten and Chmura, 2008] Reinhard Selten and Thorsten Chmura. Stationary concepts for experimental 2x2-games. American Economic Review, 98(3):938-66, June 2008.","title":"Stationary concepts for experimental 2x2-games","authors":"Chmura Selten, Reinhard Selten, Thorsten Chmura","year":"2008","venue":"American Economic Review","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Selten et al., 2005] Reinhard Selten, Klaus Abbink, and Ri- carda Cox. Learning direction theory and the winner's curse. Experimental Economics, 8(1):5-20, 2005.","title":"Learning direction theory and the winner's curse","authors":"Reinhard Selten, Klaus Abbink, Ricarda Cox","year":"2005","venue":"Experimental Economics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Re- search, 15(1):1929-1958, 2014.","title":"Dropout: A simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sundermeyer et al., 2012] Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. Lstm neural networks for language modeling. In Thirteenth annual conference of the international speech communication association, 2012.","title":"Lstm neural networks for language modeling","authors":"Martin Sundermeyer, Ralf Schlüter, Hermann Ney","year":"2012","venue":"Thirteenth annual conference of the international speech communication association","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabi- novich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.","title":"Going deeper with convolutions","authors":"Szegedy","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}