{"type":"ScholarlyArticle","title":"Deep Forest: Towards An Alternative to Deep Neural Networks","authors":[{"name":"Zhi-Hua Zhou","slug":"zhi-hua-zhou","given":"Zhi-Hua","family":"Zhou","affiliation":"National Key Lab for Novel Software Technology , Nanjing University , Nanjing 210023 , China"},{"name":"Ji Feng","slug":"ji-feng","given":"Ji","family":"Feng","affiliation":"National Key Lab for Novel Software Technology , Nanjing University , Nanjing 210023 , China"}],"abstract":"In this paper, we propose gcForest, a decision tree ensemble approach with performance highly competitive to deep neural networks in a broad range of tasks. In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train; even when it is applied to different data across different domains in our experiments, excellent performance can be achieved by almost same settings of hyper-parameters. The training process of gcForest is efficient, and users can control training cost according to computational resource available. The efficiency may be further enhanced because gcForest is naturally apt to parallel implementation. Furthermore, in contrast to deep neural networks which require large-scale training data, gcForest can work well even when there are only small-scale training data.","doi":"10.24963/ijcai.2017/497","url":"https://ijcai.org/proceedings/2017/497","pdf":"https://ijcai.org/proceedings/2017/0497.pdf","markdown":"https://ijcai.org/proceedings/2017/497.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/497","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":497,"pages":{"first":3553,"last":3559},"keywords":["Machine Learning: Deep Learning","Machine Learning: Ensemble Methods"],"videos":[],"references":[{"position":1,"raw":"[Breiman, 1996] L. Breiman. Stacked regressions. Machine Learning, 24(1):49-64, 1996.","title":"Stacked regressions","authors":"L Breiman","year":"1996","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Breiman, 2000] L. Breiman. Randomizing outputs to in- crease prediction accuracy. Machine Learning, 40(3):113- 120, 2000.","title":"Randomizing outputs to increase prediction accuracy","authors":"L Breiman","year":"2000","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Breiman, 2001] L. Breiman. Random forests. Machine Learning, 45(1):5-32, 2001.","title":"Random forests","authors":"L Breiman","year":"2001","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] K. Cho, B. van Meriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Ben- gio. Learning phrase representations using RNN encoder- decoder for statistical machine translation. In EMNLP, pages 1724-1734, 2014.","title":"Learning phrase representations using RNN encoderdecoder for statistical machine translation","authors":"K Cho, B Van Meriënboer, C Gulcehre, D Bahdanau, F Bougares, H Schwenk, Y Bengio","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dietterich et al., 1997] T. G. Dietterich, R. H. Lathrop, and T. Lozano-Pérez. Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1- 2):31-71, 1997.","title":"Solving the multiple-instance problem with axis-parallel rectangles","authors":"T G Dietterich, R H Lathrop, T Lozano-Pérez","year":"1997","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Freund and Schapire, 1997] Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and Sys- tem Sciences, 55(1):119-139, 1997.","title":"A decision-theoretic generalization of on-line learning and an application to boosting","authors":"Y Freund, R E Schapire","year":"1997","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gers et al., 2001] F. A. Gers, D. Eck, and J. Schmidhuber. Applying LSTM to time series predictable through time- window approaches. In ICANN, pages 669-676, 2001.","title":"Applying LSTM to time series predictable through timewindow approaches","authors":"F A Gers, D Eck, J Schmidhuber","year":"2001","venue":"ICANN","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2016] I.","title":null,"authors":"Goodfellow","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Goodfellow, Y. Bengio, and A. Courville. Deep Learning. MIT Press, Cambridge, MA, 2016.","title":"Deep Learning","authors":"Y Goodfellow, A Bengio, Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Graves et al., 2013] A. Graves, A. R. Mohamed, and G. Hinton. Speech recognition with deep recurrent neu- ral networks. In ICASSP, pages 6645-6649, 2013.","title":"Speech recognition with deep recurrent neural networks","authors":"A Graves, A R Mohamed, G Hinton","year":"2013","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hinton et al., 2006] G. E. Hinton, S. Osindero, and Y.-W. Simon. A fast learning algorithm for deep belief nets. Neu- ral Computation, 18(7):1527-1554, 2006.","title":"A fast learning algorithm for deep belief nets","authors":"G E Hinton, S Osindero, Y.-W Simon","year":"2006","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hinton et al., 2012] G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, and B. Kingbury. Deep neural net- works for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 29(6):82-97, 2012.","title":"Deep neural networks for acoustic modeling in speech recognition","authors":"G Hinton, L Deng, D Yu, G Dahl, A Mohamed, N Jaitly, A Senior, V Vanhoucke, P Nguyen, T Sainath, B Kingbury","year":"2012","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kim, 2014] Y. Kim. Convolutional neural networks for sen- tence classification. arXiv:1408.5882, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Y Kim","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kontschieder et al., 2015] P. Kontschieder, M. Fiterau, A. Criminisi, and S. R. Bulò. Deep neural decision forests. In ICCV, pages 1467-1475, 2015.","title":"Deep neural decision forests","authors":"P Kontschieder, M Fiterau, A Criminisi, S R Bulò","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":"/proceedings/2016/628"},{"position":15,"raw":"[Krizhenvsky et al., 2012] A. Krizhenvsky, I. Sutskever, and G. Hinton. ImageNet classification with deep convolu- tional neural networks. In NIPS, pages 1097-1105. 2012.","title":"ImageNet classification with deep convolutional neural networks","authors":"A Krizhenvsky, I Sutskever, G Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kuncheva and Whitaker, 2003] L. I. Kuncheva and C. J. Whitaker. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Ma- chine Learning, 51(2):181-207, 2003.","title":"Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy","authors":"L I Kuncheva, C J Whitaker","year":"2003","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[LeCun et al., 1998] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Y Lecun, L Bottou, Y Bengio, P Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lichman, 2013] M. Lichman. UCI machine learning repos- itory, 2013.","title":"UCI machine learning repository","authors":"M Lichman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2008] F. T. Liu, K. M. Ting, Y. Yu, and Z.-H. Zhou. Spectrum of variable-random trees. Journal of Ar- tificial Intelligence Research, 32:355-384, 2008.","title":"Spectrum of variable-random trees","authors":"F T Liu, K M Ting, Y Yu, Z.-H Zhou","year":"2008","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Maas et al., 2011] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning word vectors for sentiment analysis. In ACL, pages 142-150, 2011.","title":"Learning word vectors for sentiment analysis","authors":"A L Maas, R E Daly, P T Pham, D Huang, A Y Ng, C Potts","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mallat, 1999] S. Mallat. A Wavelet Tour of Signal Process- ing. Academic Press, London, UK, 2nd edition, 1999.","title":"A Wavelet Tour of Signal Processing","authors":"S Mallat","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Maron and Lozano-Pérez, 1998] O. Maron and T. Lozano- Pérez. A framework for multiple-instance learning. In NIPS, pages 570-576. 1998.","title":"A framework for multiple-instance learning","authors":"O Maron, T Lozano-Pérez","year":"1998","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mhaskar et al., 2017] H. Mhaskar, Q. Liao, and T. A. Pog- gio. When and why are deep networks better than shallow ones? In AAAI, pages 2343-2349, 2017.","title":"When and why are deep networks better than shallow ones?","authors":"H Mhaskar, Q Liao, T A Poggio","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Samaria and Harter, 1994] F. Samaria and A. C. Harter. Pa- rameterisation of a stochastic model for human face iden- tification. In 2nd IEEE Workshop on Applications of Com- puter Vision, pages 138-142, 1994.","title":"Parameterisation of a stochastic model for human face identification","authors":"F Samaria, A C Harter","year":"1994","venue":"2nd IEEE Workshop on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sapsanis et al., 2013] C. Sapsanis, G. Georgoulas, A. Tzes, and D. Lymberopoulos. Improving EMG based classifi- cation of basic hand movements using EMD. In 35th An- nual International Conference on the IEEE Engineering in Medicine and Biology Society, pages 5754-5757, 2013.","title":"Improving EMG based classification of basic hand movements using EMD","authors":"C Sapsanis, G Georgoulas, A Tzes, D Lymberopoulos","year":"2013","venue":"35th Annual International Conference on the IEEE Engineering in Medicine and Biology Society","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Simonyan and Zisserman, 2014] K. Simonyan and A. Zis- serman. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"K Simonyan, A Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ting and Witten, 1999] K. M. Ting and I. H. Witten. Issues in stacked generalization. Journal of Artificial Intelligence Research, 10:271-289, 1999.","title":"Issues in stacked generalization","authors":"K M Ting, I H Witten","year":"1999","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tzanetakis and Cook, 2002] G. Tzanetakis and P. R. Cook. Musical genre classification of audio signals. IEEE Trans. Speech and Audio Processing, 10(5):293-302, 2002.","title":"Musical genre classification of audio signals","authors":"G Tzanetakis, P R Cook","year":"2002","venue":"IEEE Trans. Speech and Audio Processing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Viola and Jones, 2001] P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In CVPR, pages 511-518, 2001.","title":"Rapid object detection using a boosted cascade of simple features","authors":"P Viola, M Jones","year":"2001","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Webb, 2000] G. I. Webb. MultiBoosting: A technique for combining boosting and wagging. Machine Learning, 40(2):159-196, 2000.","title":"MultiBoosting: A technique for combining boosting and wagging","authors":"G I Webb","year":"2000","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wei and Zhou, 2016] X.-S. Wei and Z.-H. Zhou. An em- pirical study on image bag generators for multi-instance learning. Machine Learning, 105(2):155-198, 2016.","title":"An empirical study on image bag generators for multi-instance learning","authors":"Zhou ; X.-S Wei, Z.-H Wei, Zhou","year":"2016","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wolpert, 1992] D. H. Wolpert. Stacked generalization. Neu- ral Networks, 5(2):241-260, 1992.","title":"Stacked generalization","authors":"D H Wolpert","year":"1992","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhou, 2012] Z.-H. Zhou. Ensemble Methods: Foundations and Algorithms. CRC, Boca Raton, FL, 2012.","title":"Ensemble Methods: Foundations and Algorithms","authors":"Z.-H Zhou","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}