{"type":"ScholarlyArticle","title":"Inter-node Hellinger Distance based Decision Tree","authors":[{"name":"Pritom Saha Akash","slug":"pritom-saha-akash","given":"Pritom Saha","family":"Akash","affiliation":""},{"name":"Md. Eusha Kadir","slug":"md-eusha-kadir","given":"Md. Eusha","family":"Kadir","affiliation":"Institute of Information Technology , University of Dhaka , Bangladesh"},{"name":"Amin Ahsan Ali","slug":"amin-ahsan-ali","given":"Amin Ahsan","family":"Ali","affiliation":"Department of Computer Science & Engineering , Independent University , Bangladesh"},{"name":"Mohammad Shoyaib","slug":"mohammad-shoyaib","given":"Mohammad","family":"Shoyaib","affiliation":"Institute of Information Technology , University of Dhaka , Bangladesh"}],"abstract":"This paper introduces a new splitting criterion called Inter-node Hellinger Distance (iHD) and a weighted version of it (iHDw) for constructing decision trees. iHD measures the distance between the parent and each of the child nodes in a split using Hellinger distance. We prove that this ensures the mutual exclusiveness between the child nodes. The weight term in iHDw is concerned with the purity of individual child node considering the class imbalance problem. The combination of the distance and weight term in iHDw thus favors a partition where child nodes are purer and mutually exclusive, and skew insensitive. We perform an experiment over twenty balanced and twenty imbalanced datasets. The results show that decision trees based on iHD win against six other state-of-the-art methods on at least 14 balanced and 10 imbalanced datasets. We also observe that adding the weight to iHD improves the performance of decision trees on imbalanced datasets. Moreover, according to the result of the Friedman test, this improvement is statistically significant compared to other methods.","doi":"10.24963/ijcai.2019/272","url":"https://ijcai.org/proceedings/2019/272","pdf":"https://ijcai.org/proceedings/2019/0272.pdf","markdown":"https://ijcai.org/proceedings/2019/272.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/272","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":272,"pages":{"first":1967,"last":1973},"keywords":["Machine Learning: Classification","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Alcalá-Fdez et al., 2011] Jesús Alcalá-Fdez, Alberto Fernández, Julián Luengo, Joaquín Derrac, and Salvador García. Keel data-mining software tool: Data set reposi- tory, integration of algorithms and experimental analysis framework. Journal of Multiple-Valued Logic and Soft Computing, 17(2-3):255-287, 2011.","title":"Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework","authors":"Alcalá-Fdez","year":"2011","venue":"Journal of Multiple-Valued Logic and Soft Computing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Breiman et al., 1984] Leo Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.","title":"Classification and Regression Trees","authors":"J H Leo Breiman, R A Friedman, C J Olshen, Stone","year":"1984","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chandra et al., 2010] B Chandra, Ravi Kothari, and Pallath Paul. A new node splitting measure for decision tree con- struction. Pattern Recognition, 43(8):2725-2731, 2010.","title":"A new node splitting measure for decision tree construction","authors":"Ravi Chandra, Pallath Kothari, Paul","year":"2010","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chawla et al., 2004] Nitesh V Chawla, Nathalie Japkowicz, and Aleksander Kotcz. Special issue on learning from im- balanced data sets. ACM Sigkdd Explorations Newsletter, 6(1):1-6, 2004.","title":"Special issue on learning from imbalanced data sets","authors":"Nathalie Nitesh V Chawla, Aleksander Japkowicz, Kotcz","year":"2004","venue":"ACM Sigkdd Explorations Newsletter","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cichocki and Amari, 2010] Andrzej Cichocki and Shun- ichi Amari. Families of alpha-beta-and gamma- divergences: Flexible and robust measures of similarities. Entropy, 12(6):1532-1568, 2010.","title":"Families of alpha-beta-and gammadivergences: Flexible and robust measures of similarities","authors":"Andrzej Cichocki, Shunichi Amari","year":"2010","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cieslak and Chawla, 2008] David A Cieslak and Nitesh V Chawla. Learning decision trees for unbalanced data. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 241-256. Springer, 2008.","title":"Learning decision trees for unbalanced data","authors":"A David, Nitesh V Cieslak, Chawla","year":"2008","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cieslak et al., 2012] David A Cieslak, T Ryan Hoens, Nitesh V Chawla, and W Philip Kegelmeyer. Hellinger distance decision trees are robust and skew-insensitive. Data Mining and Knowledge Discovery, 24(1):136-158, 2012.","title":"Hellinger distance decision trees are robust and skew-insensitive","authors":"Ryan David A Cieslak, Nitesh V Hoens, Philip Chawla, Kegelmeyer","year":"2012","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Demšar, 2006] Janez Demšar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning research, 7(Jan):1-30, 2006.","title":"Janez Demšar. Statistical comparisons of classifiers over multiple data sets","authors":"Demšar","year":"2006","venue":"Journal of Machine learning research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Drummond and Holte, 2000] Chris Drummond and Robert C Holte. Exploiting the cost (in) sensitivity of decision tree splitting criteria. In ICML, volume 1, pages 239-246, 2000.","title":"Exploiting the cost (in) sensitivity of decision tree splitting criteria","authors":"Chris Drummond, Robert C Holte","year":"2000","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dua and Graff, 2017] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.","title":"Dheeru Dua and Casey Graff. UCI machine learning repository","authors":"Graff Dua","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Friedman, 1937] Milton Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the american statistical associa- tion, 32(200):675-701, 1937.","title":"The use of ranks to avoid the assumption of normality implicit in the analysis of variance","authors":"Milton Friedman","year":"1937","venue":"Journal of the american statistical association","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Friedman, 1940] Milton Friedman. A comparison of alter- native tests of significance for the problem of m rankings. The Annals of Mathematical Statistics, 11(1):86-92, 1940.","title":"A comparison of alternative tests of significance for the problem of m rankings","authors":"Milton Friedman","year":"1940","venue":"The Annals of Mathematical Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hand and Till, 2001] David J Hand and Robert J Till. A simple generalisation of the area under the roc curve for multiple class classification problems. Machine learning, 45(2):171-186, 2001.","title":"A simple generalisation of the area under the roc curve for multiple class classification problems","authors":"J David, Robert J Hand, Till","year":"2001","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hoens et al., 2012] T Ryan Hoens, Qi Qian, Nitesh V Chawla, and Zhi-Hua Zhou. Building decision trees for the multi-class imbalance problem. In Pacific-Asia Con- ference on Knowledge Discovery and Data Mining, pages 122-134. Springer, 2012.","title":"Building decision trees for the multi-class imbalance problem","authors":"Ryan Hoens, Qi Qian, Nitesh V Chawla, Zhi-Hua Zhou","year":"2012","venue":"Pacific-Asia Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Iman and Davenport, 1980] Ronald L Iman and James M Davenport. Approximations of the critical region of the fbietkan statistic. Communications in Statistics-Theory and Methods, 9(6):571-595, 1980.","title":"Approximations of the critical region of the fbietkan statistic","authors":"L Ronald, James M Iman, Davenport","year":"1980","venue":"Communications in Statistics-Theory and Methods","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Iqbal et al., 2017] Md Tauhid Bin Iqbal, Mohammad Shoy- aib, Byungyong Ryu, M Abdullah-Al-Wadud, and Oksam Chae. Directional age-primitive pattern (dapp) for human age group recognition and age estimation. IEEE Transac- tions on Information Forensics and Security, 12(11):2505- 2517, 2017.","title":"Directional age-primitive pattern (dapp) for human age group recognition and age estimation","authors":"Md Tauhid, Bin Iqbal, Mohammad Shoyaib, Byungyong Ryu, M Abdullah-Al-Wadud, Oksam Chae","year":"2017","venue":"IEEE Transactions on Information Forensics and Security","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kotsiantis et al., 2007] Sotiris B Kotsiantis, I Zaharakis, and P Pintelas. Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering, 160:3-24, 2007.","title":"Supervised machine learning: A review of classification techniques","authors":"I Sotiris B Kotsiantis, Zaharakis, Pintelas","year":"2007","venue":"Emerging artificial intelligence applications in computer engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2010] Wei Liu, Sanjay Chawla, David A Cieslak, and Nitesh V Chawla. A robust decision tree algorithm for imbalanced data sets. In Proceedings of the 2010 SIAM International Conference on Data Mining, pages 766-777.","title":"A robust decision tree algorithm for imbalanced data sets","authors":"Wei Liu, Sanjay Chawla, David A Cieslak, Nitesh V Chawla","year":null,"venue":"Proceedings of the 2010 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nemenyi, 1963] Peter Nemenyi. Distribution-free multiple comparisons. PhD thesis, Princeton University, 1963.","title":"Distribution-free multiple comparisons","authors":"Peter Nemenyi","year":"1963","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Quinlan, 1986] J. Ross Quinlan. Induction of decision trees. Machine learning, 1(1):81-106, 1986.","title":"Induction of decision trees","authors":"J, Ross Quinlan","year":"1986","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Quinlan, 1993] J. Ross Quinlan. C4.5: Programs for Ma- chine Learning. Morgan Kaufmann Publishers Inc., 1993.","title":"C4.5: Programs for Machine Learning","authors":"J, Ross Quinlan","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sharmin et al., 2019] Sadia Sharmin, Mohammad Shoyaib, Amin Ahsan Ali, Muhammad Asif Hossain Khan, and Ok- sam Chae. Simultaneous feature selection and discretiza- tion based on mutual information. Pattern Recognition, 91:162-174, 2019.","title":"Simultaneous feature selection and discretization based on mutual information","authors":"Sadia Sharmin, Mohammad Shoyaib, Ahsan Amin, Muhammad Ali, Asif Hossain, Oksam Khan, Chae","year":"2019","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shih, 1999] Y-S Shih. Families of splitting criteria for clas- sification trees. Statistics and Computing, 9(4):309-315, 1999.","title":"Families of splitting criteria for classification trees","authors":"Y-S Shih","year":"1999","venue":"Statistics and Computing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Su and Cao, 2019] Chong Su and Jie Cao. Improving lazy decision tree for imbalanced classification by using skew- insensitive criteria. Applied Intelligence, 49(3):1127- 1145, 2019.","title":"Improving lazy decision tree for imbalanced classification by using skewinsensitive criteria","authors":"Chong Su, Jie Cao","year":"2019","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Taylor and Silverman, 1993] Paul C Taylor and Bernard W Silverman. Block diagrams and splitting criteria for clas- sification trees. Statistics and Computing, 3(4):147-161, 1993.","title":"Block diagrams and splitting criteria for classification trees","authors":"C Paul, Bernard W Taylor, Silverman","year":"1993","venue":"Statistics and Computing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}