{"type":"ScholarlyArticle","title":"Supervised Deep Features for Software Functional Clone Detection by Exploiting Lexical and Syntactical Information in Source Code","authors":[{"name":"Huihui Wei","slug":"huihui-wei","given":"Huihui","family":"Wei","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing 210023 , China"},{"name":"Ming Li","slug":"ming-li","given":"Ming","family":"Li","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing 210023 , China"}],"abstract":"Software clone detection, aiming at identifying out code fragments with similar functionalities, has played an important role in software maintenance and evolution. Many clone detection approaches have been proposed. However, most of them represent source codes with hand-crafted features using lexical or syntactical information, or unsupervised deep features, which makes it difficult to detect the functional clone pairs, i.e., pieces of codes with similar functionality but differing in both syntactical and lexical level. In this paper, we address the software functional clone detection problem by learning supervised deep features. We formulate the clone detection as a supervised learning to hash problem and propose an end-to-end deep feature learning framework called CDLH for functional clone detection. Such framework learns hash codes  by exploiting the lexical and syntactical information for fast computation of functional similarity between code fragments. Experiments on software clone detection benchmarks indicate that the CDLH approach is effective and outperforms the state-of-the-art approaches in software functional clone detection.","doi":"10.24963/ijcai.2017/423","url":"https://ijcai.org/proceedings/2017/423","pdf":"https://ijcai.org/proceedings/2017/0423.pdf","markdown":"https://ijcai.org/proceedings/2017/423.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/423","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":423,"pages":{"first":3034,"last":3040},"keywords":["Multidisciplinary Topics and Applications: Knowledge-based Software Engineering","Machine Learning: Data Mining","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Baker, 1995] Brenda S. Baker. On finding duplication and near-duplication in large software systems. In Proceedings of the 2nd Working Conference on Reverse Engineering, pages 86-95, Toronto, Canada, 1995.","title":"On finding duplication and near-duplication in large software systems","authors":"Brenda S Baker","year":"1995","venue":"Proceedings of the 2nd Working Conference on Reverse Engineering","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baxter et al., 1998] Ira D. Baxter, Andrew Yahin, Leonar- do Mendonc ¸a de Moura, Marcelo Sant'Anna, and Lorraine Bier. Clone detection using abstract syntax trees. In Pro- ceedings of the 1998 International Conference on Software Maintenance, pages 368-377, Bethesda, Maryland, USA, 1998.","title":"Clone detection using abstract syntax trees","authors":"Ira D Baxter, Andrew Yahin, Leonardo Mendonc ¸a De Moura, Marcelo Sant'anna, Lorraine Bier","year":"1998","venue":"Proceedings of the 1998 International Conference on Software Maintenance","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brixtel et al., 2010] Romain Brixtel, Mathieu Fontaine, Boris Lesner, Cyril Bazin, and Romain Robbes. Language-independent clone detection applied to plagia- rism detection. In Proceedings of the 10th IEEE Interna- tional Working Conference on Source Code Analysis and Manipulation, pages 77-86, Timisoara, Romania, 2010.","title":"Language-independent clone detection applied to plagiarism detection","authors":"Romain Brixtel, Mathieu Fontaine, Boris Lesner, Cyril Bazin, Romain Robbes","year":"2010","venue":"Proceedings of the 10th IEEE International Working Conference on Source Code Analysis and Manipulation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Huo et al., 2016] Xuan Huo, Ming Li, and Zhi-Hua Zhou. Learning unified features from natural and programming languages for locating buggy source code. In Proceedings of the 25th International Joint Conference on Artificial In- telligence, pages 1606-1612, New York, NY, USA, 2016.","title":"Learning unified features from natural and programming languages for locating buggy source code","authors":"Xuan Huo, Ming Li, Zhi-Hua Zhou","year":"2016","venue":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/230"},{"position":5,"raw":"[Jiang et al., 2007] Lingxiao Jiang, Ghassan Misherghi, Zhendong Su, and Stéphane Glondu. DECKARD: scal- able and accurate tree-based detection of code clones. In Proceedings of the 29th International Conference on Soft- ware Engineering, pages 96-105, Minneapolis, MN, USA, 2007.","title":"DECKARD: scalable and accurate tree-based detection of code clones","authors":"Jiang","year":"2007","venue":"Proceedings of the 29th International Conference on Software Engineering","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kamiya et al., 2002] Toshihiro Kamiya, Shinji Kusumoto, and Katsuro Inoue. Ccfinder: A multilinguistic token- based code clone detection system for large scale source code. IEEE Transactions on Software Engineering, 28(7):654-670, 2002.","title":"Ccfinder: A multilinguistic tokenbased code clone detection system for large scale source code","authors":"Toshihiro Kamiya, Shinji Kusumoto, Katsuro Inoue","year":"2002","venue":"IEEE Transactions on Software Engineering","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kong and Li, 2012] Weihao Kong and Wu-Jun Li. Isotropic hashing. In Proceedings of the 26th Annual Conference on Neural Information Processing Systems, pages 1655- 1663, Lake Tahoe, Nevada, USA, 2012.","title":"Weihao Kong and Wu-Jun Li. Isotropic hashing","authors":"Li Kong","year":"2012","venue":"Proceedings of the 26th Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lin et al., 2014] Guosheng Lin, Chunhua Shen, Qinfeng Shi, Anton van den Hengel, and David Suter. Fast super- vised hashing with decision trees for high-dimensional da- ta. In Proceedings of the 2014 IEEE Conference on Com- puter Vision and Pattern Recognition, pages 1971-1978, Columbus, OH, USA, 2014.","title":"Fast supervised hashing with decision trees for high-dimensional data","authors":"Guosheng Lin, Chunhua Shen, Qinfeng Shi, Anton Van Den, David Hengel, Suter","year":"2014","venue":"Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mou et al., 2016] Lili Mou, Ge Li, Lu Zhang, Tao Wang, and Zhi Jin. Convolutional neural networks over tree struc- tures for programming language processing. In Proceed- ings of the 30th AAAI Conference on Artificial Intelligence, pages 1287-1293, Phoenix, Arizona, USA, 2016.","title":"Convolutional neural networks over tree structures for programming language processing","authors":"Lili Mou, Ge Li, Lu Zhang, Tao Wang, Zhi Jin","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Roy and Cordy, 2007] Chanchal Kumar Roy and James R Cordy. A survey on software clone detection research. Queens School of Computing TR, 541(115):64-68, 2007.","title":"A survey on software clone detection research","authors":"Chanchal Kumar, Roy, James R Cordy","year":"2007","venue":"Queens School of Computing TR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Roy and Cordy, 2008] Chanchal Kumar Roy and James R. Cordy. NICAD: accurate detection of near-miss intention- al clones using flexible pretty-printing and code normal- ization. In Proceedings of the 16th IEEE International Conference on Program Comprehension, pages 172-181, Amsterdam, The Netherlands, 2008.","title":"NICAD: accurate detection of near-miss intentional clones using flexible pretty-printing and code normalization","authors":"Chanchal Kumar, Roy, James R Cordy","year":"2008","venue":"Proceedings of the 16th IEEE International Conference on Program Comprehension","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Sajnani et al., 2016] Hitesh Sajnani, Vaibhav Saini, Jeffrey Svajlenko, Chanchal K. Roy, and Cristina V. Lopes. Sourcerercc: scaling code clone detection to big-code. In Proceedings of the 38th International Conference on Soft- ware Engineering, pages 1157-1168, Austin, TX, USA, 2016.","title":"Sourcerercc: scaling code clone detection to big-code","authors":"Hitesh Sajnani, Vaibhav Saini, Jeffrey Svajlenko, Chanchal K Roy, Cristina V Lopes","year":"2016","venue":"Proceedings of the 38th International Conference on Software Engineering","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Socher et al., 2011] Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng, and Christopher D. Man- ning. Semi-supervised recursive autoencoders for predict- ing sentiment distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 151-161, Edinburgh, UK, 2011.","title":"Manning. Semi-supervised recursive autoencoders for predicting sentiment distributions","authors":"Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, Christopher D","year":"2011","venue":"Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Svajlenko et al., 2014] Jeffrey Svajlenko, Judith F. Islam, Iman Keivanloo, Chanchal Kumar Roy, and Moham- mad Mamun Mia. Towards a big data curated bench- mark of inter-project code clones. In Proceedings of the 30th IEEE International Conference on Software Mainte- nance and Evolution, pages 476-480, Victoria, BC, Cana- da, 2014.","title":"Towards a big data curated benchmark of inter-project code clones","authors":"Jeffrey Svajlenko, Judith F Islam, Iman Keivanloo, Chanchal Kumar Roy, Mohammad Mamun, Mia","year":"2014","venue":"Proceedings of the 30th IEEE International Conference on Software Maintenance and Evolution","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Christopher D. Manning. Improved semantic represen- tations from tree-structured long short-term memory net- works. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th In- ternational Joint Conference on Natural Language Pro- cessing of the Asian Federation of Natural Language Pro- cessing, pages 1556-1566, Beijing, China, 2015.","title":"Improved semantic representations from tree-structured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Christopher D Manning","year":"2015","venue":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[White et al., 2016] Martin White, Michele Tufano, Christo- pher Vendome, and Denys Poshyvanyk. Deep learning code fragments for code clone detection. In Proceedings of the 31st IEEE/ACM International Conference on Automat- ed Software Engineering, pages 87-98, Singapore, 2016.","title":"Deep learning code fragments for code clone detection","authors":"Martin White, Michele Tufano, Christopher Vendome, Denys Poshyvanyk","year":"2016","venue":"Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Zaremba and Sutskever, 2014] Wojciech Zaremba and Ilya Sutskever. Learning to execute. CoRR, abs/1410.4615, 2014.","title":"Learning to execute","authors":"Wojciech Zaremba, Ilya Sutskever","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zeiler, 2012] Matthew D. Zeiler. ADADELTA: an adaptive learning rate method. CoRR, abs/1212.5701, 2012.","title":"ADADELTA: an adaptive learning rate method","authors":"D Matthew, Zeiler","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}