{"type":"ScholarlyArticle","title":"Probabilistic Equivalence Verification Approach for Automatic Mathematical Solution Assessment","authors":[{"name":"Minh Luan Nguyen","slug":"minh-luan-nguyen","given":"Minh Luan","family":"Nguyen","affiliation":""},{"name":"Siu Cheung Hui","slug":"siu-cheung-hui","given":"Siu Cheung","family":"Hui","affiliation":"School of Computer Engineering Nanyang Technological University Singapore"},{"name":"Alvis Cheuk M. Fong","slug":"alvis-cheuk-m-fong","given":"Alvis Cheuk M.","family":"Fong","affiliation":"School of Computing and Math Sciences Auckland University of Technology New Zealand"}],"abstract":"Automatic mathematical solution assessment checks the equivalence of mathematical expressions in the user answer and standard solution. It is a challenging problem as the semantics of mathematical expressions are highly symbolic and equivalent mathematical expressions can be expressed in different forms. In this paper, we propose an effective Probabilistic Equivalence Verification (PEV) approach for automatic mathematical solution assessment. The proposed PEV approach is a randomized method based on the probabilistic numerical equivalence testing of two mathematical expressions. It can avoid false negative errors completely while guaranteeing a small probability of false positive errors to occur. The performance results have shown that the proposed PEV approach has outperformed other popular techniques in Computer Algebra Systems such as Maple and Mathematica.","doi":null,"url":"https://ijcai.org/proceedings/2013/299","pdf":"https://ijcai.org/proceedings/2013/0299.pdf","markdown":"https://ijcai.org/proceedings/2013/299.md","bibtex":"https://ijcai.org/proceedings/2013/bibtex/299","volume":{"slug":"2013","title":"Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2013","url":"https://ijcai.org/proceedings/2013/","year":2013,"location":"Beijing, China","dates":"3–9 August 2013","editors":"Francesca Rossi","isbn":"978-1-57735-633-2","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":299,"pages":{"first":2028,"last":2033},"keywords":[],"videos":[],"references":[{"position":1,"raw":"[Agrawal et al., 2004] M. Agrawal, N. Kayal, and N. Sax- ena. Primes is in p. Annals of mathematics, 160(2):781- 793, 2004.","title":"Primes is in p","authors":"M Agrawal, N Kayal, N Saxena","year":"2004","venue":"Annals of mathematics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alon and Spencer, 2008] N. Alon and J. H. Spencer. The probabilistic method, volume 73. Wiley-Interscience, 2008.","title":"The probabilistic method","authors":"N Alon, J H Spencer","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Apostolico and Galil, 1997] A. Apostolico and Z. Galil. Pattern matching algorithms. Oxford University Press, USA, 1997.","title":"Pattern matching algorithms","authors":"A Apostolico, Z Galil","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Beizer, 2002] B. Beizer. Software testing techniques. Dreamtech Press, 2002.","title":"Software testing techniques","authors":"B Beizer","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Blum et al., 1980] M. Blum, A. K. Chandra, and M. N. Wegman. Equivalence of free boolean graphs can be de- cided probabilistically in polynomial time. Information Processing Letters, 10(2):80-82, 1980.","title":"Equivalence of free boolean graphs can be decided probabilistically in polynomial time","authors":"M Blum, A K Chandra, M N Wegman","year":"1980","venue":"Information Processing Letters","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Buchberger and Loos, 1982] B. Buchberger and R. Loos. Algebraic simplification. Computer Algebra-Symbolic and Algebraic Computation, pages 11-43, 1982.","title":"Algebraic simplification","authors":"B Buchberger, R Loos","year":"1982","venue":"Computer Algebra-Symbolic and Algebraic Computation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cohen, 2003] J.S. Cohen. Computer algebra and symbolic computation: Mathematical methods. Universities Press, 2003. [Coursera, 2013] Coursera. https://www.coursera.org/. 2013. [Edventure, 2013] Edventure. Edventure, https://edventure.ntu.edu.sg. 2013.","title":"Computer algebra and symbolic computation: Mathematical methods","authors":"J S Cohen","year":"2003","venue":null,"doi":null,"url":"https://edventure.ntu.edu.sg.2013","internal_url":null},{"position":8,"raw":"[Gulwani et al., 2011] S. Gulwani, V.A. Korthikanti, and A. Tiwari. Synthesizing geometry constructions. In ACM SIGPLAN Notices, volume 46, pages 50-61. ACM, 2011.","title":"Synthesizing geometry constructions","authors":"S Gulwani, V A Korthikanti, A Tiwari","year":"2011","venue":"ACM SIGPLAN Notices","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guzman and Conejo, 2007] E. Guzman and R. Conejo. Im- proving student performance using self-assessment tests. IEEE Intelligent Systems, 22(4):46-52, 2007.","title":"Improving student performance using self-assessment tests","authors":"E Guzman, R Conejo","year":"2007","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hardy and Wright, 1980] G. H. Hardy and E. M. Wright. An introduction to the theory of numbers. Oxford Uni- versity Press, USA, 1980.","title":"An introduction to the theory of numbers","authors":"G H Hardy, E M Wright","year":"1980","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"M. A. Hearst. The debate on automated essay grading. IEEE Intelligent Systems and their Applications, 15(5):22-37, 2000.","title":"The debate on automated essay grading","authors":"M A Hearst","year":"2000","venue":"IEEE Intelligent Systems and their Applications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hyman, 2012] Paul Hyman. In the year of disruptive edu- cation. Commun. ACM, 55(12):20-22, December 2012.","title":"In the year of disruptive education","authors":"Paul Hyman","year":"2012","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ibarra and Moran, 1983] O.H. Ibarra and S. Moran. Proba- bilistic algorithms for deciding equivalence of straight-line programs. Journal of the ACM (JACM), 30(1):217-228, 1983.","title":"Probabilistic algorithms for deciding equivalence of straight-line programs","authors":"O H Ibarra, S Moran","year":"1983","venue":"Journal of the ACM (JACM)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mapple, 2011] Mapple. Version 15, http://www.maplesoft.com/. 2011. [Mathematica, 2011] Mathematica. http://www.wolfram.com/mathematica/. 2011.","title":"Mapple. Version 15","authors":"Mapple","year":"2011","venue":null,"doi":null,"url":"http://www.wolfram.com/mathematica/.2011","internal_url":null},{"position":15,"raw":"[Mohler et al., 2011] M. Mohler, R. Bunescu, and R. Mihal- cea. Learning to grade short answer questions using se- mantic similarity measures and dependency graph align- ments. In Proceedings of Association for Computational Linguistics, pages 752-762, 2011.","title":"Learning to grade short answer questions using semantic similarity measures and dependency graph alignments","authors":"M Mohler, R Bunescu, R Mihalcea","year":"2011","venue":"Proceedings of Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Moses, 1971] J. Moses. Algebraic simplification a guide for the perplexed. In Proceedings of the second ACM sympo- sium on Symbolic and algebraic manipulation, pages 282- 304, 1971.","title":"Algebraic simplification a guide for the perplexed","authors":"J Moses","year":"1971","venue":"Proceedings of the second ACM symposium on Symbolic and algebraic manipulation","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Motwani and Raghavan, 1996] R. Motwani and P. Ragha- van. Randomized algorithms. ACM Computing Surveys (CSUR), 28(1):37, 1996.","title":"Randomized algorithms","authors":"R Motwani, P Raghavan","year":"1996","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ngo and Du, 2000] H. Q. Ngo and D. Z. Du. A survey on combinatorial group testing algorithms with applications to dna library screening. Discrete mathematical problems with medical applications, 55:171-182, 2000.","title":"A survey on combinatorial group testing algorithms with applications to dna library screening","authors":"H Q Ngo, D Z Du","year":"2000","venue":"Discrete mathematical problems with medical applications","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rabin, 1981] M. O. Rabin. Fingerprinting by random poly- nomials. Center for Research in Computing Techn., Aiken Computation Laboratory, University, 1981.","title":"Fingerprinting by random polynomials","authors":"M O Rabin","year":"1981","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rasila et al., 2007] A. Rasila, M. Harjula, and K. Zenger. Automatic assessment of mathematics exercises: Experi- ences and future prospects. ReflekTori Symposium of En- gineering Education, pages 70-80, 2007.","title":"Automatic assessment of mathematics exercises: Experiences and future prospects","authors":"A Rasila, M Harjula, K Zenger","year":"2007","venue":"ReflekTori Symposium of Engineering Education","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rasila et al., 2010] A. Rasila, L. Havola, H. Majander, and J. Malinen. Automatic assessment in engineering mathe- matics: evaluation of the impact. ReflekTori Symposium of Engineering Education, pages 37-45, 2010.","title":"Automatic assessment in engineering mathematics: evaluation of the impact","authors":"A Rasila, L Havola, H Majander, J Malinen","year":"2010","venue":"ReflekTori Symposium of Engineering Education","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shatnawi and Youssef, 2007] M. Shatnawi and A. Youssef. Equivalence detection using parse-tree normalization for math search. In 2nd International Conference on Digital Information Management, ICDIM, volume 2, pages 643- 648, 2007.","title":"Equivalence detection using parse-tree normalization for math search","authors":"M Shatnawi, A Youssef","year":"2007","venue":"2nd International Conference on Digital Information Management, ICDIM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sim, 2013] Simpy, http://sympy.org/en/index.html, 2013.","title":"Simpy","authors":"Sim","year":"2013","venue":null,"doi":null,"url":"http://sympy.org/en/index.html","internal_url":null},{"position":24,"raw":"[Singh et al., 2012] R. Singh, S. Gulwani, and S. Rajamani. Automatically generating algebra problems. AAAI, 2012.","title":"Automatically generating algebra problems","authors":"R Singh, S Gulwani, S Rajamani","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Von Zur Gathen and Gerhard, 2003] J. Von Zur Gathen and J. Gerhard. Modern computer algebra. Cambridge Uni- versity Press, 2003.","title":"Modern computer algebra","authors":"Von Zur, Gathen, Gerhard ;, J Von Zur Gathen, J Gerhard","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yap, 2000] C. K. Yap. Fundamental problems of algorith- mic algebra, volume 54. Oxford University Press Oxford, 2000.","title":"Fundamental problems of algorithmic algebra","authors":"C K Yap","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Youssef and Shatnawi, 2006] A. Youssef and M. Shatnawi. Math search with equivalence detection using parse-tree normalization. In The 4th International Conference on Computer Science and Information Technology, 2006.","title":"Math search with equivalence detection using parse-tree normalization","authors":"A Youssef, M Shatnawi","year":"2006","venue":"The 4th International Conference on Computer Science and Information Technology","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Shasha, 1989] K. Zhang and D. Shasha. Sim- ple fast algorithms for the editing distance between trees and related problems. SIAM journal on computing, 18(6):1245-1262, 1989.","title":"Simple fast algorithms for the editing distance between trees and related problems","authors":"K Zhang, D Shasha","year":"1989","venue":"SIAM journal on computing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}