{"type":"ScholarlyArticle","title":"Direction-Optimizing Breadth-First Search with External Memory Storage","authors":[{"name":"Shuli Hu","slug":"shuli-hu","given":"Shuli","family":"Hu","affiliation":"Northeast Normal University , Changchun , Jilin , China"},{"name":"Nathan R. Sturtevant","slug":"nathan-r-sturtevant","given":"Nathan R.","family":"Sturtevant","affiliation":"University of Alberta , Edmonton , AB , Canada"}],"abstract":"While computing resources have continued to grow, methods for building and using large heuristics have not seen significant advances in recent years. We have observed that direction-optimizing breadth-first search, developed for and used broadly in the Graph 500 competition, can also be applied for building heuristics. But, the algorithm cannot run efficiently using external memory -- when the heuristics being built are larger than RAM. This paper shows how to modify direction-optimizing breadth-first search to build external-memory heuristics. We show that the new approach is not effective in state spaces with low asymptotic branching factors, but in other domains we are able to achieve up to a 3x reducing in runtime when building an external-memory heuristic. The approach is then used to build a 2.6TiB Rubik's Cube heuristic with 5.8 trillion entries, the largest pattern database heuristic ever built.","doi":"10.24963/ijcai.2019/175","url":"https://ijcai.org/proceedings/2019/175","pdf":"https://ijcai.org/proceedings/2019/0175.pdf","markdown":"https://ijcai.org/proceedings/2019/175.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/175","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Heuristic Search and Game Playing","track":"Main track","number":175,"pages":{"first":1258,"last":1264},"keywords":["Heuristic Search and Game Playing: Combinatorial Search and Optimisation","Heuristic Search and Game Playing: Heuristic Search"],"videos":[],"references":[{"position":1,"raw":"[Beamer et al., 2012] Scott Beamer, Krste Asanović, and David Patterson. Direction-optimizing breadth-first search. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 12:1-12:10, Los Alamitos, CA, USA, 2012. IEEE Computer Society Press.","title":"Direction-optimizing breadth-first search","authors":"Scott Beamer, Krste Asanović, David Patterson","year":"2012","venue":"Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Buluc ¸et al., 2017] Aydin Buluc ¸, Scott Beamer, Kamesh Madduri, Krste Asanovic, and David A. Patterson. Distributed-memory breadth-first search on massive graphs. CoRR, abs/1705.04590, 2017.","title":"Distributed-memory breadth-first search on massive graphs","authors":"Aydin Buluc, ¸, Scott Beamer, Kamesh Madduri, Krste Asanovic, David A Patterson","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Culberson and Schaeffer, 1998] Joseph C. Culberson and Jonathan Schaeffer. Pattern databases. Computational In- telligence, 14(3):318-334, 1998.","title":"Pattern databases","authors":"Joseph C Culberson, Jonathan Schaeffer","year":"1998","venue":"Computational Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Döbbelin et al., 2013] Robert Döbbelin, Thorsten Schütt, and Alexander Reinefeld. Building large compressed pdbs for the sliding tile puzzle. In IJCAI Workshop on Computer Games, pages 16-27, 2013.","title":"Building large compressed pdbs for the sliding tile puzzle","authors":"Robert Döbbelin, Thorsten Schütt, Alexander Reinefeld","year":"2013","venue":"IJCAI Workshop on Computer Games","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Felner et al., 2007] Ariel Felner, Richard E. Korf, Ram Meshulam, and Robert Holte. Compressed pattern databases. Journal of Artificial Intelligence Research, 30:213-247, 2007.","title":"Compressed pattern databases","authors":"Ariel Felner, Richard E Korf, Ram Meshulam, Robert Holte","year":"2007","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hatem et al., 2011] Matthew Hatem, Ethan Burns, and Wheeler Ruml. Heuristic search for large problems with real costs. In AAAI Conference on Artificial Intelligence, pages 30-35, 2011.","title":"Heuristic search for large problems with real costs","authors":"Matthew Hatem, Ethan Burns, Wheeler Ruml","year":"2011","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Holte et al., 2004] Robert C. Holte, Jack Newton, Ariel Fel- ner, Ram Meshulam, and David Furcy. Multiple pat- tern databases. In International Conference on Automated Planning and Scheduling (ICAPS), pages 122-131, 2004.","title":"Multiple pattern databases","authors":"Robert C Holte, Jack Newton, Ariel Felner, Ram Meshulam, David Furcy","year":"2004","venue":"International Conference on Automated Planning and Scheduling (ICAPS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Korf and Felner, 2007] Richard E. Korf and Ariel Felner. Recent progress in heuristic search: A case study of the four-peg Towers of Hanoi problem. In International Joint Conference on Artificial Intelligence (IJCAI), pages 2324- 2329, 2007.","title":"Recent progress in heuristic search: A case study of the four-peg Towers of Hanoi problem","authors":"Richard E Korf, Ariel Felner","year":"2007","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2007/374"},{"position":9,"raw":"[Korf and Shultze, 2005] Richard E. Korf and Peter Shultze. Large-scale, parallel breadth-first search. In National Con- ference on Artificial Intelligence, pages 1380-1385, 2005.","title":"Large-scale, parallel breadth-first search","authors":"Richard E Korf, Peter Shultze","year":"2005","venue":"National Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Korf et al., 2001] Richard E. Korf, Michael Reid, and Ste- fan Edelkamp. Time complexity of iterative-deepening- A*. Artificial Intelligence, 129(1-2):199-218, 2001.","title":"Time complexity of iterative-deepening-A*","authors":"Richard E Korf, Michael Reid, Stefan Edelkamp","year":"2001","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Korf, 2004] Richard E. Korf. Best-first frontier search with delayed duplicate detection. In National Conference on Artificial Intelligence (AAAI), pages 650-657, 2004.","title":"Best-first frontier search with delayed duplicate detection","authors":"Richard E Korf","year":"2004","venue":"National Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Korf, 2008] Richard E. Korf. Minimizing disk I/O in two- bit breadth-first search. In AAAI Conference on Artificial Intelligence, pages 317-324, 2008.","title":"Minimizing disk I/O in twobit breadth-first search","authors":"Richard E Korf","year":"2008","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Myrvold and Ruskey, 2001] W. Myrvold and F. Ruskey. Ranking and unranking permutations in linear time. In- formation Processing Letters, 79:281-284, 2001.","title":"Ranking and unranking permutations in linear time","authors":"W Myrvold, F Ruskey","year":"2001","venue":"Information Processing Letters","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Roschke and Sturtevant, 2014] Max Roschke and Nathan R. Sturtevant. UCT enhancements in chinese checkers using an endgame database. In Tristan Cazenave, Mark H.M. Winands, and Hiroyuki Iida, editors, Computer Games, pages 57-70, 2014. [Schadd and Winands, 2011]","title":"UCT enhancements in chinese checkers using an endgame database","authors":"Sturtevant Roschke, Max Roschke, Nathan R Sturtevant","year":"2014","venue":"Computer Games","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Maarten P. D. Schadd and Mark H. M. Winands. Best reply search for multiplayer games. IEEE Transactions on Computational Intelligence and AI in Games, 3(1):57-66, 2011.","title":"Best reply search for multiplayer games","authors":"P D Maarten, Schadd, H M Mark, Winands","year":"2011","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sturtevant and Rutherford, 2013] Nathan R. Sturtevant and Matthew J. Rutherford. Minimizing writes in parallel ex- ternal memory search. International Joint Conference on Artificial Intelligence (IJCAI), pages 666-673, 2013.","title":"Minimizing writes in parallel external memory search","authors":"Nathan R Sturtevant, Matthew J Rutherford","year":"2013","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2013/105"},{"position":17,"raw":"[Sturtevant, 2002] Nathan R. Sturtevant. A comparison of algorithms for multi-player games. In Computers and Games, pages 108-122, 2002.","title":"A comparison of algorithms for multi-player games","authors":"Nathan R Sturtevant","year":"2002","venue":"Computers and Games","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ueno et al., 2016] Koji Ueno, Toyotaro Suzumura, Naoya Maruyama, Katsuki Fujisawa, and Satoshi Matsuoka. Ex- treme scale breadth-first search on supercomputers. In 2016 IEEE International Conference on Big Data (Big Data), pages 1040-1047, Dec 2016.","title":"Extreme scale breadth-first search on supercomputers","authors":"Koji Ueno, Toyotaro Suzumura, Naoya Maruyama, Katsuki Fujisawa, Satoshi Matsuoka","year":"2016","venue":"2016 IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhou and Hansen, 2004] Rong Zhou and Eric A. Hansen. Structured duplicate detection in external-memory graph search. In National Conference on Artificial Intelligence (AAAI), pages 683-689, 2004.","title":"Structured duplicate detection in external-memory graph search","authors":"Rong Zhou, Eric A Hansen","year":"2004","venue":"National Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhou and Hansen, 2007] Rong Zhou and Eric A. Hansen. Edge partitioning in external-memory graph search. In In- ternational Joint Conference on Artifical Intelligence (IJ- CAI), pages 2410-2416, 2007.","title":"Edge partitioning in external-memory graph search","authors":"Rong Zhou, Eric A Hansen","year":"2007","venue":"ternational Joint Conference on Artifical Intelligence (IJ-CAI)","doi":null,"url":null,"internal_url":"/proceedings/2007/388"},{"position":21,"raw":"[Zhou and Hansen, 2011] Rong Zhou and Eric A. Hansen. Dynamic state-space partitioning in external-memory graph search. In International Conference on Automated Planning and Scheduling (ICAPS), pages 290-297, 2011.","title":"Dynamic state-space partitioning in external-memory graph search","authors":"Rong Zhou, Eric A Hansen","year":"2011","venue":"International Conference on Automated Planning and Scheduling (ICAPS)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}