{"type":"ScholarlyArticle","title":"Learning to Assemble Geometric Shapes","authors":[{"name":"Jinhwi Lee","slug":"jinhwi-lee","given":"Jinhwi","family":"Lee","affiliation":"Pohang University of Science and Technology (POSTECH ); POSCO"},{"name":"Jungtaek Kim","slug":"jungtaek-kim","given":"Jungtaek","family":"Kim","affiliation":"Pohang University of Science and Technology (POSTECH )"},{"name":"Hyunsoo Chung","slug":"hyunsoo-chung","given":"Hyunsoo","family":"Chung","affiliation":"Pohang University of Science and Technology (POSTECH )"},{"name":"Jaesik Park","slug":"jaesik-park","given":"Jaesik","family":"Park","affiliation":"Pohang University of Science and Technology (POSTECH )"},{"name":"Minsu Cho","slug":"minsu-cho","given":"Minsu","family":"Cho","affiliation":"Pohang University of Science and Technology (POSTECH )"}],"abstract":"Assembling parts into an object is a combinatorial problem that arises in a variety of contexts in the real world and involves numerous applications in science and engineering. Previous related work tackles limited cases with identical unit parts or jigsaw-style parts of textured shapes, which greatly mitigate combinatorial challenges of the problem. In this work, we introduce the more challenging problem of shape assembly, which involves textureless fragments of arbitrary shapes with indistinctive junctions, and then propose a learning-based approach to solving it. We demonstrate the effectiveness on shape assembly tasks with various scenarios, including the ones with abnormal fragments (e.g., missing and distorted), the different number of fragments, and different rotation discretization.","doi":"10.24963/ijcai.2022/146","url":"https://ijcai.org/proceedings/2022/146","pdf":"https://ijcai.org/proceedings/2022/0146.pdf","markdown":"https://ijcai.org/proceedings/2022/146.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/146","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":146,"pages":{"first":1046,"last":1052},"keywords":["Computer Vision: Applications","Computer Vision: Machine Learning for Vision","Machine Learning: Applications","Machine Learning: Attention Models"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986885","length_sec":105,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985028","length_sec":356,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bapst et al., 2019] Victor Bapst, Alvaro Sanchez-Gonzalez, Carl Doersch, Kimberly Lauren Stachenfeld, Pushmeet Kohli, Peter W. Battaglia, and Jessica B. Hamrick. Struc- tured agents for physical construction. In Proceedings of the International Conference on Machine Learning (ICML), pages 464-474, Long Beach, California, USA, 2019.","title":"Structured agents for physical construction","authors":"Victor Bapst, Alvaro Sanchez-Gonzalez, Carl Doersch, Kimberly, Lauren Stachenfeld, Pushmeet Kohli, Peter W Battaglia, Jessica B Hamrick","year":"2019","venue":"Proceedings of the International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boussaïd et al., 2013] Ilhem Boussaïd, Julien Lepagnot, and Patrick Siarry. A survey on optimization metaheuris- tics. Information Sciences, 237:82-117, 2013.","title":"Ilhem Boussaïd, Julien Lepagnot, and Patrick Siarry. A survey on optimization metaheuristics","authors":"Boussaïd","year":"2013","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brochu et al., 2010] Eric Brochu, Vlad M. Cora, and Nando de Freitas. A tutorial on Bayesian optimization of expen- sive cost functions, with application to active user model- ing and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010.","title":"A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning","authors":"Eric Brochu, Vlad M Cora, Nando De Freitas","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown, 1971] Arthur Robert Brown. Optimum packing and depletion. Macdonald and Co.; New York, American El- sevier, 1971.","title":"Optimum packing and depletion","authors":"Arthur Robert Brown","year":"1971","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cho et al., 2010] Taeg Sang Cho, Shai Avidan, and William T. Freeman. A probabilistic image jigsaw puz- zle solver. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pages 183-190, San Francisco, California, USA, 2010.","title":"A probabilistic image jigsaw puzzle solver","authors":"Cho","year":"2010","venue":"Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chung et al., 2021] Hyunsoo Chung, Jungtaek Kim, Boris Knyazev, Jinhwi Lee, Graham W. Taylor, Jaesik Park, and Minsu Cho. Brick-by-Brick: Combinatorial construction with deep reinforcement learning. In Advances in Neu- ral Information Processing Systems (NeurIPS), volume 34, pages 5745-5757, Virtual, 2021.","title":"Brick-by-Brick: Combinatorial construction with deep reinforcement learning","authors":"Hyunsoo Chung, Jungtaek Kim, Boris Knyazev, Jinhwi Lee, Graham W Taylor, Jaesik Park, Minsu Cho","year":"2021","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Coffman et al., 1996] Edward Grady Coffman, Michael Randolph Garey, and David Stifler Johnson. Approximation algorithms for bin packing: A survey. Approximation Algorithms for NP-hard Problems, pages 46-93, 1996.","title":"Approximation algorithms for bin packing: A survey","authors":"Edward Grady Coffman, Michael Randolph Garey, David Stifler, Johnson","year":"1996","venue":"Approximation Algorithms for NP-hard Problems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Derech et al., 2018] Niv Derech, Ayellet Tal, and Ilan Shimshoni. Solving archaeological puzzles. arXiv preprint arXiv:1812.10553, 2018.","title":"Niv Derech, Ayellet Tal, and Ilan Shimshoni","authors":"Derech","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goyal and Deng, 2020] Ankit Goyal and Jia Deng. PackIt: A virtual environment for geometric planning. In Proceed- ings of the International Conference on Machine Learning (ICML), pages 3700-3710, Virtual, 2020.","title":"PackIt: A virtual environment for geometric planning","authors":"Ankit Goyal, Jia Deng","year":"2020","venue":"Proceedings of the International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2020] Jungtaek Kim, Hyunsoo Chung, Jinhwi Lee, Minsu Cho, and Jaesik Park. Combinatorial 3D shape generation via sequential assembly. In Neural Infor- mation Processing Systems Workshop on Machine Learn- ing for Engineering Modeling, Simulation, and Design (ML4Eng), Virtual, 2020.","title":"Combinatorial 3D shape generation via sequential assembly","authors":"Kim","year":"2020","venue":"Neural Information Processing Systems Workshop on Machine Learning for Engineering Modeling, Simulation, and Design (ML4Eng)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2020] Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, and Tingfa Xu. LayoutGAN: Synthe- sizing graphic layouts with vector-wireframe adversarial networks. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2020. [","title":"LayoutGAN: Synthesizing graphic layouts with vector-wireframe adversarial networks","authors":"Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, Tingfa Xu","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lodi et al., 2002] Andrea Lodi, Silvano Martello, and Michele Monaci. Two-dimensional packing problems: A survey. European Journal of Operational Research, 141(2):241-252, 2002.","title":"Two-dimensional packing problems: A survey","authors":"Andrea Lodi, Silvano Martello, Michele Monaci","year":"2002","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Noroozi and Favaro, 2016] Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In Proceedings of the European Conference on Computer Vision (ECCV), pages 69-84, Amsterdam, The Netherlands, 2016.","title":"Unsupervised learning of visual representations by solving jigsaw puzzles","authors":"Favaro Noroozi, Mehdi Noroozi, Paolo Favaro","year":"2016","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pincus, 1970] Martin Pincus. Letter to the editor -a Monte Carlo method for the approximate solution of certain types of constrained optimization problems. Operations Re- search, 18(6):1225-1228, 1970.","title":"Letter to the editor -a Monte Carlo method for the approximate solution of certain types of constrained optimization problems","authors":"Martin Pincus","year":"1970","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Roy and Teh, 2008] Daniel M. Roy and Yee Whye Teh. The Mondrian process. In Advances in Neural Information Processing Systems (NeurIPS), volume 21, pages 1377- 1384, Vancouver, British Columbia, Canada, 2008.","title":"The Mondrian process","authors":"M Daniel, Yee Whye Roy, Teh","year":"2008","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sanches and Soma, 2009] Carlos Alberto Alonso Sanches and Nei Yoshihiro Soma. A polynomial-time DNA com- puting solution for the bin-packing problem. Applied Mathematics and Computation, 215(6):2055-2062, 2009.","title":"A polynomial-time DNA computing solution for the bin-packing problem","authors":"Carlos Alberto, Alonso Sanches, Nei Yoshihiro, Soma","year":"2009","venue":"Applied Mathematics and Computation","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Schumacher et al., 1969] Robert Schumacher, Brigitta Brand, Maurice Gilliland, and Werner Sharp. Study for applying computer-generated images to visual simulation. Technical Report AFHRL-TR-69-14, Air Force Human Resources Laboratory, Air Force Systems Command, 1969.","title":"Study for applying computer-generated images to visual simulation","authors":"Robert Schumacher, Brigitta Brand, Maurice Gilliland, Werner Sharp","year":"1969","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Slocum, 2003] Jerry Slocum. Tangram: The World's First Puzzle Craze. Sterling, 2003.","title":"Jerry Slocum. Tangram: The World's First Puzzle Craze","authors":"Slocum","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Torsvik, 2003] Trond H. Torsvik. The Rodinia jigsaw puz- zle. Science, 300(5624):1379-1381, 2003.","title":"The Rodinia jigsaw puzzle","authors":"H Trond, Torsvik","year":"2003","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pages 5998-6008, Long Beach, California, USA, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zakka et al., 2020] Kevin Zakka, Andy Zeng, Johnny Lee, and Shuran Song. Form2Fit: Learning shape priors for generalizable assembly from disassembly. In Proceedings of the International Conference on Robotics and Automa- tion (ICRA), pages 9404-9410, Virtual, 2020.","title":"Form2Fit: Learning shape priors for generalizable assembly from disassembly","authors":"Kevin Zakka, Andy Zeng, Johnny Lee, Shuran Song","year":"2020","venue":"Proceedings of the International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}