{"type":"ScholarlyArticle","title":"Deep Interactive Surface Creation from 3D Sketch Strokes","authors":[{"name":"Sukanya Bhattacharjee","slug":"sukanya-bhattacharjee","given":"Sukanya","family":"Bhattacharjee","affiliation":"Department of Computer Science and Engineering , IIT Bombay"},{"name":"Parag Chaudhuri","slug":"parag-chaudhuri","given":"Parag","family":"Chaudhuri","affiliation":"Department of Computer Science and Engineering , IIT Bombay"}],"abstract":"We present a deep neural framework that allows users to create surfaces from a stream of sparse 3D sketch strokes. Our network consists of a global surface estimation module followed by a local surface refinement. This facilitates in the incremental prediction of surfaces. Thus, our proposed method works with 3D sketch strokes and estimate a surface interactively in real time. We compare the proposed method with various state-of-the-art methods and show its efficacy for surface fitting. Further, we integrate our method into an existing Blender based 3D content creation pipeline to show its usefulness in 3D modelling.","doi":"10.24963/ijcai.2022/680","url":"https://ijcai.org/proceedings/2022/680","pdf":"https://ijcai.org/proceedings/2022/0680.pdf","markdown":"https://ijcai.org/proceedings/2022/680.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/680","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Research Papers","track":"AI and Arts","number":680,"pages":{"first":4908,"last":4914},"keywords":["Theory and philosophy of arts and creativity in AI systems: Support of human creativity","Application domains: Other domains of art or creativity","Theory and philosophy of arts and creativity in AI systems: Computational paradigms, architectures and models for creativity"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987165","length_sec":73,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985308","length_sec":714,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Barron, 2019] Jonathan T Barron. A general and adaptive robust loss function. In Proc. of CVPR, pages 4331-4339, 2019.","title":"A general and adaptive robust loss function","authors":"Jonathan T Barron","year":"2019","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barrow et al., 1977] Harry G Barrow, Jay M Tenenbaum, Robert C Bolles, and Helen C Wolf. Parametric corre- spondence and chamfer matching: Two new techniques for image matching. Technical report, Sri International AI Center, 1977.","title":"Parametric correspondence and chamfer matching: Two new techniques for image matching","authors":"Jay M Harry G Barrow, Robert C Tenenbaum, Helen C Bolles, Wolf","year":"1977","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/1977/158"},{"position":3,"raw":"[Bendels et al., 2006] Gerhard H Bendels, Michael Guthe, and Reinhard Klein. Free-form modelling for surface in- painting. In Proc. of the 4th International Conference on Computer graphics, Virtual Reality, Visualisation and In- teraction in Africa, pages 49-58, 2006.","title":"Free-form modelling for surface inpainting","authors":"Gerhard H Bendels, Michael Guthe, Reinhard Klein","year":"2006","venue":"Proc. of the 4th International Conference on Computer graphics, Virtual Reality, Visualisation and Interaction in Africa","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bhattacharjee and Chaudhuri, 2020] Sukanya Bhattachar- jee and Parag Chaudhuri. A survey on sketch based content creation: from the desktop to virtual and aug- mented reality. In Computer Graphics Forum, volume 39, pages 757-780. Wiley Online Library, 2020.","title":"Sukanya Bhattacharjee and Parag Chaudhuri. A survey on sketch based content creation: from the desktop to virtual and augmented reality","authors":"Chaudhuri Bhattacharjee","year":"2020","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bingol and Krishnamurthy, 2019] Onur Rauf Bingol and Adarsh Krishnamurthy. NURBS-Python: An open-source object-oriented NURBS modeling framework in Python. SoftwareX, 9:85-94, 2019.","title":"NURBS-Python: An open-source object-oriented NURBS modeling framework in Python","authors":"Krishnamurthy ; Bingol, Rauf Onur, Adarsh Bingol, Krishnamurthy","year":"2019","venue":"SoftwareX","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Coons, 1967] Steven A Coons. Surfaces for computer-aided design of space forms. Technical report, MIT Cambridge Project MAC, 1967.","title":"Surfaces for computer-aided design of space forms","authors":"A Steven, Coons","year":"1967","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[De Paoli and Singh, 2015] Chris De Paoli and Karan Singh. Secondskin: sketch-based construction of layered 3d mod- els. ACM Trans. on Graphics, 34(4):126, 2015.","title":"Secondskin: sketch-based construction of layered 3d models","authors":"De Paoli, Chris De Singh, Karan Paoli, Singh","year":"2015","venue":"ACM Trans. on Graphics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2020] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learn- ing for 3d point clouds: A survey. IEEE TPAMI, 2020.","title":"Deep learning for 3d point clouds: A survey","authors":"Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, Mohammed Bennamoun","year":"2020","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Harary et al., 2014] Gur Harary, Ayellet Tal, and Eitan Grinspun. Context-based coherent surface completion. ACM Trans. on Graphics, 33(1):1-12, 2014.","title":"Context-based coherent surface completion","authors":"Gur Harary, Ayellet Tal, Eitan Grinspun","year":"2014","venue":"ACM Trans. on Graphics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proc. of CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Igarashi et al., 1999] Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. Teddy: a sketching interface for 3d freeform design. In SIGGRAPH, pages 409-416. ACM, 1999.","title":"Teddy: a sketching interface for 3d freeform design","authors":"Takeo Igarashi, Satoshi Matsuoka, Hidehiko Tanaka","year":"1999","venue":"SIGGRAPH","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Koch et al., 2019] Sebastian Koch, Albert Matveev, Zhong- shi Jiang, Francis Williams, Alexey Artemov, Evgeny Bur- naev, Marc Alexa, Denis Zorin, and Daniele Panozzo. Abc: A big cad model dataset for geometric deep learn- ing. In Proc. of CVPR, pages 9601-9611, 2019.","title":"Abc: A big cad model dataset for geometric deep learning","authors":"Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, Daniele Panozzo","year":"2019","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lui et al., 2012] Lok Ming Lui, Chengfeng Wen, and Xian- feng Gu. A conformal approach for surface inpainting. arXiv preprint arXiv:1212.0981, 2012.","title":"A conformal approach for surface inpainting","authors":"Ming Lok, Chengfeng Lui, Xianfeng Wen, Gu","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mehr et al., 2019] Eloi Mehr, Ariane Jourdan, Nicolas Thome, Matthieu Cord, and Vincent Guitteny. Disconet: Shapes learning on disconnected manifolds for 3d editing. In Proc. of CVPR, pages 3474-3483, 2019.","title":"Disconet: Shapes learning on disconnected manifolds for 3d editing","authors":"Eloi Mehr, Ariane Jourdan, Nicolas Thome, Matthieu Cord, Vincent Guitteny","year":"2019","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nealen et al., 2007] Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. Fibermesh: designing freeform surfaces with 3d curves. In ACM SIGGRAPH 2007 papers, pages 41-50. ACM, 2007.","title":"Fibermesh: designing freeform surfaces with 3d curves","authors":"Nealen","year":"2007","venue":"ACM SIGGRAPH 2007 papers","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Qi et al., 2017] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proc. of CVPR, pages 652-660, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Hao Charles R Qi, Kaichun Su, Leonidas J Mo, Guibas","year":"2017","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rosales et al., 2019] Enrique Rosales, Jafet Rodriguez, and Alla Sheffer. Surfacebrush: from virtual reality draw- ings to manifold surfaces. ACM Trans. on Graphics, 38(4):96:1-96:15, 2019.","title":"Surfacebrush: from virtual reality drawings to manifold surfaces","authors":"Enrique Rosales, Jafet Rodriguez, Alla Sheffer","year":"2019","venue":"ACM Trans. on Graphics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sahay and Rajagopalan, 2015] Pratyush Sahay and AN Ra- jagopalan. Geometric inpainting of 3d structures. In Proc. of CVPR Workshops, pages 1-7, 2015.","title":"Geometric inpainting of 3d structures","authors":"Rajagopalan Sahay","year":"2015","venue":"Proc. of CVPR Workshops","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sharma et al., 2020] Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, and Radomír Měch. Parsenet: A parametric surface fitting net- work for 3d point clouds. In Proc. of ECCV, pages 261- 276. Springer, 2020.","title":"Parsenet: A parametric surface fitting network for 3d point clouds","authors":"Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, Radomír Měch","year":"2020","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Smirnov et al., 2019] Dmitriy Smirnov, Mikhail Bessmelt- sev, and Justin Solomon. Deep sketch-based modeling of man-made shapes. arXiv preprint arXiv:1906.12337, 2019.","title":"Deep sketch-based modeling of man-made shapes","authors":"Dmitriy Smirnov, Mikhail Bessmeltsev, Justin Solomon","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Torrence, 2006] Ann Torrence. Martin newell's original teapot. In ACM SIGGRAPH 2006 Teapot, pages 29-es. 2006.","title":"Martin newell's original teapot","authors":"Ann Torrence","year":"2006","venue":"ACM SIGGRAPH 2006 Teapot","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Verdera et al., 2003] Joan Verdera, Vicent Caselles, Marcelo Bertalmio, and Guillermo Sapiro. Inpainting surface holes. In Proc. of ICIP, volume 2, pages II-903. IEEE, 2003.","title":"Inpainting surface holes","authors":"Joan Verdera, Vicent Caselles, Marcelo Bertalmio, Guillermo Sapiro","year":"2003","venue":"Proc. of ICIP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2019] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Trans. On Graphics, 38(5):1-12, 2019.","title":"Dynamic graph cnn for learning on point clouds","authors":"Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, Justin M Solomon","year":"2019","venue":"ACM Trans. On Graphics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2022] Shuangbu Wang, Yu Xia, Lihua You, Hassan Ugail, Alfonso Carriazo, Andres Iglesias, and Jian- jun Zhang. Interactive pde patch-based surface modeling from vertex-frames. Engineering with Computers, pages 1-19, 2022.","title":"Interactive pde patch-based surface modeling from vertex-frames","authors":"Shuangbu Wang, Yu Xia, Lihua You, Hassan Ugail, Alfonso Carriazo, Andres Iglesias, Jianjun Zhang","year":"2022","venue":"Engineering with Computers","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2018] Pengfei Xu, Hongbo Fu, Youyi Zheng, Karan Singh, Hui Huang, and Chiew-Lan Tai. Model- guided 3d sketching. IEEE TVCG, pages 2927-2939, 2018.","title":"Modelguided 3d sketching","authors":"Pengfei Xu, Hongbo Fu, Youyi Zheng, Karan Singh, Hui Huang, Chiew-Lan Tai","year":"2018","venue":"IEEE TVCG","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2018] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proc. of CVPR, pages 206-215, 2018.","title":"Foldingnet: Point cloud auto-encoder via deep grid deformation","authors":"Yaoqing Yang, Chen Feng, Yiru Shen, Dong Tian","year":"2018","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhong and Qin, 2016] Ming Zhong and Hong Qin. Surface inpainting with sparsity constraints. CAGD, 41:23-35, 2016.","title":"Surface inpainting with sparsity constraints","authors":"Ming Zhong, Hong Qin","year":"2016","venue":"CAGD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}