{"type":"ScholarlyArticle","title":"Fine-tuning Deep Neural Networks by Interactively Refining the 2D Latent Space of Ambiguous Images","authors":[{"name":"Jiafu Wei","slug":"jiafu-wei","given":"Jiafu","family":"Wei","affiliation":"Jilin University"},{"name":"Haoran Xie","slug":"haoran-xie","given":"Haoran","family":"Xie","affiliation":"Japan Advanced Institute of Science and Technology"},{"name":"Chia-Ming Chang","slug":"chia-ming-chang","given":"Chia-Ming","family":"Chang","affiliation":"The University of Tokyo"},{"name":"Xi Yang","slug":"xi-yang","given":"Xi","family":"Yang","affiliation":"Jilin University"}],"abstract":"Deep neural networks (DNNs) have achieved excellent results currently in classification, while they may still suffer from ambiguous images which are similar across classes. By contrast, humans have a relatively good ability to distinguish these categories of images. Therefore, we propose a human-in-the-loop solution to assist the network to better classify the images by leveraging human knowledge. To achieve this, we project the high-dimensional latent space trained by the network onto a two-dimensional workspace. The users can interactively modify the projected coordinates of inputs on the workspace using our designed tools, then the modified information will be fed back to the network to fine-tune it, which in turn affects the network's classification results, thereby improving the accuracy of network classification.","doi":"10.24963/ijcai.2022/861","url":"https://ijcai.org/proceedings/2022/861","pdf":"https://ijcai.org/proceedings/2022/0861.pdf","markdown":"https://ijcai.org/proceedings/2022/861.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/861","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Demo Track","track":"Demo Track","number":861,"pages":{"first":5948,"last":5951},"keywords":["Humans and AI: Human-AI Collaboration","Humans and AI: Human-Computer Interaction"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987208","length_sec":85,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985351","length_sec":312,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Asai et al., 2020] Kentaro Asai, Tsukasa Fukusato, and Takeo Igarashi. Integrated development environment with interactive scatter plot for examining statistical modeling. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1-7, 2020.","title":"Integrated development environment with interactive scatter plot for examining statistical modeling","authors":"Kentaro Asai, Tsukasa Fukusato, Takeo Igarashi","year":"2020","venue":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2021] Chia-Ming Chang, Chia-Hsien Lee, and Takeo Igarashi. Spatial labeling: Leveraging spatial lay- out for improving label quality in non-expert image an- notation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1-12, 2021.","title":"Spatial labeling: Leveraging spatial layout for improving label quality in non-expert image annotation","authors":"Chia-Ming Chang, Chia-Hsien Lee, Takeo Igarashi","year":"2021","venue":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Huang and Canny, 2019] Forrest Huang and John F Canny. Sketchforme: Composing sketched scenes from text de- scriptions for interactive applications. In Proceedings of the 32nd annual ACM symposium on user interface soft- ware and technology, pages 209-220, 2019.","title":"Sketchforme: Composing sketched scenes from text descriptions for interactive applications","authors":"Forrest Huang, John F Canny","year":"2019","venue":"Proceedings of the 32nd annual ACM symposium on user interface software and technology","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Igarashi et al., 2016] Takeo Igarashi, Naoyuki Shono, Taichi Kin, and Toki Saito. Interactive volume segmenta- tion with threshold field painting. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, pages 403-413, 2016.","title":"Interactive volume segmentation with threshold field painting","authors":"Igarashi","year":"2016","venue":"Proceedings of the 29th Annual Symposium on User Interface Software and Technology","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Jongejan et al., 2016] Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. The quick, draw!-ai experiment. Mount View, CA, accessed Feb, 17(2018):4, 2016.","title":"The quick, draw!-ai experiment","authors":"Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, Nick Fox-Gieg","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Larsson et al., 2020] Maria Larsson, Hironori Yoshida, Nobuyuki Umetani, and Takeo Igarashi. Tsugite: Interac- tive design and fabrication of wood joints. In UIST, pages 317-327, 2020.","title":"Tsugite: Interactive design and fabrication of wood joints","authors":"Maria Larsson, Hironori Yoshida, Nobuyuki Umetani, Takeo Igarashi","year":"2020","venue":"UIST","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Nashed and Biswas, 2018] Samer Nashed and Joydeep Biswas. Human-in-the-loop slam. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.","title":"Human-in-the-loop slam","authors":"Samer Nashed, Joydeep Biswas","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Sakata et al., 2019] Yusuke Sakata, Yukino Baba, and Hisashi Kashima. Crownn: Human-in-the-loop network with crowd-generated inputs. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7555-7559. IEEE, 2019.","title":"Crownn: Human-in-the-loop network with crowd-generated inputs","authors":"Sakata","year":"2019","venue":"ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Wang et al., 2014] Fangzhou Wang, Yang Li, Daisuke Sakamoto, and Takeo Igarashi. Hierarchical route maps for efficient navigation. In Proceedings of the 19th inter- national conference on Intelligent User Interfaces, pages 169-178, 2014.","title":"Hierarchical route maps for efficient navigation","authors":"Wang","year":"2014","venue":"Proceedings of the 19th international conference on Intelligent User Interfaces","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Yang et al., 2019] Xi Yang, Bojian Wu, Issei Sato, and Takeo Igarashi. Directing dnns attention for facial attribu- tion classification using gradient-weighted class activation mapping. In CVPR Workshops, pages 103-106, 2019.","title":"Directing dnns attention for facial attribution classification using gradient-weighted class activation mapping","authors":"Xi Yang, Bojian Wu, Issei Sato, Takeo Igarashi","year":"2019","venue":"CVPR Workshops","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}