{"type":"ScholarlyArticle","title":"Answering Mixed Type Questions about Daily Living Episodes","authors":[{"name":"Taiki Miyanishi","slug":"taiki-miyanishi","given":"Taiki","family":"Miyanishi","affiliation":"Advanced Telecommunications Research Institute International (ATR) , Kyoto , Japan"},{"name":"Jun-ichiro Hirayama","slug":"jun-ichiro-hirayama","given":"Jun-ichiro","family":"Hirayama","affiliation":"RIKEN Center for Advanced Intelligence Project (AIP) , Tokyo , Japan; Advanced Telecommunications Research Institute International (ATR) , Kyoto , Japan"},{"name":"Atsunori Kanemura","slug":"atsunori-kanemura","given":"Atsunori","family":"Kanemura","affiliation":"National Institute of Advanced Industrial Science and Technology (AIST) , Ibaraki , Japan; Advanced Telecommunications Research Institute International (ATR) , Kyoto , Japan"},{"name":"Motoaki Kawanabe","slug":"motoaki-kawanabe","given":"Motoaki","family":"Kawanabe","affiliation":"Advanced Telecommunications Research Institute International (ATR) , Kyoto , Japan; RIKEN Center for Advanced Intelligence Project (AIP) , Tokyo , Japan"}],"abstract":"We propose a physical-world question-answering (QA) method, where the system answers a text question about the physical world by searching a given sequence of sentences about daily-life episodes. To address various information needs in a physical world situation, the physical-world QA methods have to generate mixed-type responses (e.g. word sequence, word set, number, and time as well as a single word) according to the content of questions, after reading physical-world event stories. Most existing methods only provide words or choose answers from multiple candidates. In this paper, we use multiple decoders to generate a mixed-type answer encoding daily episodes with a memory architecture that can capture short- and long-term event dependencies. Results using house-activity stories show that the use of multiple decoders with memory components is effective for answering various physical-world QA questions.","doi":"10.24963/ijcai.2018/593","url":"https://ijcai.org/proceedings/2018/593","pdf":"https://ijcai.org/proceedings/2018/0593.pdf","markdown":"https://ijcai.org/proceedings/2018/593.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/593","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":593,"pages":{"first":4265,"last":4271},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Question Answering"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bao and Intille, 2004] Ling Bao and Stephen S Intille. Ac- tivity recognition from user-annotated acceleration data. In Pervasive, pages 1-17, 2004.","title":"Activity recognition from user-annotated acceleration data","authors":"Ling Bao, Stephen S Intille","year":"2004","venue":"Pervasive","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bordes et al., 2010] Antoine Bordes, Nicolas Usunier, Ro- nan Collobert, and Jason Weston. Towards understanding situated natural language. In AISTATS, 2010.","title":"Towards understanding situated natural language","authors":"Antoine Bordes, Nicolas Usunier, Ronan Collobert, Jason Weston","year":"2010","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP, pages 1724-1734, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collobert and Weston, 2008] Ronan Collobert and Jason Weston. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In ICML, pages 160-167, 2008.","title":"A unified architecture for natural language processing: Deep neural networks with multitask learning","authors":"Weston ; Collobert, Ronan Collobert, Jason Weston","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gemmell et al., 2002] Jim Gemmell, Gordon Bell, Roger Lueder, Steven Drucker, and Curtis Wong. Mylifebits: Fulfilling the memex vision. In ACMMM, pages 235-238, 2002.","title":"Mylifebits: Fulfilling the memex vision","authors":"Jim Gemmell, Gordon Bell, Roger Lueder, Steven Drucker, Curtis Wong","year":"2002","venue":"ACMMM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Graves et al., 2016] Alex Graves, Greg Wayne, Mal- colm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471-476, 2016.","title":"Hybrid computing using a neural network with dynamic external memory","authors":"Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hashimoto et al., 2017] Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. A joint many-task model: Growing a neural network for multiple nlp tasks. EMNLP, 2017.","title":"A joint many-task model: Growing a neural network for multiple nlp tasks","authors":"Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, Richard Socher","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hermann et al., 2015] Karl Moritz Hermann, Tomas Ko- cisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In NIPS, pages 1693-1701, 2015.","title":"Teaching machines to read and comprehend","authors":"Hermann","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hill et al., 2016] Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading chil- dren's books with explicit memory representations. ICLR, 2016.","title":"The goldilocks principle: Reading children's books with explicit memory representations","authors":"Felix Hill, Antoine Bordes, Sumit Chopra, Jason Weston","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krishna et al., 2017] Ranjay Krishna, Kenji Hata, Fred- eric Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense- captioning events in videos. In ICCV, pages 706-715, 2017.","title":"Densecaptioning events in videos","authors":"Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, Juan Carlos Niebles","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kumar et al., 2016] Ankit Kumar, Ozan Irsoy, Peter On- druska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. Ask me anything: Dynamic memory networks for natural lan- guage processing. In ICML, volume 48, pages 1378-1387, 2016.","title":"Ask me anything: Dynamic memory networks for natural language processing","authors":"Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, Richard Socher","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luong et al., 2016] Minh-Thang Luong, Quoc V Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. Multi-task sequence to sequence learning. ICLR, 2016.","title":"Multi-task sequence to sequence learning","authors":"Luong","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mikolov et al., 2010] Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. Re- current neural network based language model. In INTER- SPEECH, 2010.","title":"Recurrent neural network based language model","authors":"Mikolov","year":"2010","venue":"INTER-SPEECH","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Miller et al., 2017] A. H. Miller, W. Feng, A. Fisch, J. Lu, D. Batra, A. Bordes, D. Parikh, and J. Weston. Par- lai: A dialog research software platform. EMNLP System Demonstrations, pages 79-94, 2017.","title":"Parlai: A dialog research software platform","authors":"A H Miller, W Feng, A Fisch, J Lu, D Batra, A Bordes, D Parikh, J Weston","year":"2017","venue":"EMNLP System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Miyanishi et al., 2016] Taiki Miyanishi, Jun-ichiro Hi- rayama, Takuya Maekawa, Quan Kong, Hiroki Moriya, and Takayuki Suyama. Egocentric video search via physi- cal interactions. In AAAI, pages 330-336, 2016.","title":"Egocentric video search via physical interactions","authors":"Miyanishi","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Miyanishi et al., 2018] Taiki Miyanishi, Jun-ichiro Hi- rayama, Takuya Maekawa, and Motoaki Kawanabe. Gen- erating an event timeline about daily activities from a se- mantic concept stream. In AAAI, 2018.","title":"Generating an event timeline about daily activities from a semantic concept stream","authors":"Miyanishi","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Murugesan and Carbonell, 2017] Keerthiram Murugesan and Jaime Carbonell. Self-paced multitask learning with shared knowledge. In IJCAI, pages 2522-2528, 2017.","title":"Keerthiram Murugesan and Jaime Carbonell. Self-paced multitask learning with shared knowledge","authors":"Carbonell Murugesan","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100,000+ ques- tions for machine comprehension of text. In EMNLP, pages 2383-2392, 2016.","title":"Squad: 100,000+ questions for machine comprehension of text","authors":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Richardson et al., 2013] Matthew Richardson, Christo- pher JC Burges, and Erin Renshaw. Mctest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP, pages 193-203, 2013.","title":"Mctest: A challenge dataset for the open-domain machine comprehension of text","authors":"Matthew Richardson, J C Christopher, Erin Burges, Renshaw","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schuster and Paliwal, 1997] Mike Schuster and Kuldip K Paliwal. Bidirectional recurrent neural networks. IEEE Trans. Signal Process, 45(11):2673-2681, 1997.","title":"Bidirectional recurrent neural networks","authors":"Mike Schuster, Kuldip K Paliwal","year":"1997","venue":"IEEE Trans. Signal Process","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neu- ral networks from overfitting. JMLR, 15(1):1929-1958, 2014.","title":"Dropout: A simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sukhbaatar et al., 2015] Sainbayar Sukhbaatar, Jason We- ston, Rob Fergus, et al. End-to-end memory networks. In NIPS, pages 2440-2448, 2015.","title":"End-to-end memory networks","authors":"Sukhbaatar","year":"2015","venue":"Sainbayar Sukhbaatar","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Weston et al., 2015] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. ICLR, 2015.","title":"Memory networks","authors":"Jason Weston, Sumit Chopra","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Weston et al., 2016] Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. Towards AI-complete ques- tion answering: A set of prerequisite toy tasks. ICLR, 2016.","title":"Towards AI-complete question answering: A set of prerequisite toy tasks","authors":"Jason Weston, Antoine Bordes, Sumit Chopra, Tomas Mikolov","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2015] Yi Yang, Wen-tau Yih, and Christopher Meek. Wikiqa: A challenge dataset for open-domain ques- tion answering. In EMNLP, pages 2013-2018, 2015.","title":"Wikiqa: A challenge dataset for open-domain question answering","authors":"Yi Yang, Wen-Tau Yih, Christopher Meek","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}