{"type":"ScholarlyArticle","title":"Encoding and Recall of Spatio-Temporal Episodic Memory in Real Time","authors":[{"name":"Poo-Hee Chang","slug":"poo-hee-chang","given":"Poo-Hee","family":"Chang","affiliation":"School of Computer Science and Engineering Nanyang Technological University Singapore 639798"},{"name":"Ah-Hwee Tan","slug":"ah-hwee-tan","given":"Ah-Hwee","family":"Tan","affiliation":"School of Computer Science and Engineering Nanyang Technological University Singapore 639798"}],"abstract":"Episodic memory enables a cognitive system to improve its performance by reflecting upon past events. In this paper, we propose a computational model called STEM for encoding and recall of episodic events together with the associated contextual information in real time. Based on a class of self-organizing neural networks, STEM is designed to learn memory chunks or cognitive nodes, each encoding a set of co-occurring multi-modal activity patterns across multiple pattern channels. We present algorithms for recall of events based on partial and inexact input patterns. Our empirical results based on a public domain data set show that STEM displays a high level of efficiency and robustness in encoding and retrieval with both partial and noisy search cues when compared with a state-of-the-art associative memory model.","doi":"10.24963/ijcai.2017/206","url":"https://ijcai.org/proceedings/2017/206","pdf":"https://ijcai.org/proceedings/2017/0206.pdf","markdown":"https://ijcai.org/proceedings/2017/206.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/206","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra, IIIA-CSIC","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":206,"pages":{"first":1490,"last":1496},"keywords":["Multidisciplinary Topics and Applications: Cognitive Modeling","Machine Learning: Neural Networks"],"videos":[],"references":[{"position":1,"raw":"[Angelov and Sperduti, 2016] Plamen Angelov and Alessan- dro Sperduti. Challenges in deep learning. In Proceedings of the 24th European Symposium on Artificial Neural Net- works (ESANN). i6doc. com, pages 489-495, 2016.","title":"Challenges in deep learning","authors":"Plamen Angelov, Alessandro Sperduti","year":"2016","venue":"Proceedings of the 24th European Symposium on Artificial Neural Networks (ESANN). i6doc. com","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carpenter and Grossberg, 2003] Gail A Carpenter and Stephen Grossberg. Adaptive Resonance Theory. In Michael A Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 87-90. Cambridge, MA: MIT Press, 2003.","title":"Adaptive Resonance Theory","authors":"A Gail, Stephen Carpenter, Grossberg","year":"2003","venue":"The Handbook of Brain Theory and Neural Networks","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fisher et al., 2005] Robert Fisher, Jose Santos-Victor, and James Crowley. CAVIAR: Context aware vision using image-based active recognition, 2005.","title":"CAVIAR: Context aware vision using image-based active recognition","authors":"Robert Fisher, Jose Santos-Victor, James Crowley","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fisher, 2004] Robert B Fisher. The PETS04 surveillance ground-truth data sets. In Proceedings of the 6th IEEE in- ternational workshop on performance evaluation of track- ing and surveillance, pages 1-5, 2004.","title":"The PETS04 surveillance ground-truth data sets","authors":"B Robert, Fisher","year":"2004","venue":"Proceedings of the 6th IEEE international workshop on performance evaluation of tracking and surveillance","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Graves et al., 2014] Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv:1410.5401, 2014.","title":"Neural turing machines","authors":"Alex Graves, Greg Wayne, Ivo Danihelka","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Grossberg and Merrill, 1996] Stephen Grossberg and John W.L. Merrill. The hippocampus and cerebellum in adaptively timed learning, recognition, and movement. Journal of Cognitive Neuroscience, 8(3):257-277, 1996.","title":"The hippocampus and cerebellum in adaptively timed learning, recognition, and movement","authors":"Stephen Grossberg, John W L Merrill","year":"1996","venue":"Journal of Cognitive Neuroscience","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ho et al., 2003] Wan Ching Ho, Kerstin Dautenhahn, and Chrystopher L Nehaniv. Comparing different control ar- chitectures for autobiographic agents in static virtual envi- ronments. In International Workshop on Intelligent Virtual Agents, pages 182-191. Springer, 2003.","title":"Comparing different control architectures for autobiographic agents in static virtual environments","authors":"Wan Ching Ho, Kerstin Dautenhahn, Chrystopher L Nehaniv","year":"2003","venue":"International Workshop on Intelligent Virtual Agents","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kaiser and Sutskever, 2015] Łukasz Kaiser and Ilya Sutskever. Neural GPUs learn algorithms. arXiv:1511.08228, 2015.","title":"Neural GPUs learn algorithms","authors":"Łukasz Kaiser, Ilya Sutskever","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mueller and Shiffrin, 2006] Shane T Mueller and Richard M Shiffrin. REM-II: a model of the devel- opment co-evolution of episodic memory and semantic knowledge. In Proceedings of the International Confer- ence on Development and Learning, volume 5, 2006.","title":"REM-II: a model of the development co-evolution of episodic memory and semantic knowledge","authors":"T Shane, Richard M Mueller, Shiffrin","year":"2006","venue":"Proceedings of the International Conference on Development and Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Nuxoll and Laird, 2007] Andrew M. Nuxoll and John E. Laird. Extending cognitive architecture with episodic memory. In AAAI, pages 1560-1564. AAAI Press, 2007.","title":"Extending cognitive architecture with episodic memory","authors":"Andrew M Nuxoll, John E Laird","year":"2007","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Nuxoll, 2007] Andrew M Nuxoll. Enhancing Intelligent Agents with Episodic Memory. PhD thesis, University of Michigan, 2007.","title":"Enhancing Intelligent Agents with Episodic Memory","authors":"M Andrew, Nuxoll","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[O'Keefe and Dostrovsky, 1971] John O'Keefe and Jonathan Dostrovsky. The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat. Brain research, 34(1):171-175, 1971.","title":"John O'Keefe and Jonathan Dostrovsky. The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat","authors":"O' Keefe, Dostrovsky","year":"1971","venue":"Brain research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rinkus, 2004] Gerard J Rinkus. A neural model of episodic and semantic spatiotemporal memory. In Proceedings of the 26th Annual Conference of Cognitive Science Society, pages 1155-1160, Chicago, 2004. LEA.","title":"A neural model of episodic and semantic spatiotemporal memory","authors":"J Gerard, Rinkus","year":"2004","venue":"Proceedings of the 26th Annual Conference of Cognitive Science Society","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Samsonovich and Ascoli, 2005] Alexei V Samsonovich and Giorgio A Ascoli. A simple neural network model of the hippocampus suggesting its pathfinding role in episodic memory retrieval. Learning and Memory, 12(2):193-208, 2005.","title":"A simple neural network model of the hippocampus suggesting its pathfinding role in episodic memory retrieval","authors":"Alexei V Samsonovich, Giorgio A Ascoli","year":"2005","venue":"Learning and Memory","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shastri, 2002] Lokendra Shastri. Episodic memory and cortico-hippocampal interactions. TRENDS in cognitive sciences, 6(4):162-168, April 2002.","title":"Lokendra Shastri. Episodic memory and cortico-hippocampal interactions","authors":"Shastri","year":"2002","venue":"TRENDS in cognitive sciences","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shen et al., 2013] Furao Shen, Qiubao Ouyang, Wataru Ka- sai, and Osamu Hasegawa. A general associative memory based on self-organizing incremental neural network. Neu- rocomputing, 104:57-71, 2013.","title":"A general associative memory based on self-organizing incremental neural network","authors":"Shen","year":"2013","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Starzyk and He, 2009] Janusz A Starzyk and Haibo He. Spatio-temporal memories for machine learning: A long- term memory organization. IEEE Transactions on Neural Networks, 20(5):768-780, 2009.","title":"Spatio-temporal memories for machine learning: A longterm memory organization","authors":"A Janusz, Haibo Starzyk, He","year":"2009","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Subagdja and Tan, 2015] Budhitama Subagdja and Ah- Hwee Tan. Neural modeling of sequential inferences and learning over episodic memory. Neurocomputing, 161:229-242, 2015.","title":"Budhitama Subagdja and Ah-Hwee Tan. Neural modeling of sequential inferences and learning over episodic memory","authors":"Tan Subagdja","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Subagdja et al., 2012] Budhitama Subagdja, Wenwen Wang, Ah-Hwee Tan, Yuan-Sin Tan, and Loo-Nin Teow. Memory formation, consolidation, and forgetting in learning agents. In Proceedings, Eleventh International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), pages 1007-1014, 2012.","title":"Memory formation, consolidation, and forgetting in learning agents","authors":"Subagdja","year":"2012","venue":"Proceedings, Eleventh International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tan et al., 2007] Ah-Hwee Tan, Gail A Carpenter, and Stephen Grossberg. Intelligence through interaction: To- wards a unified theory for learning. In International Sym- posium on Neural Networks, pages 1094-1103. Springer, 2007.","title":"Intelligence through interaction: Towards a unified theory for learning","authors":"Tan","year":"2007","venue":"International Symposium on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tulving, 1983] Endel Tulving. Elements of episodic mem- ory. Oxford University Press, 1983.","title":"Elements of episodic memory","authors":"Endel Tulving, Tulving","year":"1983","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Valdez et al., 2015] André B Valdez, Megan H Papesh, David M Treiman, Kris A Smith, Stephen D Goldinger, and Peter N Steinmetz. Distributed representation of vi- sual objects by single neurons in the human brain. Journal of Neuroscience, 35(13):5180-5186, 2015.","title":"Distributed representation of visual objects by single neurons in the human brain","authors":"B André, Megan H Valdez, Papesh, Kris A David M Treiman, Stephen D Smith, Peter N Goldinger, Steinmetz","year":"2015","venue":"Journal of Neuroscience","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vere and Bickmore, 1990] Steven Vere and Timothy Bick- more. A basic agent. Computational intelligence, 6(1):41- 60, 1990.","title":"Steven Vere and Timothy Bickmore. A basic agent","authors":"Bickmore Vere","year":"1990","venue":"Computational intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang and Arbib, 1993] DeLiang Wang and Michael A Ar- bib. Timing and chunking in processing temporal order. IEEE Transactions on Systems, Man, and Cybernetics, 23(4):993-1009, 1993.","title":"DeLiang Wang and Michael A Arbib. Timing and chunking in processing temporal order","authors":"Arbib Wang","year":"1993","venue":"IEEE Transactions on Systems, Man, and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2012] Wenwen Wang, Budhitama Subagdja, Ah-Hwee Tan, and Janusz A Starzyk. Neural modeling of episodic memory: Encoding, retrieval, and forgetting. IEEE transactions on neural networks and learning sys- tems, 23(10):1574-1586, 2012.","title":"Neural modeling of episodic memory: Encoding, retrieval, and forgetting","authors":"Wenwen Wang, Budhitama Subagdja, Ah-Hwee Tan, Janusz A Starzyk","year":"2012","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2016] Di Wang, Ah-Hwee Tan, and Chunyan Miao. Modeling autobiographical memory in human-like autonomous agents. In Proceedings of the 2016 Interna- tional Conference on Autonomous Agents & Multiagent Systems, pages 845-853, 2016.","title":"Modeling autobiographical memory in human-like autonomous agents","authors":"Di Wang, Ah-Hwee Tan, Chunyan Miao","year":"2016","venue":"Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Weston et al., 2014] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. arXiv:1410.3916, 2014.","title":"Memory networks","authors":"Jason Weston, Sumit Chopra","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}