{"type":"ScholarlyArticle","title":"Convergence Time Characteristics of an Associative Memory for Natural Language Processing","authors":[{"name":"Nigel Collier","slug":"nigel-collier","given":"Nigel","family":"Collier","affiliation":"Department of Language Engineering UMIST PO Box 88 , Manchester M60 lQD , United Kingdom"}],"abstract":"We take a new look at one of the fundamental properties of discrete time associative memory and show how it can be adapted for natural language processing (NLP). Many tasks in NLP could benefit from such associative functionality particularly those which are traditionally regarded as being context driven such as word sense disambiguation . The results describe the typical time to convergence of a Hopfield network when trained on patterns representing sentences from a large corpus. Through numerical simulation we estimate the time order of convergence and compare this to previous findings for randomly generated, unbiased and uncorrelated patterns.","doi":null,"url":"https://ijcai.org/proceedings/1997/160","pdf":"https://ijcai.org/proceedings/1997/0160.pdf","markdown":"https://ijcai.org/proceedings/1997/160.md","bibtex":"https://ijcai.org/proceedings/1997/bibtex/160","volume":{"slug":"1997","title":"Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 1997","url":"https://ijcai.org/proceedings/1997/","year":1997,"location":"Nagoya, Japan","dates":"","editors":"Martha E. Pollack","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Neural Networks","track":"Neural Networks","number":160,"pages":{"first":1106,"last":1112},"keywords":[],"videos":[],"references":[{"position":1,"raw":"[Amit et al., 1987a] D. Amit, H. Gutfeund, and H . Som- polinsky. Information storage in neural networks with low levels of activity. Phys. Rev. A, 35(5):2293+, 1987.","title":"Information storage in neural networks with low levels of activity","authors":"Amit","year":"1987","venue":"Phys. Rev. A","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amit et al., 1987b] D. Amit, H. Gutfeund, and H. Som- polinsky. Statistical mechanics of neural networks near saturation. Ann. Phys. , 173:30+, 1987.","title":"Statistical mechanics of neural networks near saturation","authors":"Amit","year":"1987","venue":"Ann. Phys","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Amit, 1989] D. Amit. Modeling Brain Function -The world of attroctor neural networks. Cambridge, Eng- land: Cambridge University Press, 1989.","title":"Modeling Brain Function -The world of attroctor neural networks","authors":"D Amit","year":"1989","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bruce et al., 1987] A. Bruce, E. Gardner, and D. Wal- lace. Dynamics and statistical mechanics of the Hop- field model. Journal of Physics A, 20:2909-2934, 1987.","title":"Dynamics and statistical mechanics of the Hopfield model","authors":"A Bruce, E Gardner, D Wallace","year":"1987","venue":"Journal of Physics A","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collier, 1996] N. Collier. Storage of natural language sentences in a Hopfield network . In International Conference on New Methods in Language Processing (NeMLaP-II), Bilkent University, Ankam, Turkey, September 11-13th 1996.","title":"Storage of natural language sentences in a Hopfield network","authors":"N Collier","year":"1996","venue":"International Conference on New Methods in Language Processing (NeMLaP-II)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gardner, 1986] E. Gardner. Structure of metastable states in the Hopfield model. Journal of Physics A, 19:L1047-L1052, 1986.","title":"Structure of metastable states in the Hopfield model","authors":"E Gardner","year":"1986","venue":"Journal of Physics A","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Grensing et al., 1987] D. Grensing, R. Kiihn, and J . van Hammen. Storing patterns in a spin-glass model of neural networks near saturation. Journal of Physics A, 20:2935-2947, 1987.","title":"Storing patterns in a spin-glass model of neural networks near saturation","authors":"D Grensing, R Kiihn, J Van Hammen","year":"1987","venue":"Journal of Physics A","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hopfield, 1982] J .J. Hopfield. Neural networks and physical systems with emergent selective computa- tional abilities. Proceedings of the National Academy of Science, USA, 79:2554+, 1982.","title":"Neural networks and physical systems with emergent selective computational abilities","authors":"J J Hopfield","year":"1982","venue":"Proceedings of the National Academy of Science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Hopfield, 1984)","title":null,"authors":"Hopfield","year":"1984","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"J .J. Hopfield. Neurons with graded re- sponse have collective computational properties like those of two-state neurons. Proceedings of the Na- tional Academy of Science, USA, 81:3088-3092, May 1984.","title":"Neurons with graded response have collective computational properties like those of two-state neurons","authors":"J J Hopfield","year":"1984","venue":"Proceedings of the National Academy of Science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koiran, 1994] P. Koiran. Dynamics of discrete time, continuous state Hopfield networks. Neural Computa- tion , 6(3):459-468, May 1994.","title":"Dynamics of discrete time, continuous state Hopfield networks","authors":"P Koiran","year":"1994","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Perez-Vicente and Amit, 1989] C. Perez-Vicente and D. Amit. Optimised network for sparsely coded pat- terns. Journal of Physics A, 22:559-569, 1989.","title":"Optimised network for sparsely coded patterns","authors":"C Perez-Vicente, D Amit","year":"1989","venue":"Journal of Physics A","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Tanaka and Yamada, 1993] T. Tanaka and M. Yamada. The characteristics of the convergence time of associa- tive naural networks. Neural Computation, 5(3):463- 472, May 1993.","title":"The characteristics of the convergence time of associative naural networks","authors":"T Tanaka, M Yamada","year":"1993","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Treves and Amit, 1988] A. Treves and D. Amit. Metastable states in asymmetrically diluted hopfield networks. Journal of Physics A, 21:3155-3169, 1988.","title":"Metastable states in asymmetrically diluted hopfield networks","authors":"A Treves, D Amit","year":"1988","venue":"Journal of Physics A","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}