{"type":"ScholarlyArticle","title":"A Theoretical Perspective on Hyperdimensional Computing (Extended Abstract)","authors":[{"name":"Anthony Thomas","slug":"anthony-thomas","given":"Anthony","family":"Thomas","affiliation":"Department of Computer Science and Engineering , UC San Diego , La Jolla CA"},{"name":"Sanjoy Dasgupta","slug":"sanjoy-dasgupta","given":"Sanjoy","family":"Dasgupta","affiliation":"Department of Computer Science and Engineering , UC San Diego , La Jolla CA"},{"name":"Tajana Rosing","slug":"tajana-rosing","given":"Tajana","family":"Rosing","affiliation":"Department of Computer Science and Engineering , UC San Diego , La Jolla CA"}],"abstract":"Hyperdimensional (HD) computing is a set of neurally inspired methods for computing on high-dimensional, low-precision, distributed representations of data. These representations can be combined with simple, neurally plausible algorithms to effect a variety of information processing tasks. HD computing has recently garnered significant interest from the computer hardware community as an energy-efficient, low-latency, and noise-robust tool for solving learning problems. We present a novel mathematical framework that unifies analysis of HD computing architectures, and provides general, non-asymptotic, sufficient conditions under which HD information processing techniques will succeed.","doi":"10.24963/ijcai.2022/808","url":"https://ijcai.org/proceedings/2022/808","pdf":"https://ijcai.org/proceedings/2022/0808.pdf","markdown":"https://ijcai.org/proceedings/2022/808.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/808","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Journal Track","track":"Journal Track","number":808,"pages":{"first":5772,"last":5776},"keywords":["Machine Learning: Symbolic methods","Knowledge Representation and Reasoning: Learning and reasoning","Knowledge Representation and Reasoning: Knowledge Representation Languages"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987242","length_sec":76,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985385","length_sec":718,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bell and Sejnowski, 1995] Anthony Bell and Terence Se- jnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129-1159, 1995.","title":"An information-maximization approach to blind separation and blind deconvolution","authors":"Anthony Bell, Terence Sejnowski","year":"1995","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bloom, 1970] Burton H Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7):422-426, 1970.","title":"Space/time trade-offs in hash coding with allowable errors","authors":"H Burton, Bloom","year":"1970","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Donoho et al., 2005] David L Donoho, Michael Elad, and Vladimir N Temlyakov. Stable recovery of sparse over- complete representations in the presence of noise. IEEE Transactions on Information Theory, 52(1):6-18, 2005.","title":"Stable recovery of sparse overcomplete representations in the presence of noise","authors":"Michael David L Donoho, Vladimir N Elad, Temlyakov","year":"2005","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Frady et al., 2018] E Paxon Frady, Denis Kleyko, and Friedrich T Sommer. A theory of sequence indexing and working memory in recurrent neural networks. Neural Computation, 30(6):1449-1513, 2018.","title":"A theory of sequence indexing and working memory in recurrent neural networks","authors":"Frady","year":"2018","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fukushima, 1980] Kunihiko Fukushima. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biolog- ical Cybernetics, 36(4):193-202, 1980.","title":"Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position","authors":"Kunihiko Fukushima","year":"1980","venue":"Biological Cybernetics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gallant and Okaywe, 2013] Stephen I Gallant and T Wendy Okaywe. Representing objects, relations, and sequences. Neural Computation, 25(8):2038-2078, 2013.","title":"Representing objects, relations, and sequences","authors":"I Stephen, T Gallant, Okaywe Wendy","year":"2013","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gupta et al., 2018] Saransh Gupta, Mohsen Imani, and Ta- jana Rosing. Felix: Fast and energy-efficient logic in memory. In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pages 1-7. IEEE, 2018.","title":"Felix: Fast and energy-efficient logic in memory","authors":"Gupta","year":"2018","venue":"IEEE/ACM International Conference on Computer-Aided Design (ICCAD)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Imani et al., 2017] Mohsen Imani, Abbas Rahimi, Deqian Kong, Tajana Rosing, and Jan M Rabaey. Exploring hy- perdimensional associative memory. In 2017 IEEE Inter- national Symposium on High Performance Computer Ar- chitecture (HPCA), pages 445-456. IEEE, 2017.","title":"Exploring hyperdimensional associative memory","authors":"Mohsen Imani, Abbas Rahimi, Deqian Kong, Tajana Rosing, Jan M Rabaey","year":"2017","venue":"IEEE International Symposium on High Performance Computer Architecture (HPCA)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Imani et al., 2019] Mohsen Imani, Sahand Salamat, Saransh Gupta, Jiani Huang, and Tajana Rosing. Fach: Fpga-based acceleration of hyperdimensional computing by reducing computational complexity. In Proceedings of the 24th Asia and South Pacific Design Automation Conference, pages 493-498. ACM, 2019.","title":"Fach: Fpga-based acceleration of hyperdimensional computing by reducing computational complexity","authors":"Mohsen Imani, Sahand Salamat, Saransh Gupta, Jiani Huang, Tajana Rosing","year":"2019","venue":"Proceedings of the 24th Asia and South Pacific Design Automation Conference","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kanerva, 1994] Pentti Kanerva. The spatter code for encod- ing concepts at many levels. In International Conference on Artificial Neural Networks, pages 226-229. Springer, 1994.","title":"The spatter code for encoding concepts at many levels","authors":"Pentti Kanerva, Kanerva","year":"1994","venue":"International Conference on Artificial Neural Networks","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kanerva, 1995] Pentti Kanerva. A family of binary spatter codes. In International Conference on Artificial Neural Networks, volume 1, pages 517-522, 1995.","title":"A family of binary spatter codes","authors":"Pentti Kanerva","year":"1995","venue":"International Conference on Artificial Neural Networks","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kanerva, 2009] Pentti Kanerva. Hyperdimensional comput- ing: An introduction to computing in distributed represen- tation with high-dimensional random vectors. Cognitive Computation, 1(2):139-159, 2009.","title":"Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors","authors":"Pentti Kanerva","year":"2009","venue":"Cognitive Computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kleyko et al., 2018] Denis Kleyko, Abbas Rahimi, Dmitri A Rachkovskij, Evgeny Osipov, and Jan M Rabaey. Classification and recall with binary hyperdi- mensional computing: Tradeoffs in choice of density and mapping characteristics. IEEE Transactions on Neural Networks and Learning Systems, 29(12):5880-5898, 2018.","title":"Classification and recall with binary hyperdimensional computing: Tradeoffs in choice of density and mapping characteristics","authors":"Denis Kleyko, Abbas Rahimi, Dmitri A Rachkovskij, Evgeny Osipov, Jan M Rabaey","year":"2018","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Levy and Gayler, 2008] Simon D Levy and Ross Gayler. Vector symbolic architectures: A new building material for artificial general intelligence. In Conference on Artificial General Intelligence, pages 414-418. IOS Press, 2008.","title":"Vector symbolic architectures: A new building material for artificial general intelligence","authors":"D Simon, Ross Levy, Gayler","year":"2008","venue":"Conference on Artificial General Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Olshausen and Field, 1996] Bruno Olshausen and David Field. Emergence of simple-cell receptive field proper- ties by learning a sparse code for natural images. Nature, 381:607-609, 1996.","title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","authors":"Field Olshausen, Bruno Olshausen, David Field","year":"1996","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Plate, 1995] Tony A Plate. Holographic reduced representa- tions. IEEE Transactions on Neural Networks, 6(3):623- 641, 1995.","title":"Holographic reduced representations","authors":"Tony A Plate","year":"1995","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Plate, 2003] Tony Plate. Holographic Reduced Representa- tion: Distributed Representation for Cognitive Structures. CSLI Lecture Notes (CSLI-CHUP) Series. CSLI Publica- tions, 2003.","title":"Holographic Reduced Representation: Distributed Representation for Cognitive Structures","authors":"Tony Plate","year":"2003","venue":"CSLI Lecture Notes","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rahimi et al., 2018] Abbas Rahimi, Pentti Kanerva, Luca Benini, and Jan M Rabaey. Efficient biosignal processing using hyperdimensional computing: Network templates for combined learning and classification of ExG signals. Proceedings of the IEEE, 107(1):123-143, 2018.","title":"Efficient biosignal processing using hyperdimensional computing: Network templates for combined learning and classification of ExG signals","authors":"Abbas Rahimi, Pentti Kanerva, Luca Benini, Jan M Rabaey","year":"2018","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rosenblatt, 1958] Frank Rosenblatt. The Perceptron: A probabilistic model for information storage and organiza- tion in the brain. Psychological Review, 65(6):386-408, 1958.","title":"The Perceptron: A probabilistic model for information storage and organiza-tion in the brain","authors":"Frank Rosenblatt","year":"1958","venue":"Psychological Review","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rumelhart et al., 1986] David Rumelhart, James McClel- land, and the PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructure of Cogni- tion, Volume 1: Foundations. MIT Press, 1986.","title":"James McClelland, and the PDP Research Group","authors":"David Rumelhart","year":"1986","venue":"Parallel Distributed Processing: Explorations in the Microstructure of Cognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Salamat et al., 2019] Sahand Salamat, Mohsen Imani, Behnam Khaleghi, and Tajana Rosing. F5-hd: Fast flexible fpga-based framework for refreshing hyperdimensional computing. In Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pages 53-62, 2019.","title":"F5-hd: Fast flexible fpga-based framework for refreshing hyperdimensional computing","authors":"Salamat","year":"2019","venue":"Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schmuck et al., 2019] Manuel Schmuck, Luca Benini, and Abbas Rahimi. Hardware optimizations of dense binary hyperdimensional computing: Rematerialization of hyper- vectors, binarized bundling, and combinational associative memory. ACM Journal on Emerging Technologies in Com- puting Systems (JETC), 15(4):1-25, 2019.","title":"Hardware optimizations of dense binary hyperdimensional computing: Rematerialization of hypervectors, binarized bundling, and combinational associative memory","authors":"Manuel Schmuck, Luca Benini, Abbas Rahimi","year":"2019","venue":"ACM Journal on Emerging Technologies in Computing Systems (JETC)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Thomas et al., 2021] Anthony Thomas, Sanjoy Dasgupta, and Tajana Rosing. Theoretical foundations of hyperdi- mensional computing. Journal of Artificial Intelligence Research, 72:215-249, 2021.","title":"Theoretical foundations of hyperdimensional computing","authors":"Anthony Thomas, Sanjoy Dasgupta, Tajana Rosing","year":"2021","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}