{"type":"ScholarlyArticle","title":"Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?","authors":[{"name":"Beren Millidge","slug":"beren-millidge","given":"Beren","family":"Millidge","affiliation":"MRC Brain Network Dynamics Unit , University of Oxford , UK"},{"name":"Tommaso Salvatori","slug":"tommaso-salvatori","given":"Tommaso","family":"Salvatori","affiliation":"Department of Computer Science , University of Oxford , UK"},{"name":"Yuhang Song","slug":"yuhang-song","given":"Yuhang","family":"Song","affiliation":"Department of Computer Science , University of Oxford , UK; MRC Brain Network Dynamics Unit , University of Oxford , UK"},{"name":"Rafal Bogacz","slug":"rafal-bogacz","given":"Rafal","family":"Bogacz","affiliation":"MRC Brain Network Dynamics Unit , University of Oxford , UK"},{"name":"Thomas Lukasiewicz","slug":"thomas-lukasiewicz","given":"Thomas","family":"Lukasiewicz","affiliation":"Institute of Logic and Computation , TU Wien , Austria; Department of Computer Science , University of Oxford , UK"}],"abstract":"The backpropagation of error algorithm (BP) used to train deep neural networks has been fundamental to the successes of deep learning. However, it requires sequential backwards updates and non-local computations which make it challenging to parallelize at scale and is unlike how learning works in the brain. Neuroscience-inspired learning algorithms, however, such as \\emph{predictive coding} which utilize local learning have the potential to overcome these limitations and advance beyond deep learning technologies in the future. While predictive coding originated in theoretical neuroscience as a model of information processing in the cortex, recent work has developed the idea into a general-purpose algorithm  able to train neural networks using only local computations. In this survey, we review works that have contributed to this perspective and demonstrate the close connection between predictive coding and backpropagation in terms of generalization quality, as well as works that highlight the multiple advantages of using predictive coding models over backprop-trained neural networks. Specifically, we show the substantially greater flexibility of  predictive coding networks against equivalent deep neural networks, which can function as classifiers, generators, and associative memories simultaneously, and can be defined on arbitrary graph topologies. Finally, we  review direct benchmarks of predictive coding networks on machine learning classification tasks, as well as its close connections to control theory and applications in robotics.","doi":"10.24963/ijcai.2022/774","url":"https://ijcai.org/proceedings/2022/774","pdf":"https://ijcai.org/proceedings/2022/0774.pdf","markdown":"https://ijcai.org/proceedings/2022/774.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/774","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Survey Track","track":"Survey Track","number":774,"pages":{"first":5538,"last":5545},"keywords":["Survey Track: Multidisciplinary Topics and Applications","Survey Track: Machine Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987310","length_sec":91,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985453","length_sec":190,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Attias, 2003] H. Attias. Planning by probabilistic inference. In Proc. AISTATS. PMLR, 2003.","title":"Planning by probabilistic inference","authors":"H Attias","year":"2003","venue":"Proc. AISTATS. PMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Auksztulewicz and Friston, 2016] R. Auksztulewicz and K. Friston. Repetition suppression and its contextual de- terminants in predictive coding. Cortex, 2016.","title":"Repetition suppression and its contextual determinants in predictive coding","authors":"R Auksztulewicz, K Friston","year":"2016","venue":"Cortex","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Avena-Koenigsberger et al., 2018] A. Avena-Koenigsber- ger, B. Misic, and O. Sporns. Communication dynamics in complex brain networks. Nat. Rev. Neurosci., 19, 2018.","title":"Communication dynamics in complex brain networks","authors":"Avena-Koenigsberger","year":"2018","venue":"Nat. Rev. Neurosci","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Baltieri and Buckley, 2019] M. Baltieri and C. Buckley. PID control as a process of active inference with linear generative models. Entropy, 2019.","title":"PID control as a process of active inference with linear generative models","authors":"M Baltieri, C Buckley","year":"2019","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Barlow, 2001] H. Barlow. Redundancy reduction revisited. Network, 2001.","title":"Redundancy reduction revisited","authors":"H Barlow","year":"2001","venue":"Network","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Barron et al., 2020] H. Barron, R. Auksztulewicz, and K. Friston. Prediction and memory: A predictive coding account. Prog. Neurobiol., 2020.","title":"Prediction and memory: A predictive coding account","authors":"H Barron, R Auksztulewicz, K Friston","year":"2020","venue":"Prog. Neurobiol","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Beal, 2003] M. Beal. Variational algorithms for approxi- mate Bayesian inference. UCL, London, 2003.","title":"Variational algorithms for approximate Bayesian inference","authors":"M Beal","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bogacz, 2017] R. Bogacz. A tutorial on the free-ener- gy framework for modelling perception and learning. J. Math. Psychol., 2017.","title":"A tutorial on the free-energy framework for modelling perception and learning","authors":"R Bogacz","year":"2017","venue":"J. Math. Psychol","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Bogacz, 2020] R. Bogacz. Dopamine role in learning and action inference. Elife, 2020.","title":"Dopamine role in learning and action inference","authors":"R Bogacz","year":"2020","venue":"Elife","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Bradbury, 2000] J. Bradbury. Linear predictive coding. Mc- Graw Hill, 2000.","title":"Linear predictive coding","authors":"J Bradbury","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Buckley et al., 2017] C. Buckley, C. S. Kim, S. McGregor, and A. Seth. The free energy principle for action and per- ception: A mathematical review. J. Math. Psychol., 2017.","title":"The free energy principle for action and perception: A mathematical review","authors":"C Buckley, C S Kim, S Mcgregor, A Seth","year":"2017","venue":"J. Math. Psychol","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Clark, 2015] A. Clark. Surfing Uncertainty: Prediction, Ac- tion, and the Embodied Mind. OUP, 2015.","title":"Surfing Uncertainty: Prediction, Action, and the Embodied Mind","authors":"A Clark","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Friston et al., 2006] K. Friston, J. Kilner, and L. Harrison. A free energy principle for the brain. J. Physiol., 2006.","title":"A free energy principle for the brain","authors":"K Friston, J Kilner, L Harrison","year":"2006","venue":"J. Physiol","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Friston et al., 2007] K. Friston, J. Mattout, N. Trujillo- Barreto, J. Ashburner, and W. Penny. Variational free en- ergy and the Laplace approximation. Neuroimage, 2007.","title":"Variational free energy and the Laplace approximation","authors":"K Friston, J Mattout, N Trujillo-Barreto, J Ashburner, W Penny","year":"2007","venue":"Neuroimage","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Friston et al., 2009] K. Friston, J. Daunizeau, S. Kiebel. Re- inforcement learning or active inference? PloS One, 2009.","title":"Reinforcement learning or active inference?","authors":"K Friston, J Daunizeau, S Kiebel","year":"2009","venue":"PloS One","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Friston et al., 2017a] K. Friston, T. FitzGerald, F. Rigoli, P. Schwartenbeck, and G. Pezzulo. Active inference: A process theory. Neural Comput., 2017.","title":"Active inference: A process theory","authors":"Friston","year":"2017","venue":"Neural Comput","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Friston et al., 2017b] K. J. Friston, T. Parr, and B. de Vries. The graphical brain: Belief propagation and active infer- ence. Netw. Neurosci., 2017.","title":"The graphical brain: Belief propagation and active inference","authors":"Friston","year":"2017","venue":"Netw. Neurosci","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Friston, 2003] K. Friston. Learning and inference in the brain. Neural Netw., 2003.","title":"Learning and inference in the brain","authors":"K Friston","year":"2003","venue":"Neural Netw","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Friston, 2005] K. Friston. A theory of cortical responses. Philos. Trans. R. Soc. Lond., B, Biol. Sci., 2005.","title":"A theory of cortical responses","authors":"K Friston","year":"2005","venue":"Philos. Trans. R. Soc. Lond., B, Biol. Sci","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Friston, 2008] K. Friston. Hierarchical models in the brain. PLoS Comput. Biol., 2008.","title":"Hierarchical models in the brain","authors":"K Friston","year":"2008","venue":"PLoS Comput. Biol","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Friston, 2011] K. Friston. What is optimal about motor con- trol? Neuron, 2011.","title":"What is optimal about motor control?","authors":"K Friston","year":"2011","venue":"Neuron","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Han et al., 2018] K. Han, H. Wen, Y. Zhang, D. Fu, E. Cu- lurciello, and Z. Liu. Deep predictive coding network with local recurrent processing for object recognition. Proc. NeurIPS, 2018.","title":"Deep predictive coding network with local recurrent processing for object recognition","authors":"K Han, H Wen, Y Zhang, D Fu, E Culurciello, Z Liu","year":"2018","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[He et al., 2016] K. He, X. Zhang, S. Ren, J. Sun. Deep resi- dual learning for image recognition. In Proc. CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"K He, X Zhang, S Ren, J Sun","year":"2016","venue":"Proc. CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Hinton et al., 2012] G. E. Hinton, N. Srivastava, A. Kri- zhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature de- tectors. arXiv:1207.0580, 2012.","title":"Improving neural networks by preventing co-adaptation of feature detectors","authors":"G E Hinton, N Srivastava, A Krizhevsky, I Sutskever, R R Salakhutdinov","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Hinton, 2007] G. E. Hinton. To recognize shapes, first learn to generate images. Prog. Brain Res., 165:535-547, 2007.","title":"To recognize shapes, first learn to generate images","authors":"G E Hinton","year":"2007","venue":"Prog. Brain Res","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Hochreiter and Schmidhuber, 1997] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput., 1997.","title":"Long short-term memory","authors":"S Hochreiter, J Schmidhuber","year":"1997","venue":"Neural Comput","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Hohwy et al., 2008] J. Hohwy, A. Roepstorff, and K. Fris- ton. Predictive coding explains binocular rivalry: An epis- temological review. Cognition, 2008.","title":"Predictive coding explains binocular rivalry: An epistemological review","authors":"J Hohwy, A Roepstorff, K Friston","year":"2008","venue":"Cognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ioffe and Szegedy, 2015] S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by re- ducing internal covariate shift. In Proc. ICML, pages 448- 456. PMLR, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"S Ioffe, C Szegedy","year":"2015","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Johnson and Moradi, 2005] M. Johnson and M. Moradi. PID Control. Springer, 2005.","title":"PID Control","authors":"M Johnson, M Moradi","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Jordan et al., 1998] M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul. An introduction to variational methods for graphical models. In Learning in Graphical Models. 1998.","title":"An introduction to variational methods for graphical models","authors":"M Jordan, Z Ghahramani, T Jaakkola, L Saul","year":"1998","venue":"Learning in Graphical Models","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Kaplan et al., 2020] J. Kaplan, S. McCandlish, T Henighan, T. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv:2001.08361, 2020.","title":"Scaling laws for neural language models","authors":"J Kaplan, S Mccandlish, T Henighan, B Brown, R Chess, S Child, A Gray, J Radford, D Wu, Amodei","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Kinghorn et al., 2021] P. Kinghorn, B. Millidge, and C. Buckley. Habitual and reflective control in hierarchi- cal predictive coding. arXiv:2109.00866, 2021.","title":"Habitual and reflective control in hierarchical predictive coding","authors":"P Kinghorn, B Millidge, C Buckley","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Knill and Pouget, 2004] D. Knill and A. Pouget. The Bayesian brain: The role of uncertainty in neural coding and computation. Tends Neurosci., 2004.","title":"The Bayesian brain: The role of uncertainty in neural coding and computation","authors":"D Knill, A Pouget","year":"2004","venue":"Tends Neurosci","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Lanillos and Cheng, 2018] P. Lanillos and G. Cheng. Adap- tive robot body learning and estimation through predictive coding. In Proc. IROS, 2018.","title":"Adaptive robot body learning and estimation through predictive coding","authors":"P Lanillos, G Cheng","year":"2018","venue":"Proc. IROS","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Lanillos et al., 2021] P. Lanillos, C. Meo, C. Pezzato, et al. Active inference in robotics and artificial agents: Survey and challenges. arXiv:2112.01871, 2021.","title":"Active inference in robotics and artificial agents: Survey and challenges","authors":"P Lanillos, C Meo, C Pezzato","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Levine, 2018] S. Levine. Reinforcement learning and control as probabilistic inference: Tutorial and review. arXiv:1805.00909, 2018.","title":"Reinforcement learning and control as probabilistic inference: Tutorial and review","authors":"S Levine","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Lillicrap et al., 2020] T. Lillicrap, A. Santoro, L. Marris, C. Akerman, and G. Hinton. Backpropagation and the brain. Nat. Rev. Neurosci., 2020.","title":"Backpropagation and the brain","authors":"T Lillicrap, A Santoro, L Marris, C Akerman, G Hinton","year":"2020","venue":"Nat. Rev. Neurosci","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Lotter et al., 2016] W. Lotter, G. Kreiman, and D. Cox. Deep predictive coding networks for video prediction and unsupervised learning. arXiv:1605.08104, 2016.","title":"Deep predictive coding networks for video prediction and unsupervised learning","authors":"W Lotter, G Kreiman, D Cox","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Meera and Wisse, 2020] A. Meera and M. Wisse. Free en- ergy principle based state and input observer design for linear systems with colored noise. In Proc. ACC, 2020.","title":"Free energy principle based state and input observer design for linear systems with colored noise","authors":"A Meera, M Wisse","year":"2020","venue":"Proc. ACC","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Meera and Wisse, 2021] A. Meera and M. Wisse. A brain inspired learning algorithm for the perception of a quadro- tor in wind. arXiv:2109.11971, 2021.","title":"A brain inspired learning algorithm for the perception of a quadrotor in wind","authors":"A Meera, M Wisse","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Millidge et al., 2020a] B. Millidge, A. Tschantz, C. Buck- ley. Predictive coding approximates backprop along arbi- trary computation graphs. arXiv:2006.04182, 2020.","title":"Predictive coding approximates backprop along arbitrary computation graphs","authors":"Millidge","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Millidge et al., 2020b] B. Millidge, A. Tschantz, A. Seth, and C. Buckley. On the relationship between active in- ference and control as inference. In Proc. IWAI, 2020.","title":"On the relationship between active inference and control as inference","authors":"Millidge","year":"2020","venue":"Proc. IWAI","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Millidge et al., 2020c] B. Millidge, A. Tschantz, A. Seth, and C. Buckley. Relaxing the constraints on predictive coding models. arXiv:2010.01047, 2020.","title":"Relaxing the constraints on predictive coding models","authors":"Millidge","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Millidge et al., 2021] B. Millidge, A. Seth, and C. Buckley. Predictive coding: A theoretical and experimental review. arXiv:2107.12979, 2021.","title":"Predictive coding: A theoretical and experimental review","authors":"B Millidge, A Seth, C Buckley","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Movellan, 1991] J. R. Movellan. Contrastive Hebbian learn- ing in the continuous Hopfield model. In Connectionist Models, pages 10-17. Elsevier, 1991.","title":"Contrastive Hebbian learning in the continuous Hopfield model","authors":"J R Movellan","year":"1991","venue":"Connectionist Models","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Oliver et al., 2021] G. Oliver, P. Lanillos, and G. Cheng. An empirical study of active inference on a humanoid robot. IEEE Trans. Cogn. Dev. Syst., 2021.","title":"An empirical study of active inference on a humanoid robot","authors":"G Oliver, P Lanillos, G Cheng","year":"2021","venue":"IEEE Trans. Cogn. Dev. Syst","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Ororbia and Kifer, 2020] A. Ororbia and D. Kifer. The neural coding framework for learning generative models. arXiv:2012.03405, 2020.","title":"The neural coding framework for learning generative models","authors":"A Ororbia, D Kifer","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Ororbia and Mali, 2019] A. Ororbia and A. Mali. Biolog- ically motivated algorithms for propagating local target representations. In Proc. AAAI, 2019.","title":"Biologically motivated algorithms for propagating local target representations","authors":"A Ororbia, A Mali","year":"2019","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Papadimitriou et al., 2020] C.. Papadimitriou, S.. Vempala, D. Mitropolsky, M. Collins, and W. Maass. Brain compu- tation by assemblies of neurons. PNAS, 2020.","title":"Brain computation by assemblies of neurons","authors":"C Papadimitriou, S Vempala, D Mitropolsky, M Collins, W Maass","year":"2020","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Pezzato et al., 2020] C. Pezzato, R. Ferrari, and C. Corbato. A novel adaptive controller for robot manipulators based on active inference. IEEE Robot. Autom. Lett., 2020.","title":"A novel adaptive controller for robot manipulators based on active inference","authors":"C Pezzato, R Ferrari, C Corbato","year":"2020","venue":"IEEE Robot. Autom. Lett","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Radhakrishnan et al., 2019] A. Radhakrishnan, M. Belkin, and C. Uhler. Overparameterized neural networks imple- ment associative memory. arXiv:1909.12362, 2019.","title":"Overparameterized neural networks implement associative memory","authors":"A Radhakrishnan, M Belkin, C Uhler","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Ramsauer et al., 2021] H. Ramsauer, B. Schäfl, J. Lehner, et al. Hopfield networks is all you need. Proc. ICLR, 2021.","title":"Hopfield networks is all you need","authors":"H Ramsauer, B Schäfl, J Lehner","year":"2021","venue":"Proc. ICLR","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Rao and Ballard, 1999] R. Rao and D. Ballard. Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci., 1999.","title":"Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects","authors":"R Rao, D Ballard","year":"1999","venue":"Nat. Neurosci","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Rumelhart et al., 1986] D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back- propagating errors. Nature, 1986.","title":"Learning representations by backpropagating errors","authors":"D E Rumelhart, G E Hinton, R J Williams","year":"1986","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Salvatori et al., 2021a] T. Salvatori, Y. Song, Y. Hong, L. Sha, S. Frieder, Z. Xu, R. Bogacz, and T. Lukasiewicz. Associative memories via predictive coding. In Proc. NeurIPS, 2021.","title":"Associative memories via predictive coding","authors":"Salvatori","year":"2021","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Salvatori et al., 2021b] T. Salvatori, Y. Song, T. Luka- siewicz, R. Bogacz, and Z. Xu. Predictive coding can do exact backpropagation on any neural network. arXiv:2103.04689, 2021.","title":"Predictive coding can do exact backpropagation on any neural network","authors":"Salvatori","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Salvatori et al., 2022] T. Salvatori, L. Pinchetti, B. Millidge, Y. Song, T. Bao, R. Bogacz, and T. Lukasiewicz. Learn- ing on arbitrary graph topologies via predictive coding. arXiv:2201.13180, 2022.","title":"Learning on arbitrary graph topologies via predictive coding","authors":"T Salvatori, L Pinchetti, B Millidge, Y Song, T Bao, R Bogacz, T Lukasiewicz","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Sancaktar et al., 2020] C. Sancaktar, M. van Gerven, P. La- nillos. End-to-end pixel-based deep active inference for body perception and action. In Proc. ICDL-EpiRob, 2020.","title":"End-to-end pixel-based deep active inference for body perception and action","authors":"C Sancaktar, M Van Gerven, P Lanillos","year":"2020","venue":"Proc. ICDL-EpiRob","doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Song et al., 2020] Y. Song, T. Lukasiewicz, Z. Xu, and R. Bogacz. Can the brain do backpropagation? -Exact implementation of backpropagation in predictive coding networks. In Proc. NeurIPS, 2020.","title":"Can the brain do backpropagation? -Exact implementation of backpropagation in predictive coding networks","authors":"Y Song, T Lukasiewicz, Z Xu, R Bogacz","year":"2020","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Spratling, 2017] M. W. Spratling. A review of predictive co- ding algorithms. Brain Cogn., 2017.","title":"A review of predictive coding algorithms","authors":"M W Spratling","year":"2017","venue":"Brain Cogn","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Srinivasan et al., 1982] M. Srinivasan, S. Laughlin, and A. Dubs. Predictive coding: A fresh view of inhibition in the retina. Proc. R. Soc. B: Biol. Sci., 1982.","title":"Predictive coding: A fresh view of inhibition in the retina","authors":"M Srinivasan, S Laughlin, A Dubs","year":"1982","venue":"Proc. R. Soc. B: Biol. Sci","doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Srivastava et al., 2014] N. Srivastava, G. Hinton, A. Kri- zhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 15(1):1929-1958, 2014.","title":"Dropout: A simple way to prevent neural networks from overfitting","authors":"N Srivastava, G Hinton, A Krizhevsky, I Sutskever, R Salakhutdinov","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Sun and Orchard, 2020] W. Sun and J. Orchard. A predicti- ve-coding network that is both discriminative and genera- tive. Neural Comput., 2020.","title":"A predictive-coding network that is both discriminative and generative","authors":"W Sun, J Orchard","year":"2020","venue":"Neural Comput","doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Toussaint and Storkey, 2006] M. Toussaint and A. Storkey. Probabilistic inference for solving discrete and continuous state Markov decision processes. In Proc. ICML, 2006.","title":"Probabilistic inference for solving discrete and continuous state Markov decision processes","authors":"M Toussaint, A Storkey","year":"2006","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Vaswani et al., 2017] A. Vaswani, N. Shazeer, N. Parmar, et al. Attention is all you need. In Proc. NIPS, 2017.","title":"Attention is all you need","authors":"A Vaswani, N Shazeer, N Parmar","year":"2017","venue":"Proc. NIPS","doi":null,"url":null,"internal_url":null},{"position":66,"raw":"[Vincent et al., 2010] P. Vincent, H. Larochelle, I. Lajoie, et al. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. JMLR, 11(12), 2010.","title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","authors":"P Vincent, H Larochelle, I Lajoie","year":"2010","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":67,"raw":"[Wainwright and Jordan, 2008] M. Wainwright and M. Irwin Jordan. Graphical Models, Exponential Families, and Variational Inference. Now Publishers Inc, 2008.","title":"Graphical Models, Exponential Families, and Variational Inference","authors":"M Wainwright, M Irwin Jordan","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":68,"raw":"[Whittington and Bogacz, 2017] J. C. R. Whittington and R. Bogacz. An approximation of the error backpropaga- tion algorithm in a predictive coding network with local Hebbian synaptic plasticity. Neural Comput., 2017.","title":"An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity","authors":"J C R Whittington, R Bogacz","year":"2017","venue":"Neural Comput","doi":null,"url":null,"internal_url":null},{"position":69,"raw":"[Williams and Zipser, 1989] R. J. Williams and D. Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural Comput., 1(2):270-280, 1989.","title":"A learning algorithm for continually running fully recurrent neural networks","authors":"R J Williams, D Zipser","year":"1989","venue":"Neural Comput","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}