{"type":"ScholarlyArticle","title":"Making Sense of Raw Input (Extended Abstract)","authors":[{"name":"Richard Evans","slug":"richard-evans","given":"Richard","family":"Evans","affiliation":""},{"name":"Matko Bošnjak","slug":"matko-bosnjak","given":"Matko","family":"Bošnjak","affiliation":""},{"name":"Lars Buesing","slug":"lars-buesing","given":"Lars","family":"Buesing","affiliation":""},{"name":"Kevin Ellis","slug":"kevin-ellis","given":"Kevin","family":"Ellis","affiliation":"Massachusetts Institute of Technology"},{"name":"David Pfau","slug":"david-pfau","given":"David","family":"Pfau","affiliation":""},{"name":"Pushmeet Kohli","slug":"pushmeet-kohli","given":"Pushmeet","family":"Kohli","affiliation":""},{"name":"Marek Sergot","slug":"marek-sergot","given":"Marek","family":"Sergot","affiliation":"Imperial College London"}],"abstract":"How should a machine intelligence perform unsupervised structure discovery over streams of sensory input? One approach to this problem is to cast it as an apperception task. Here, the task is to construct an explicit interpretable theory that both explains the sensory sequence and also satisfies a set of  unity conditions, designed to ensure that the constituents of the theory are connected in a relational structure. \r\n\r\nHowever, the original formulation of the apperception task had one fundamental limitation: it assumed the raw sensory input had already been parsed using a set of discrete categories, so that all the system had to do was receive this already-digested symbolic input, and make sense of it. But what if we don't have access to pre-parsed input? What if our sensory sequence is raw unprocessed information?\r\n\r\nThe central contribution of this paper is a neuro-symbolic framework for distilling interpretable theories out of streams of raw, unprocessed sensory experience.\r\nFirst, we extend the definition of the apperception task to include ambiguous (but still symbolic) input: sequences of sets of disjunctions. Next, we use a neural network to map raw sensory input to disjunctive input. Our binary neural network is encoded as a logic program, so the weights of the network and the rules of the theory can be solved jointly as a single SAT problem. This way, we are able to jointly learn how to perceive (mapping raw sensory information to concepts) and apperceive (combining concepts into declarative rules).","doi":"10.24963/ijcai.2022/799","url":"https://ijcai.org/proceedings/2022/799","pdf":"https://ijcai.org/proceedings/2022/0799.pdf","markdown":"https://ijcai.org/proceedings/2022/799.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/799","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Journal Track","track":"Journal Track","number":799,"pages":{"first":5727,"last":5731},"keywords":["Machine Learning: Explainable/Interpretable Machine Learning","Knowledge Representation and Reasoning: Learning and reasoning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987245","length_sec":89,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985388","length_sec":682,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Buesing et al., 2018] Lars Buesing, Theophane Weber, Se- bastien Racaniere, SM Eslami, Danilo Rezende, David P Reichert, Fabio Viola, Frederic Besse, Karol Gregor, Demis Hassabis, et al. Learning and querying fast gen- erative models for reinforcement learning. arXiv preprint arXiv:1802.03006, 2018.","title":"Learning and querying fast generative models for reinforcement learning","authors":"Buesing","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chalmers et al., 1992] David J Chalmers, Robert M French, and Douglas R Hofstadter. High-level perception, repre- sentation, and analogy: A critique of artificial intelligence methodology. Journal of Experimental & Theoretical Ar- tificial Intelligence, 4(3):185-211, 1992.","title":"High-level perception, representation, and analogy: A critique of artificial intelligence methodology","authors":"J David, Robert M Chalmers, Douglas R French, Hofstadter","year":"1992","venue":"Journal of Experimental & Theoretical Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Clark, 2013] Andy Clark. Whatever next? predictive brains, situated agents, and the future of cognitive science. Behav- ioral and brain sciences, 36(3):181-204, 2013.","title":"Whatever next? predictive brains, situated agents, and the future of cognitive science","authors":"Andy Clark","year":"2013","venue":"Behavioral and brain sciences","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cropper et al., 2019] Andrew Cropper, Richard Evans, and Mark Law. Inductive general game playing. arXiv preprint arXiv:1906.09627, 2019.","title":"Inductive general game playing","authors":"Andrew Cropper, Richard Evans, Mark Law","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ellis et al., 2015] Kevin Ellis, Armando Solar-Lezama, and Josh Tenenbaum. Unsupervised learning by program syn- thesis. In Advances in Neural Information Processing Sys- tems, pages 973-981, 2015.","title":"Unsupervised learning by program syn-thesis","authors":"Kevin Ellis, Armando Solar-Lezama, Josh Tenenbaum","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Evans and Grefenstette, 2018] Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1-64, 2018.","title":"Learning explanatory rules from noisy data","authors":"Grefenstette Evans, Richard Evans, Edward Grefenstette","year":"2018","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Evans et al., 2021] Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, and Marek Sergot. Making sense of sensory input. Artificial Intelligence, 293:103438, 2021.","title":"Making sense of sensory input","authors":"Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, Marek Sergot","year":"2021","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodman et al., 2011] Noah D Goodman, Tomer D Ull- man, and Joshua B Tenenbaum. Learning a theory of causality. Psychological review, 118(1):110, 2011.","title":"Learning a theory of causality","authors":"Noah D Goodman, Joshua B Tomer D Ullman, Tenenbaum","year":"2011","venue":"Psychological review","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ha and Schmidhuber, 2018] David Ha and Jürgen Schmid- huber. World models. arXiv preprint arXiv:1803.10122, 2018.","title":"World models","authors":"David Ha, Jürgen Schmidhuber","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hofstadter, 1995] Douglas R Hofstadter. Fluid Concepts and Creative Analogies. Basic Books, 1995.","title":"Fluid Concepts and Creative Analogies","authors":"Douglas R Hofstadter","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Inoue et al., 2005] Katsumi Inoue, Hideyuki Bando, and Hidetomo Nabeshima. Inducing causal laws by regular inference. In International Conference on Inductive Logic Programming, pages 154-171. Springer, 2005.","title":"Inducing causal laws by regular inference","authors":"Katsumi Inoue, Hideyuki Bando, Hidetomo Nabeshima","year":"2005","venue":"International Conference on Inductive Logic Programming","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Inoue et al., 2014] Katsumi Inoue, Tony Ribeiro, and Chi- aki Sakama. Learning from interpretation transition. Ma- chine Learning, 94(1):51-79, 2014.","title":"Learning from interpretation transition","authors":"Katsumi Inoue, Tony Ribeiro, Chiaki Sakama","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Katzouris et al., 2015] Nikos Katzouris, Alexander Artikis, and Georgios Paliouras. Incremental learning of event definitions with inductive logic programming. Machine Learning, 100(2-3):555-585, 2015.","title":"Incremental learning of event definitions with inductive logic programming","authors":"Nikos Katzouris, Alexander Artikis, Georgios Paliouras","year":"2015","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li and Vitányi, 2008] Ming Li and Paul Vitányi. An intro- duction to Kolmogorov complexity and its applications, volume 3. Springer, 2008.","title":"An introduction to Kolmogorov complexity and its applications","authors":"Ming Li, Paul Vitányi","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Miller, 2018] Tim Miller. Explanation in artificial intelli- gence: Insights from the social sciences. Artificial Intelli- gence, 2018.","title":"Explanation in artificial intelligence: Insights from the social sciences","authors":"Tim Miller, Miller","year":"2018","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mitchell, 1993] Melanie Mitchell. Analogy-making as per- ception: A computer model. MIT Press, 1993.","title":"Analogy-making as perception: A computer model","authors":"Melanie Mitchell","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Murdoch et al., 2019] W James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. Interpretable machine learning: definitions, methods, and applications. PNAS, 2019.","title":"Interpretable machine learning: definitions, methods, and applications","authors":"James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, Bin Yu","year":"2019","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pathak et al., 2017] Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 16-17, 2017.","title":"Curiosity-driven exploration by self-supervised prediction","authors":"Deepak Pathak, Pulkit Agrawal, Alexei A Efros, Trevor Darrell","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ray, 2009] Oliver Ray. Nonmonotonic abductive inductive learning. Journal of Applied Logic, 7(3):329-340, 2009.","title":"Nonmonotonic abductive inductive learning","authors":"Oliver Ray, Ray","year":"2009","venue":"Journal of Applied Logic","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rudin, 2018] Cynthia Rudin. Please stop explaining black box models for high stakes decisions. arXiv preprint arXiv:1811.10154, 2018.","title":"Please stop explaining black box models for high stakes decisions","authors":"Cynthia Rudin","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sermanet et al., 2018] Pierre Sermanet, Corey Lynch, Yev- gen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, and Sergey Levine. Time-contrastive networks: Self- supervised learning from pixels. In 2018 IEEE Interna- tional Conference on Robotics and Automation (ICRA), pages 1134-1141. IEEE, 2018.","title":"Time-contrastive networks: Selfsupervised learning from pixels","authors":"Sermanet","year":"2018","venue":"2018 IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shanahan et al., 2019] Murray Shanahan, Kyriacos Niki- forou, Antonia Creswell, Christos Kaplanis, David Barrett, and Marta Garnelo. An explicitly relational neural network architecture. arXiv preprint arXiv:1905.10307, 2019.","title":"An explicitly relational neural network architecture","authors":"Murray Shanahan, Kyriacos Nikiforou, Antonia Creswell, Christos Kaplanis, David Barrett, Marta Garnelo","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shanahan, 2005] Murray Shanahan. Perception as abduc- tion. Cognitive Science, 29(1):103-134, 2005.","title":"Perception as abduction","authors":"Murray Shanahan, Shanahan","year":"2005","venue":"Cognitive Science","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}