{"type":"ScholarlyArticle","title":"Towards Architecture-Agnostic Neural Transfer: a Knowledge-Enhanced Approach","authors":[{"name":"Seán Quinn","slug":"sean-quinn","given":"Seán","family":"Quinn","affiliation":"Insight Centre for Data Analytics , Dublin City University"},{"name":"Alessandra Mileo","slug":"alessandra-mileo","given":"Alessandra","family":"Mileo","affiliation":"Insight Centre for Data Analytics , Dublin City University"}],"abstract":"The ability to enhance deep representations with prior knowledge is receiving a lot of attention from the AI community as a key enabler to improve the way modern Artificial Neural Networks (ANN) learn. In this paper we introduce our approach to this task, which comprises of a knowledge extraction algorithm, a knowledge injection algorithm and a common intermediate knowledge representation as an alternative to traditional neural transfer. As a result of this research, we envisage a knowledge-enhanced ANN, which will be able to learn, characterise and reuse knowledge extracted from the learning process, thus enabling more robust architecture-agnostic neural transfer, greater explainability and further integration of neural and symbolic approaches to learning.","doi":"10.24963/ijcai.2019/915","url":"https://ijcai.org/proceedings/2019/915","pdf":"https://ijcai.org/proceedings/2019/0915.pdf","markdown":"https://ijcai.org/proceedings/2019/915.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/915","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":915,"pages":{"first":6452,"last":6453},"keywords":["Machine Learning: Knowledge-based Learning","Machine Learning: Transfer, Adaptation, Multi-task Learning","Computer Vision: Structural and Model-Based Approaches, Knowledge Representation and Reasoning","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Bau et al., 2017] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Computer Vision and Pattern Recognition, 2017.","title":"Network dissection: Quantifying interpretability of deep visual representations","authors":"David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, Antonio Torralba","year":"2017","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Besold et al., 2017] Tarek R Besold, Artur d'Avila Garcez, Sebastian Bader, Howard Bowman, Pedro Domingos, Pas- cal Hitzler, Kai-Uwe Kühnberger, Luis C Lamb, Daniel Lowd, Priscila Machado Vieira Lima, et al. Neural- symbolic learning and reasoning: A survey and interpre- tation. arXiv preprint arXiv:1711.03902, 2017.","title":"Neuralsymbolic learning and reasoning: A survey and interpretation","authors":"Artur D'avila Tarek R Besold, Sebastian Garcez, Howard Bader, Pedro Bowman, Pascal Domingos, Kai-Uwe Hitzler, Luis C Kühnberger, Daniel Lamb, Priscila Machado Lowd, Vieira Lima","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Garcez et al., 2001] AS d'Avila Garcez, Krysia Broda, and Dov M Gabbay. Symbolic knowledge extraction from trained neural networks: A sound approach. Artificial In- telligence, 125(1-2):155-207, 2001.","title":"Symbolic knowledge extraction from trained neural networks: A sound approach","authors":"Garcez","year":"2001","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Kumar, 2012] Koushal Kumar. Knowledge extraction from trained neural networks. International Journal of Informa- tion and Network Security, 1(4):282, 2012.","title":"Knowledge extraction from trained neural networks","authors":"Koushal Kumar, Kumar","year":"2012","venue":"International Journal of Information and Network Security","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Lake et al., 2017] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences, 40, 2017.","title":"Building machines that learn and think like people","authors":"Brenden M Lake, Joshua B Tomer D Ullman, Samuel J Tenenbaum, Gershman","year":"2017","venue":"Behavioral and Brain Sciences","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey Hinton. Deep learning. nature, 521(7553):436, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Maclin and Shavlik, 1996] Richard Maclin and Jude W Shavlik. Creating advice-taking reinforcement learners. Machine Learning, 22(1-3):251-281, 1996.","title":"Creating advice-taking reinforcement learners","authors":"Shavlik Maclin, Richard Maclin, Jude W Shavlik","year":"1996","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Sourek et al., 2015] Gustav Sourek, Vojtech Aschenbren- ner, Filip Zelezny, and Ondrej Kuzelka. Lifted relational neural networks. arXiv preprint arXiv:1508.05128, 2015.","title":"Gustav Sourek, Vojtech Aschenbrenner","authors":"Sourek","year":"2015","venue":"Lifted relational neural networks","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Tran and Garcez, 2018] Son N Tran and Artur S d'Avila Garcez. Deep logic networks: Inserting and extracting knowledge from deep belief networks. IEEE transactions on neural networks and learning systems, 29(2):246-258, 2018.","title":"Deep logic networks: Inserting and extracting knowledge from deep belief networks","authors":"Tran, Garcez ; ] Son, N Tran, Artur S D'avila Garcez","year":"2018","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}