{"type":"ScholarlyArticle","title":"Evaluating Relaxations of Logic for Neural Networks: A Comprehensive Study","authors":[{"name":"Mattia Medina Grespan","slug":"mattia-medina-grespan","given":"Mattia","family":"Medina Grespan","affiliation":""},{"name":"Ashim Gupta","slug":"ashim-gupta","given":"Ashim","family":"Gupta","affiliation":""},{"name":"Vivek Srikumar","slug":"vivek-srikumar","given":"Vivek","family":"Srikumar","affiliation":""}],"abstract":"Symbolic knowledge can provide crucial inductive bias for training neural models, especially in low data regimes. A successful strategy for incorporating such knowledge involves relaxing logical statements into sub-differentiable losses for optimization. In this paper, we study the question of how best to relax logical expressions that represent labeled examples and knowledge about a problem; we focus on sub-differentiable t-norm relaxations of logic. We present theoretical and empirical criteria for characterizing which relaxation would perform best in various scenarios. In our theoretical study driven by the goal of preserving tautologies, the Lukasiewicz t-norm performs best. However, in our empirical analysis on the text chunking and digit recognition tasks, the product t-norm achieves best predictive performance. We analyze this apparent discrepancy, and conclude with a list of best practices for defining loss functions via logic.","doi":"10.24963/ijcai.2021/387","url":"https://ijcai.org/proceedings/2021/387","pdf":"https://ijcai.org/proceedings/2021/0387.pdf","markdown":"https://ijcai.org/proceedings/2021/387.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/387","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Machine Learning","track":"Main Track","number":387,"pages":{"first":2812,"last":2818},"keywords":["Machine Learning: Neuro-Symbolic Methods","Machine Learning: Knowledge Aided Learning","Machine Learning: Deep Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}