{"type":"ScholarlyArticle","title":"Uncertainty-aware Binary Neural Networks","authors":[{"name":"Junhe Zhao","slug":"junhe-zhao","given":"Junhe","family":"Zhao","affiliation":""},{"name":"Linlin Yang","slug":"linlin-yang","given":"Linlin","family":"Yang","affiliation":""},{"name":"Baochang Zhang","slug":"baochang-zhang","given":"Baochang","family":"Zhang","affiliation":""},{"name":"Guodong Guo","slug":"guodong-guo","given":"Guodong","family":"Guo","affiliation":""},{"name":"David Doermann","slug":"david-doermann","given":"David","family":"Doermann","affiliation":""}],"abstract":"Binary Neural Networks (BNN) are promising machine learning solutions for deployment on resource-limited devices.  Recent approaches to training BNNs have produced impressive results, but minimizing the drop in accuracy from full precision networks is still challenging. One reason is that conventional BNNs ignore the uncertainty caused by weights that are near zero, resulting in the instability or frequent flip while learning. In this work, we investigate the intrinsic uncertainty of vanishing near-zero weights, making the training vulnerable to instability. We introduce an uncertainty-aware  BNN (UaBNN) by leveraging  a new mapping function called certainty-sign (c-sign)  to reduce these weights' uncertainties. Our c-sign function is the first to train BNNs with a  decreasing uncertainty for binarization.  The approach leads to a  controlled  learning process for BNNs. We also introduce a simple but effective method to measure the uncertainty-based on a Gaussian function. Extensive experiments demonstrate that our method improves multiple BNN methods by maintaining stability  of  training, and achieves a higher performance over prior arts.","doi":"10.24963/ijcai.2021/474","url":"https://ijcai.org/proceedings/2021/474","pdf":"https://ijcai.org/proceedings/2021/0474.pdf","markdown":"https://ijcai.org/proceedings/2021/474.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/474","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Machine Learning","track":"Main Track","number":474,"pages":{"first":3441,"last":3447},"keywords":["Machine Learning: Classification","Machine Learning: Deep Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}