Tessellation-Filtering ReLU Neural Networks

Tessellation-Filtering ReLU Neural Networks

Bernhard A. Moser, Michal Lewandowski, Somayeh Kargaran, Werner Zellinger, Battista Biggio, Christoph Koutschan

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 3335-3341. https://doi.org/10.24963/ijcai.2022/463

We identify tessellation-filtering ReLU neural networks that, when composed with another ReLU network, keep its non-redundant tessellation unchanged or reduce it.The additional network complexity modifies the shape of the decision surface without increasing the number of linear regions. We provide a mathematical understanding of the related additional expressiveness by means of a novel measure of shape complexity by counting deviations from convexity which results in a Boolean algebraic characterization of this special class. A local representation theorem gives rise to novel approaches for pruning and decision surface analysis.
Keywords:
Machine Learning: Theory of Deep Learning
Machine Learning: Learning Theory