Combinatorial Routing for Neural Trees

Combinatorial Routing for Neural Trees

Jiahao Li, Ruichu Cai, Yuguang Yan

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence
Main Track. Pages 4407-4415. https://doi.org/10.24963/ijcai.2024/487

Neural trees benefit from the high-level representation of neural networks and the interpretability of decision trees. Therefore, the existing works on neural trees perform outstandingly on various tasks such as architecture search. However, these works require every router to provide only one successor for each sample, causing the predictions to be dominated by the elite branch and its derivative architectures. To break this branch dominance, we propose the combinatorial routing neural tree method, termed CombRo. Unlike the previous methods employing unicast routing, CombRo performs multicast schema in each iteration, allowing the features to be routed to any combination of successors at every non-leaf. The weights of each architecture are then evaluated accordingly. We update the weights by training the routing subnetwork, and the architecture with the top weight is selected in the final step. We compare CombRo with the existing algorithms on 3 public image datasets, demonstrating its superior performance in terms of accuracy. Visualization results further validate the effectiveness of the multicast routing schema. Code is available at https://github.com/JiahaoLi-gdut/CombRo.
Keywords:
Machine Learning: ML: Deep learning architectures
Computer Vision: CV: Machine learning for vision
Machine Learning: ML: Classification
Machine Learning: ML: Ensemble methods