Proceedings

TabKD: Tabular Knowledge Distillation Through Interaction Diversity of Learned Feature Bins

Shovon Niverd Pereira, Krishna Khadka, Yu Lei

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 4723-4731.

https://doi.org/10.24963/ijcai.2026/526

Keywords

  • Machine Learning: Knowledge-aided learning
  • Machine Learning: Adversarial machine learning

Abstract

Data-free knowledge distillation enables model distillation without original training data, which is critical for privacy-sensitive tabular domains. However, existing methods do not perform well on tabular data because they do not explicitly address feature interactions, which are critical for encoding predictive knowledge. We identify interaction diversity, systematic coverage of feature combinations, as an important factor for effective tabular model distillation. To operationalize this insight, we propose TabKD, which learns adaptive feature bins aligned with teacher decision boundaries, then generates synthetic queries that ensure uniform pairwise interaction coverage. Across 4 benchmark datasets and 4 teacher architectures, TabKD achieves the highest student-teacher agreement in 14 of 16 configurations, outperforming 5 state-of-the-art baselines. We further show that interaction coverage strongly correlates with distillation quality, validating our core hypothesis. Our work establishes interaction-focused exploration as a principled framework for tabular model distillation.

Shovon Niverd Pereira, Krishna Khadka, and Yu Lei. 2026. TabKD: Tabular Knowledge Distillation Through Interaction Diversity of Learned Feature Bins. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 4723–4731. https://doi.org/10.24963/ijcai.2026/526
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards