Using a Hierarchical Bayesian Model to handle High Cardinality Attributes with Relevant Interactions in a Classification Problem

Jorge Jambeiro Filho, Jacques Wainer

We employed a multilevel hierarchical Bayesian model in the task of exploiting relevant interactions among high cardinality attributes in a classification problem without overfitting. With this model, we calculate posterior class probabilities for a pattern W combining the observations of W in the training set with prior class probabilities that are obtained recursively from the observations of patterns that are strictly more generic than W. The model achieved performance improvements over standard Bayesian network methods like Naive Bayes and Tree Augmented Naive Bayes, over Bayesian Networks where traditional conditional probability tables were substituted by Noisy-or gates, Default Tables, Decision Trees and Decision Graphs, and over Bayesian Networks constructed after a cardinality reduction preprocessing phase using the Agglomerative Information Bottleneck method.