A Theoretical Framework for Learning Bayesian Networks with Parameter Inequality Constraints

Radu Stefan Niculescu, Tom M. Mitchell, R. Bharat Rao

The task of learning models for many real-world problems requires incorporating domain knowledge into learning algorithms, to enable accurate learning from a realistic volume of training data. Domain knowledge can come in many forms. For example, expert knowledge about the relevance of variables relative to a certain problem can help perform better feature selection. Domain knowledge about the conditional independence relationships among variables can help learning of the Bayesian Network structure. This paper considers a different type of domain knowledge for constraining parameter estimates when learning Bayesian Networks. In particular, we consider domain knowledge that comes in the form of inequality constraints among subsets of parameters in a Bayesian Network with known structure. These parameter constraints are incorporated into learning procedures for Bayesian Networks, by formulating this task as a constrained optimization problem. The main contribution of this paper is the derivation of closed form Maximum Likelihood parameter estimators in the above setting.