Dependency Relationship Based Decision Combination in Multiple Classifier Systems

Abstract

Although many decision combination methods have been proposed, most of them did not focus on dependency relationship among classifiers in combining multiple decisions. That makes classification performance of combining multiple decisions be degraded and biased, in case of adding highly dependent inferior classifiers. To overcome such weaknesses and obtain robust classification performance, the present study used dependency relationship for better combining multiple decisions. In order to identify dependency relationship by observing outputs of multiple classifiers, two methods are used on the basis of first-order dependency relationship. One is to use the concept of mutual information, and the other one is to use the concept of statistically measured association. The first-order dependencies identified are used to combine multiple decisions, using Bayesian formalism. A number of multiple classifier systems are applied to totally uncontrained on-line handwritten numerals and the English alphabet recognition. The experimental results show that the classification performance of a multiple classifier system is superior to that of indiyidual classifiers. Also, they show that considering the dependency relationship outperforms others in accuracy, when the highly dependent inferior classifiers are added.