---
title: "Combining the Predictions of Multiple Classifiers: Using Competitive Learning to Initialize Neural Networks"
authors:
  - "Richard Maclin"
  - "Jude W. Shavlik"
doi: ""
url: "https://ijcai.org/proceedings/1995/69"
pdf: "https://ijcai.org/proceedings/1995/0069.pdf"
venue: "Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence"
year: 1995
pages: "524-530"
track: "Connectionist Models"
keywords:
---

# Combining the Predictions of Multiple Classifiers: Using Competitive Learning to Initialize Neural Networks

Richard Maclin, Jude W. Shavlik

*Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence*, pages 524-530.

## Abstract

The primary goal of inductive learning is to generalize well - that is, induce a function that accurately produces the correct output for future inputs. Hansen and Salamon showed that ,_ u~der certain assumptions, combining the predrctwns of several separately trained neural networks will improve generalization . One of their key assumptions is that the individual networks should be independent in the errors they produce. In the standard way of perform_ing backpropagation this assumption may be vrolated, because the standard procedure is to _initialize network weights in the region of werght space near the origin. This means that backpropagation's gradient-descent search may only reach a small subset of the possible local minima . In this paper we present an approach to initializing neural networks that uses competitive learning to intelligently create networks that are originally located far from the origin of weight space, thereby potentially increasing the set of reachable local minima. We report experiments on two real-world datasets where combinations of networks initialized with our method generalize better than combinations of networks initialized the traditional way.