A Fusion of Stacking with Dynamic Integration

Niall Rooney, David Patterson

In this paper we present a novel method that fuses the ensemble meta-techniques of Stack-ing and Dynamic Integration (DI) for regres-sion problems, without adding any major computational overhead. The intention of the technique is to benefit from the varying per-formance of Stacking and DI for different data sets, in order to provide a more robust technique. We detail an empirical analysis of the technique referred to as weighted Meta-Combiner (wMetaComb) and compare its per-formance to Stacking and the DI technique of Dynamic Weighting with Selection. The em-pirical analysis consisted of four sets of ex-periments where each experiment recorded the cross-fold evaluation of each technique for a large number of diverse data sets, where each base model is created using random fea-ture selection and the same base learning al-gorithm. Each experiment differed in terms of the latter base learning algorithm used. We demonstrate that for each evaluation, wMeta-Comb was able to outperform DI and Stack-ing for each experiment and as such fuses the two underlying mechanisms successfully