Learning discontinuities for switching between local models

1 Introduction

Locally weighted learning techniques, in particular LWPR [Vijayakumar et al., 2002], have successfully been used for high-dimensional regression problems. Their robustness and efficient online versions are crucial in robotic domains where, for instance, an inverse model of an articulated dynamic robot has to be learned in real-time. Such models map a highdimensional state (e.g., joint angles and velocities) and a desired change of state to the required motor signals (torques).

While typically such mappings are assumed to be smooth, in real world scenarios, there are many interesting cases where the functions of interest are truly discontinuous. Some examples include contacts with other objects (in particular the ground), with other parts of the body, or with "joint limits". In fact, many interesting interactions with the environment manifest themselves through discontinuities in the sensorimotor data.

In this paper, we show how discontinuous switching between local regression models can be learned. The general topic of switching models has been discussed before, e.g., in the context of state space models [Ghahramani and Hinton, 1998; Pavlovic et al., 2000] or multiple inverse models [Wolpert and Kawato, 1998]. Generally, the question of which particular model receives responsibilities for a given input can be modeled as a hidden variable i in a generative mixture model. In our case, we assume that the responsibility index i can be predicted from the input (the robot state). Thus, inferring a model for i corresponds to classifying the input domain into regions for each sub-model.

Since in robotic domains, local learning is crucial to prevent interference and allow for online adaptation techniques, we propose a model of the responsibility index i which is itself a composition of local classifiers. Multiple pairwise classifiers are concatenated to construct a complete model in the form of a product-of-sigmoids, which is capable of learning complex, sharply bounded domains for each local model in lieu of the typical Gaussian kernels.

2 Learning a family of models

Given training data {(xk, yk)}Mk=1 with inputs xk and outputs yk, the first level goal of our algorithm is to learn a family of models {φ1, .., φN} such that every datum can be explained This general scheme of family learning can be realized with any type of models φi. In the experiments, we will choose φi to be linear functions, learned with Partial Least Squares (PLS) regression. PLS, involving an intermediate lower-dimensional projection, has been proven efficient for high-dimensional problems [Vijayakumar et al., 2002].

∗The first author acknowledges support by the German Research Foundation (DFG), Emmy Noether fellowship TO 409/1-1.

3 Products of sigmoids for switching

On the second level of our algorithm, the goal is to learn a predictive model P(i|x) of the latent responsibility index i that is more precise than the uninformed prior Ploc(i|x). Given some data, it is easy to decide whether two models are "potentially neighbored"-namely whether there exists data for which both models are eligible-based on their centers. For each pair (ij) of neighbored models, we learn a sigmoidal function ψij(x), where ψij ≡ 1 - ψji. The product of such sigmoids around a submodel i defines a coefficient βi(x) that we associate with the submodel for a given input x,

Here, Z ′ normalizes βi over i. As indicated, we represent sigmoids ψij with a scalar function φij. Fig. 1 illustrates the kind of kernels can be represented as products of sigmoids.

The sigmoids ψij(x) are meant to represent the likelihood that a model i rather than j is responsible for an input x, conditioned on that either i or j is responsible. The product combination is comparable to an AND voting. The MAP labeling ˆi we introduced in the previous section is now used to train these sigmoids. In the experiments, we consider φij to be linear functions, again learned with PLS.

4 Experiments

We tested the algorithm on piecewise linear, discontinuous test functions. A test function has 3 parameters: the input dimension d, the number L of linear pieces it is composed of and the output noise σ. The localities, slopes and boundaries of the linear pieces are sampled randomly. Fig. 2(a,b) display learning results from a 1D example in comparison to LWPR. Fig. 2(c,d) display two error curves on 10-dimensional test functions over the rather large input domain [-1, 1]10 . The family error is the MSE of the best fitting eligible model φˆi (averaged over an independent test data set); the classification error counts how often the product of sigmoids correctly predicts φˆi to be the best fitting model for a given input (i.e., argmaxiβi = ˆi). In the experiments we find that the algorithm reliably generates a family with optimal family error at the noise level (σ2 = 0.01). In 5 dimensions (not displayed here) the classification error rapidly converges to zero while in 10 dimensions, the classification error converges to around 4%. For more results see homepages.inf.ed.ac.uk/mtoussai/projects/05-ijcai.

5 Discussion

The presented model addresses the problem of handling the discontinuities that naturally arise, e.g., in sensorimotor data during interaction with a structured environment. Our model extends earlier local learning approaches in several ways: The responsibility region associated with each local model (learned with the product of sigmoids) has a much more versatile boundary shape compared to typical Gaussian kernels. Problems associated with initialization of kernel shapes or widths and the heuristic choice of an ad hoc number of submodels are circumvented by the robust incremental allocation of new models. Although we consistently used PLS as the underlying regression machinery, the general model allows to utilize any efficient single model learner to represent the local models φi as well as the classifier functions φij. Future work will in particular investigate non-linear learners for the local models as well as the boundary classifiers.