Constraints as Data: A New Perspective on Inferring Probabilities

1 Introduction

Probabilistic representations of uncertainty usually consist of a single probability distribution over a large (but finite) domain of possible states BW BP CUCS BD BN BM BM BM BN CS D2 CV. It is thus required to assign a probability value D4 CX to each state CS CX . Usually, a direct, full assessment of all these values is very difficult or impossible. All one usually is able to obtain are partial descriptions of D4 BP B4D4 BD BN BM BM BM BN D4 D2 B5 by constraints of e.g. the form D4B4BT CY BUB5 AK DE, D4B4BTB5 B7 D4B4BUB5 AK D4B4BVB5, or "BT and BU are independent", where BTBN BUBN BV are subsets of BW. Such constraints can be derived by knowledge elicitation from an expert, by direct observations of the domain, or by any other information gathering process.

A set CR BD BN BM BM BM BN CR  of constraints defines the set A1B4CR BD BN BM BM BM BN CR  B5 of probability measures on BW that are consistent with the constraints. Very rarely will A1B4CR BD BN BM BM BM BN CR  B5 consist of a single probability distribution. Instead, it will either contain more than one element, or be empty (i.e., the constraints are inconsistent). A fundamental problem in probabilistic reasoning then is to select from the admissible set A1B4CR BD BN BM BM BM BN CR  B5 a distribution D4 BPBM selB4CR BD BN BM BM BM BN CR  B5 as the best guess for the true distribution the constraints describe.

This measure selection problem is well studied in the literature, particularly for the case where the constraints are linear and consistent. It is almost unanimously suggested that in this case one should select the distribution with maximal entropy fromA1B4CR BD BN BM BM BM BN CR  B5 [Shore and Johnson, 1980;

Lemmer and Barth, 1982; Jaynes, 1982; Cheeseman, 1983; Paris and Vencovsk´a, 1990; R¨odder and Meyer, 1996]. A more general class of constraints is considered by Drudzel and van der Gaag [1995] who then employ the center of mass selection rule (according to this rule one selects the center of mass of the admissible region).

In this paper we propose a new selection rule which is radically different from either maximum entropy or center of mass. It is motivated by the observation that in spite of the very compelling justifications it has been given [Shore and Johnson, 1980; Jaynes, 1982; Paris and Vencovsk´a, 1990], maximum entropy selection has some rather counterintuitive properties. These are illustrated by the following examples.

Example 1.1 Overhearing two strangers talking at an airport, we hear the first one saying " BM BM BM Jones got at least 45% of the votes BM BM BM ", and the second replying " BM BM BM Smith didn't get any less than 5% either BM BM BM ". Before the two disappear in the crowd, we also hear them both agreeing on the fact that if anyone else had bothered to run for mayor, then neither Smith nor Jones would have had a chance of winning the election. Suppose, now, that we need to assess the probability  B4SmithB5 of an arbitrary voter in the unnamed home town of the two strangers having voted for Smith. The information we have establishes a lower bound of 0.05 and an upper bound of 0.55 on B4SmithB5. Moreover, we have learned that the relevant underlying state space only consists of Smith and Jones. If we base our probability assessment on entropy maximization, then we will obtain B4SmithB5 BP BCBMBH. Intuitively, this assessment appears to be overly optimistic from Smith's point of view.

Example 1.2 For the construction of a medical diagnosis system ten different experts are asked for bounds on the two crucial conditional probabilities  BD BP B4stylosis CY polycarpiaB5, and  BE BP B4xylopserosis CY anameaeB5. Assume that 0.41 and 0.51 are the greatest lower bound and smallest upper bound, respectively, mentioned by any expert for  BD . Having complete confidence in the experts, we will then take it as given that the true value for  BD lies in the interval [0.41,0.51]. Let [0.49,0.61] be the correspondingly defined interval for  BE . Applying maximum entropy to find the best values for  BD and  BE for our expert system, we will determine  BD BP  BE BP BCBMBH. This appears somewhat counterintuitive because we have chosen the same value for both probabilities, even though the information provided would seem to indicate a smaller value for  BD than for  BE .

The reasons why the maximum entropy solution appears counterintuitive in the two examples are very similar. In the first example, an equal percentage of 50% of votes for both Smith and Jones seems implausible, because the constraints are highly asymmetric. Experience tells us that the disparity of the given lower bounds probably reflects a similar disparity of the actual values, which will rather be assumed to be approximately 90% for Jones and 10% for Smith. Such an assessment could be based on a natural explanation for how the constraints were generated in the first place: one might suspect, for instance, that the constraints report the partial count of 50% of the votes, among which 45% were found to be for Jones, and 5% for Smith. In the second example it appears unlikely that the experts would systematically state larger upper and lower bounds for  BE than for  BD if these two probabilities were really the same.

In both examples we have thus argued that the maximum entropy distribution is a counterintuitive solution of the selection problem, because the given constraints are unlikely to be observed when this is the true distribution. Underlying this argument is a view of constraints that is fundamentally different from the view which (implicitly) underlies the use of the maximum entropy principle: entropy maximization is predicated on the view that the given constraints are just a description of a state of knowledge: the knowledge that the true distribution is a member of the admissible region defined by the constraints. We call this the constraints as knowledge perspective. In our examples - and, we would claim, in most cases where we encounter the measure selection problem - the given constraints are not only a description of our knowledge, they also are the source of our knowledge. They thereby carry not only the face-value information consisting of a restriction of the admissible region; they also carry the metainformation consisting of the fact that we observed exactly these constraints. This meta-information is relevant for the solution of the measure selection problem as it allows us to reason about the likelihood of observing the given constraints for different true distributions. We call the view of constraints that tries to take into account this meta-information the constraints as data perspective: constraints are seen as randomly sampled pieces of information. The distribution of this constraint data (the constraint distribution) is in part determined by the true distribution on the domain (the domain distribution), which we want to determine. In other words, the domain distribution is a parameter of the constraint distribution. Our problem thus becomes a statistical one: to infer a parameter of a distribution from random samples drawn from that distribution.

All statistical methods rely in part on considerations of likelihood. The most direct way to use likelihood is by maximum likelihood inference: select the parameter that gives highest probability to the observed sample. The measure selection rule we develop in this paper is likelihood maximization for the observed constraints. The main problem we face in a formal development of this intuitive principle is that statistical methods usually require a specific model on how the distribution of observed data depends on the parameter of interest, i.e. the stipulation of some underlying parametric family. Our goal, however, is to define a general rule for measure selection that does not require any knowledge about the randommechanism that produces the constraints. Our approach towards solving this dilemma is that of robust statistics: we do propose a specific model for the random generation of constraints, but this model is chosen such that in the long run it will lead to correct inferences for a fairly wide class of constraint distributions.

The constraints as data perspective coupled with statistical approaches to measure selection permits us to handle inconsistent sets of constraints just like consistent ones. Our statistical model for the constraint observation only must allow for the observation of wrong constraints, i.e. constraints not satisfied by the true distribution (as an erroneous assessment given by an expert, the premature and incorrect report of an election result, etc.). Such a model then assigns nonzero likelihoods to inconsistent sets of constraints, and a maximum likelihood solution can be found just as for consistent constraint sets.

The idea of measure selection by likelihood maximization for the observed constraints was already expressed in [Jaeger, 1998], but no concrete formalization of the idea was developed. The view of constraints as data has also been taken in somewhat different form by Dickey [1980], who proposed a model in which partial specifications of a probability distribution  were treated as random variables with a distribution depending on . A major difference between Dickey's and our work is that Dickey does not consider partial specifications by arbitrary linear constraints, but only by values for a fixed set of "aspects" of . It is interesting to note that Dickey takes it for granted that in most cases the specified aspects will overdetermine the model, i.e. be inconsistent, whereas authors in artificial intelligence assume underdetermined models.

In this paper we can only give an overview of our maximum likelihood approach to measure selection. Goal of this paper is to convey the main ideas, and to provide some insight into the feasibility of their mathematical development. More technical details, including proofs of the theorems here stated, will be given in a full technical paper.

2 The Constraint Sample Space

To treat constraints as random samples we have to view them as elements of some sample space on which probability distributions can be defined. Throughout we assume that the constraints refer to a distribution on a domain of D2 elements. The set of all these distributions can be identified with

A linear constraint then has the general form

We could identify this constraint with its parameters DC BD BN BM BM BM BN DC D2 BN DE, and thus take CA D2B7BD as our sample space. However, this would mean to view two equivalent constraints like D4 BD A0 BED4 BE AK BCBMBE and BED4 BD A0 BGD4 BE AK BCBMBG as different sample points. As it does not seem sensible that our method should depend on such representational variants of constraints, we prefer to distinguish constraints only according to the subsets of distributions they define. This can be done by writing constraints in a normal form

where D7 BMBP B4D7 BD BN BM BM BM BN D7 D2 B5 is an element of the D2 A0 BDdimensional unit sphere

As every linear constraint (1) can be transformed into a unique normal form (2), we henceforward identify linear constraints CR with points D7 BE CB D2A0BD , and model randomly observed constraints by probability distributions on CB D2A0BD .

In the binomial case (D2 BP BE), a constraint (2) is a (nontrivial) lower bound on D4 BD iff D7 BD BO BC and D7 BE BQ BC; it is a (nontrivial) upper bound iff D7 BD BQ BC and D7 BE BO BC. The following definition generalizes this classification of constraints.

Definition 2.1 A sign vector is any vector with components in CUA0BDBN BCBN BDCV. For D6 BE CA we define signB4D6B5 as A0BDBN BC or BD, depending on whether D6 BO BC, D6 BP BC, or D6 BQ BC. The sign vector signB4D7B5 for D7 BE CB D2A0BD is the vector B4signB4D7 CX B5B5 CXBPBDBNBMBMBM BND2 . Each sign-vector AG of length D2 defines a sector CB AG in CB D2A0BD :

The intuition behind this definition is that sectors contain constraints of the same qualitative type. The classification of constraints according to sectors gives rise to the following coarser, three-way distinction: a constraint D7 is vacuous iff signB4D7 CX B5 BIBPBE CUA0BDBN BCCV for all CX (a vacuous constraint is satisfied by all D4 BE A1 D2 ); D7 is a support constraint iff signB4D7 CX B5 BE CUBCBN BDCV for all CX (a support constraint is satisfied by all D4 BE A1 D2 whose set of support is a subset of CUCX CY signB4D7 CX B5 BP BCCV); D7 is proper iff signB4D7 CX B5 BP BD and signB4D7 CY B5 BP A0BD for some CXBN CY (a proper constraint D7 divides the interior of A1 D2 , i.e. there exist D4 BE intA1 D2 that satisfy D7, and D4 BC BE intA1 D2 that do not satisfy D7).

Figure 1 illustrates constraints from different sectors. Shown in the figure is the polytope A1 BF with its 3 vertices corresponding to domain distributions that assign unit mass to one of the states in BW. Six different constraints from three different sectors are represented by the halfplanes of points satisfying the constraint. Halfplanes are shown by their boundary line, and a shading that indicates to which side of the boundary the halfplane extends.

For the rest of the paper we make two simplifying assumptions. Assumption 1: All constraints in the observed sample are proper. Assumption 2: The model D4 BE A1 D2 we want to determine lies in the interior intA1 D2 of A1 D2 . The two assumptions are somewhat connected. Non-proper constraints are essentially constraints on the set of support of D4. Thus, both assumptions will be satisfied if in an initial inference step we use all observed non-proper constraints to determine a set of support for our model, and then use the method we shall develop on the remaining proper constraints to determine D4 with that set of support. With these assumptions, the measure selection problem consists of finding selection rules in the following sense.

Definition 2.2 A selection rule is any function that for every D2 BE  maps any tuple D7 BD BN BM BM BM BN D7  BE B4CB D2A0BD B5  ( BE ) of proper constraints to a set selB4D7 BD BN BM BM BM BN D7  B5 AI intA1 D2 .

This definition of a selection rule is not directly tied to constraints as data, and is very similar to Paris and Vencovsk´a's [1990] notion of an inference process. It is very general in two respects: first, it is not required that selB4D7 BD BN BM BM BM BN D7  B5 consists of a unique point. While it is obviously desirable that a selection rule yields unique solutions as often as possible, one needs to take the possibility into account that no principled statistical method can guarantee unique solutions in all cases. Second, it is not demanded that selB4D7 BD BN BM BM BM BN D7  B5 be a subset of A1B4D7 BD BN BM BM BM BN D7  B5 (the distributions on BW that actually satisfy the constraints). Such a demand, which is natural from the constraints as knowledge perspective, is not required from the constraints as data perspective. To see why, recall that in order to deal with inconsistent constraint sets (and also for greater realism) we should work with probabilistic models according to which it is possible to observe false constraints. This means that even for consistent constraint sets we must take the possibility into account that it contains false constraints, and that therefore the true domain distribution does not belong to A1B4D7 BD BN BM BM BM BN D7  B5.

3 Invariance and Equivariance

A selection rule in the sense of Definition 2.2 is a maximum likelihood selection rule, if it takes the following form: for every  BE , and for every D4 BE int A1 D2 a probability distribution BY  D4 on B4CB D2A0BD B5  is defined, and for a sample D7 BD BN BM BM BM BN D7  of  constraints we select the distributions in intA1 D2 that maximize the likelihood of the sample:

where CU  D4 is the density function of BY  D4 . Usually, constraints will be assumed to be independent, so that with CU D4 BMBP CU BD D4

Whenever some information about the random process that generates the constraints is available, then one will choose in (4) a family B4BY  D4 B5 D4 that is a plausible model for this random process. The question we shall be concerned with, however, is what to do when no particular information about the generation of the constraints is available. Thus, we address the same inference problem as addressed by maximum entropy inference: input of the selection problem is a set of constraints, and nothing else.

The justification of the maximum entropy principle, in broad outline, takes the following form: because there is no information except the constraints, one should select the domain distribution D4 that encodes the least additional information beyond the face-value information provided by the constraints. Minimal information content, in turn, is realized by distributions D4 that, roughly speaking, maximize independencies and uniformity. Our approach to dealing with the lack of information is somewhat similar, only applied to the constraint distribution: because we have no information on the family B4BY  D4 B5 D4 , we should assume the least specific structure of this family. In particular, we will assume that the constraints are independent, and that the family B4BY D4 B5 D4 is homogeneous in D4, in a sense that will be formalized by the notion of BZ-invariance, which is developed in this section.

We derive the concept of BZ-invariance from the semantic concept of a random constraint generating mechanism that works uniformly for all D4. Additional support for the BZinvariance assumption on B4BY D4 B5 D4 will be given by the observation that maximum likelihood selection with respect to BZinvariant families is BZ-equivariant, and that this can be understood as the formalization of the intuitive principle that a uniform shift applied to all constraints should induce a similar shift of the selected distributions (cf. Example 1.2).

To motivate the concept of a random constraint generating mechanism that works uniformly for all D4, reconsider Example 1.1, and the subsequently given explanation of how the constraints might have been generated from a partial count of 50% of the votes. If the true distribution is indeed CM D4 BP B4 CM D4 BD BN CM D4 BE B5 BP B4BMBDBN BMBLB5 B4BP B4B4SmithB5BN B4JonesB5B5B5, then the observation of the constraints BMBCBH AK D4 BD AK BMBHBH follow as a result of a sequence of chance events: the partial count of 50% of the votes becomes known, this partial count happens to be an accurate projection of the full count, and two strangers happen to mention these partial counts in their conversation. This sequence of chance events does not depend on the distribution CM D4. If the true distribution was D4 A3 BP B4BMBGBN BMBIB5, then the same sequence of events would occur with the same likelihood, but now generate the constraints BMBE AK D4 BD AK BMBJ.

Generalizing from this example, we obtain the (yet informal) notion of a random constraint generating process that works uniformly for all D4: the constraint generating mechanism will produce a constraint CM D7 when the true domain distribution is CM D4 with the same likelihood as it will produce a constraint D7 A3 corresponding to CM D7 when the true distribution is D4 A3 . To make this idea precise, we have to find a transformation CV on constraints that maps every D7 BE CB D2A0BD to a corresponding CVB4D7B5 BE CB D2A0BD , such that an observation of D7 under the true distribution CM D4 corresponds to an observation of CVB4D7B5 under D4 A3 . The correspondence expressed by CV should preserve elementary qualitative properties of constraints. Two natural preservation conditions are:

Sector preservation: CV maps every sector CB AG bijectively onto itself.

Implication preservation: For all CZ BE , D7 BD BN BM BM BM BN D7 CZ :

Sector preservation means that two corresponding constraints should be of the same qualitative type, as expressed by their membership in a sector. Implication preservation says that logical relationships between constraints should be preserved. Implication preservation is equivalent to the conjunction of two simpler conditions: CVB4A0D7B5 BP A0CVB4D7B5 for all D7, and CC CZA0BD CXBPBD A1B4D7 CX B5 BP BN B8 CC CZA0BD CXBPBD A1B4CVB4D7 CX B5B5 BP BN (consistency preservation).

The following definition introduces a class of transformations that satisfy both properties.

Definition 3.1 Let D6 BP B4D6 BD BN BM BM BM BN D6 D2 B5 BE B4CA B7 B5 D2 . The transformation CV D6 BM CB D2A0BD AX CB D2A0BD is defined by

We write BZ D2 for the set CUCV D6 CY D6 BE B4CA B7 B5 D2 CV.

It is obvious that transformations CV D6 satisfy sector preservation. They also satisfy a slightly strengthened version of implication preservation. For this, denote by B4D7B5 AI CA D2 the set of all real solutions of (2), without the restriction to solutions D4 BE A1 D2 (so that A1B4D7B5 BP B4D7B5 CK A1 D2 ). In analogy to (6) we can then define global implication preservation of CV by the condition

With condition (7) we look at constraints as defining sets of real numbers, not sets of probability distributions. In our context condition (6) seems to be the more pertinent one. We nevertheless here introduce the global version (7), because with this version we can prove the following representation theorem.

Theorem 3.2 Let D2 AL BF, CV BM CB D2A0BD AX CB D2A0BD . CV preserves sectors and is globally implication preserving iff CV BE BZ D2 .

The theorem does not hold for D2 BP BE. The proof is by reduction to a classical representation result in projective geometry which characterizes mappings that preserve collinearity. We may conjecture that the theorem also holds when the condition of global implication preservation is replaced by implication preservation in our preferred sense (6). A proof of this modified theorem appears to be considerably harder, however.

In light of Theorem 3.2 we see the transformations CV D6 BE BZ D2 as the adequate realization of the concept of correspondence of constraints. To relate this correspondence to different domain distributions, we define dual transformations on A1 D2 .

Definition 3.3 Let D6 BP B4D6 BD BN BM BM BM BN D6 D2 B5 BE B4CA B7 B5 D2 . The transformation AM CV D6 BM A1 D2 AXA1 D2 is defined by

We write AM BZ D2 for the set CUAM CV D6 CY D6 BE B4CA B7 B5 D2 CV.

The mapping AM CV D6 is dual to CV D6 in that it is the only transformation of A1 D2 such that for all D4,D7:

Our initial intuition of corresponding observations of constraints now can be phrased as follows: the observation of constraint D7 under the true domain distribution D4 corresponds to the observation of constraint CV D6 B4D7B5 under the true domain distribution AM CV D6 B4D4B5. Figure 2 shows three different transformations of a set of three constraints, and the dual transformations of one probability measure inside the admissible region of the constraints. Each of the three sets of constraints can be transformed into any of the other two sets by unique CV D6 BE BZ D2 . The dual transformations AM CV D6 at the same time transform the indicated points in A1 BF into each other.

AM

With the transformations BZ D2 and BZ D2 we can now finally formalize the idea of a constraint generating mechanism that works uniformly for all D4:

Definition 3.4 Let B4BY D4 B5 D4BEintA1 D2 be a family of distributions on CB D2A0BD . The family is called BZ-invariant if for all CV D6 BE BZ and all D4 BE intA1 D2 :

When the distributions BY D4 are represented by densities CU D4 relative to a suitably chosen underlying measure on CB D2A0BD , then (9) can be expressed by the condition

By using such appropriate density functions, it is immediate that the maximum likelihood selection rule given by (4) and (5) becomes BZ-equivariant in the sense of the following definition.

Definition 3.5 A selection rule sel is called G-equivariant iff for samples B4D7 BD BN BM BM BM BN D7  B5 of constraints, and every CV D6 BE BZ D2

Note that the concept of BZ-equivariance does not, in turn, rely on maximum likelihood selection rules. Indeed, independently from the constraints as data interpretation, BZequivariance captures the idea that when the given constraints undergo a shift in one directions, then the selected distributions should undergo a similar shift. In Figure 2, a BZequivariant rule would have to select the distribution indicated by a cross given the solid constraints iff it selects the distribution indicated by a diamond given the dashed constraints iff it selects the distribution indicated by a box given the dotted constraints.

A second homogeneity assumption one will make about B4BY D4 B5 D4 in the absence of any information to the contrary is permutation invariance: if AP is any permutation of BDBN BM BM BM BN D2, then for all D4

Maximum likelihood selection with respect to a permutation invariant family B4BY D4 B5 D4 leads to a permutation equivariant selection rule:

4 Robust Estimation

In the previous section we have argued that when no particular information about the constraint generating family B4BY D4 B5 D4 is given, then reasonable assumptions on this family are BZand permutation invariance. These assumptions alone are not nearly sufficient to identify a unique such family: if BY is any distribution on CB D2A0BD that satisfies BYB4APD7B5 BP BYB4D7B5 for all D7 BE CB D2A0BD and all permutations AP, then BY gives rise to a BZ- and permutation invariant family B4BY D4 B5 D4 by letting BY D9 BMBP BY, where D9 BP B4BDBPD2BN BM BM BM BN BDBPD2B5 is the uniform domain distribution, and BY D4 BMBP CV D6 B4BY D9 B5, where CV D6 BE BZ is the transformation uniquely determined by D4 BP AM CV D6 B4D9B5. Conversely, every BZ- and permutation invariant family B4BY D4 B5 D4 is uniquely determined by its member BY D9 , which has to satisfy BY D9 B4APD7B5 BP BY D9 B4D7B5.

In the following, we define a particular family B4 D4 B5 D4 by way of defining  D9 . The motivation for this family comes out of the robustness of maximum likelihood selection with respect to this family (Theorem 4.2).  D9 can be thought of as a mixture of multivariate Laplace distributions that are separately defined on each sector. The usual multivariate Laplace distribution on CA D2 has a density CUB4DCB5 that depends on the Euclidean distance between DC and the mean D1 of the distribution. To define Laplace-like distributions on the sectors of CB D2A0BD , we first introduce a suitable metric on sectors:

Definition 4.1 Let AG BE CUA0BDBN BCBN BDCV D2 , D7BN D7 BC BE CB AG . Define

A density D0 D9 B4D7B5 now is defined as a function of the distance between D7 and a reference constraint D1B4AGB5 BE CB AG , which can be thought of as the mean constraint in sector CB AG .

In order for  D9 to satisfy  D9 B4APD7B5 BP  D9 B4D7B5 for all permutations AP, the D1B4AGB5 have to be chosen such that D1B4APAGB5 BP

APD1B4AGB5 for all sign vectors AG and permutations AP. Apart from this condition, no restriction has to be imposed on the choice of theD1B4AGB5 in order to obtain our robustness result. We therefore only assume at this point that some D1B4AGB5 BE CB AG have been appropriately fixed, and define

The function D0 D9 B4D7B5 is the density of a probability distribution  D9 , which induces a BZ- and permutation invariant family B4 D4 B5 D4 . The maximum likelihood selection rule sel  based on this family is distinguished by the following robustness property.

Theorem 4.2 Let D2 AL BF. Let B4BY D4 B5 D4 be a BZ- and permutation-invariant family of probability distributions on proper constraints such that BY D9 B4CB AG B5 BQ BC for all proper sectors CB AG . Let BY BD D4 denote the distribution of an infinite sequence D7 BD BN D7 BE BN BM BM BM of independent constraints drawn according to BY D4 . Then

The theorem says that in the long run we will select with probability 1 the correct distribution D4 by using sel  , even when the constraints are actually generated according to distributions B4BY D4 B5 D4 . The conditions D2 AL BF and BY D9 B4CB AG B5 BQ BC make sure that with probability 1 sel  B4D7 BD BN BM BM BM BN D7  B5 will be a unique point for all sufficiently large . To obtain an analogous result for D2 BP BE a mild additional condition on B4BY D4 B5 D4 must be added. The proof of theorem 4.2 follows the proof of a general robustness result given as theorem 1 in [Huber, 1967].

Theorem 4.2 provides a good justification for using sel  on "large" samples. It does not provide any guarantee that sel  will show a sensible behavior on small samples. In particular, the behavior on small samples can be strongly affected by the special choice of the reference constraints D1B4AGB5. Thus, the definition of sel  and Theorem 4.2 do not yet provide a full answer to the measure selection problem from the constraints as data perspective. To extend these first results towards a fully satisfactory solution, one will have to develop suitable criteria by which to judge the performance of a maximum likelihood selection rule on small samples, and to specialize or modify the definition of sel  to obtain a selection rule that performs well according to these criteria (but retains the asymptotic behavior (16)).

5 Conclusion

We have seen that an interpretation of constraints as data, not as knowledge, leads to a completely new perspective on the measure selection problem. This perspective calls for statistical methods of parameter estimation as the tool for measure selection. The key problem we then face is that statistical methods call for a statistical model for the data generation, but that (according to the traditional problem statement that we deal with) no information about the appropriate statistical model is given. We have argued that in the absence of any such information BZ- and permutation invariance are natural homogeneity assumptions for the constraint distributions. B4 D4 B5 D4 is a relatively simple BZ- and permutation invariant family of constraint distributions that leads to a robust maximum likelihood selection rule.

Future work will have two major directions: first, the definition of sel  will be refined in order to obtain a sensible small sample behavior of the selection rule. Second, it will be explored to which degree the assumptions made in Theorem 4.2 on the constraint generating family B4BY D4 B5 D4 can be relaxed without loosing (16) for sel  .