Ranking Alternatives on the Basis of Generic Constraints and Examples - A Possibilistic Approach

1 Introduction

A classical way for comparing alternatives is to use multiple criteria for evaluating them in an absolute manner, using linearly ordered scales. These scales are often numerical, and under the hypothesis that are commensurable, different aggregation procedures that reflect various combination attitudes can be applied in order to build a complete preorder for rankordering the alternatives on the basis of the global evaluations that are obtained. However, in many practical problems (such as multiple criteria analysis, flexible constraints satisfaction problems), a numerical scale, such as [0,1] is too rich for being used, and more qualitative scales having a finite number of levels have to be preferred. But, the internal operations that can be defined on these latter scales (e.g., [Mas et al., 1999; Fodor, 2000]) have a limited discriminating power since they take values on a finite range.

The problem thus amounts to compare alternatives represented by vectors of qualitative criteria evaluations without aggregating them. Apart from the Pareto partial preorder that should constrain any complete preorder between alternatives, one may have some generic rules that further constrain these complete preorders. For instance, one may state that some criterion is more important than, or equally important as, other criteria (maybe in a limited context). One may also have at our disposal some examples of preferences between fully specified alternatives. The problem addressed here, is then to complete the Pareto partial preorder in agreement with the constraints in a way that is as little arbitrary as possible.

The paper is organized as follows. The next section introduces the problem formally, while Section 3 provides a short background on a general family of aggregation functions that can be defined under the form of a Choquet integral, which will be used in the paper as a comparison landmark. Section 4 describes the proposed approach that uses a minimal commitment principle for building a complete preorder in agreement with the constraints. This principle expresses that an alternative is good as much as there is no other alternative that are considered to be better. Section 5 applies this approach to constraints that directly mirror the way the comparative importance of criteria is stated when using a Choquet integral aggregation. This enables a comparison between the two approaches. Section 6 reviews related works, and points out the new features of the proposed approach.

2 Framework

It is assumed that objects to be rank-ordered are vectors of satisfaction levels belonging to a linearly ordered scale S = {s1, · · · , sh} with s1 < · · · < sh, each vector component referring to a particular criterion. Thus, it is supposed that there exists a unique scale S on which all the criteria can be estimated (commensurateness hypothesis).

Preferences are expressed through comparisons of such vectors ui = {ai1, · · · , ain} (written ai1 · · · ain for short), where aij ∈ S, under the form of constraints

expressing that u = a1 · · · an is preferred to (or is more satisfactory than) u′ = a′1 · · · a′n.

Let U be the set of all possible vectors u = a1 · · · an, called also alternatives, such that aj ∈ S for all j = 1, · · · , n. ⪰ is a pre-order on U if and only if it is a reflexive and transitive relation. u ≻ u′ means that u ⪰ u′ holds but not u′ ⪰ u. u = u′ means that both u ⪰ u′ and u′ ⪰ u hold, i.e. u and u′ are equally preferred, and u ∼ u′ means that neither u ⪰ u′ nor u′ ⪰ u hold, i.e. u and u′ are incomparable. ⪰ is said to be complete (or total) if all pairs of alternatives are comparable.

Some components may remain unspecified when comparing alternatives. They are replaced by a variable xj if the jth component is free to take any value in the scale. This allows to express generic preferences as for example Pareto ordering written as

In any case, Pareto ordering is always assumed to hold. Besides, other generic constraints of particular interest include those pertaining to the expression of the relative importance of criteria. The greater importance of criterion j with respect to criterion k can be expressed under different forms. One way to state it is by exchanging xj and xk and writing

One may think of other ways of expressing that j is more important than k. For instance, one may restrict the above preferences to extreme values of S for the xi's such that i ̸= j and j ̸= k, since weights of importance in conjunctive aggregation can be obtained in this way for a large family of operators (e.g., [Dubois et al., 2001]). A more drastic way for expressing relative importance would be to use a lexicographic ordering of the vector evaluations based on a linear order of the levels of importance for the criteria. Then, the problem of ordering the vectors is straightforward.

Note that the first above view of relative importance, which is used in the following, is a ceteris paribus preference of subvector (xj, xk) with respect to (xk, xj) for xj > xk, where the first (resp. second) component refers to criterion j (resp. k), which expresses preferential independence.

Another way to relate criteria is to express an equal importance between them. It can be expressed by stating that any two vectors where xj and xk are exchanged, and otherwise identical, have the same levels of satisfaction. Formally,

It is worth noticing that transitivity is required between alternatives only and not between generic constraints. More precisely if it holds that u ≻ u′ and u′ ≻ u′′ with respect to some generic constraints then we necessarily have u ≻ u′′ . However if we have two generic constraints X ≻ Y and Y ≻ Z, where X, Y and Z are three criteria, representing that X (resp. Y ) is more important than Y (resp. Z) then we do not necessarily have X ≻ Z. To illustrate this, let X, Y and Z be evaluated on a scale {a, b, c} with a > b > c. X ≻ Y and Y ≻ Z are relative importance constraints defined by xyz ≻ yxz for x > y and xyz ≻ xzy for y > z respectively. Let us now check whether we have X ≻ Z i.e. xyz ≻ zyx for x > z. We have abc ≻ cba obtained by transitivity from abc ≻ bac (w.r.t. X ≻ Y ), bac ≻ bca (w.r.t. Y ≻ Z) and bca ≻ cba (w.r.t. X ≻ Y ). However acb is not preferred to bca since we cannot reach bca from acb by transitivity using the generic constraints. Indeed generic constraints require to explicitly express each constraint that we want to have, i.e. X ≻ Z in the above example.

Definition of the problem

The problem of rank-ordering the alternatives is decribed as follows. Given a set of constraints C of the form {ui ≻ ui′ |i = 1, · · · , m}, where ui and ui′ are instantiated on S, our aim is to compute a complete pre-order ⪰ over U that satisfies all constraints of C. Such a pre-order should not add any additional constraint. One may wonder why we compute a complete pre-order from the set C of partially specified preferences. This is a debatable question and we really believe that the answer depends on the application. If C describes preferences over cars then we may permit that two cars are incomparable. However if C describes preferences over students' grades, as it is the case in our running examples, it is natural to have a complete pre-order over students.

The set C may contain generic constraints as described previously but also particular examples of preferences. Note that some pre-orders, such as those induced by the minimum aggregation operator, are excluded as soon as Pareto constraints are considered.

3 Numerical aggregation by Choquet integral

Aggregation of object attribute values in the presence of interaction between criteria is essential in many decision making problems. Choquet integrals are very popular aggregation operators as they allow to model such interactions and thus to represent preferences that cannot be captured by a simple weighted arithmetic mean [Grabisch, 1995; 1996]. Using a particular measure, they aggregate valued attributes describing alternatives into a unique value. A Choquet integral is based on a fuzzy measure defined by:

Definition 1 Let A be the set of attributes and I(A) be the set of all possible subsets of A. A fuzzy measure is a function μ from I(A) to [0, 1] such that:

• ∀X, Y ∈ I(A) if X ⊆ Y then μ(X) ≤ μ(Y ).
• μ(∅) = 0, μ(A) = 1.

A discrete Choquet integral with respect to a fuzzy measure μ is defined as follows:

Definition 2 Let μ be a fuzzy measure on A = {a1, · · · , an}. The discrete Choquet integral w.r.t. μ is defined by:

where a(i) indicates that the indices have been permuted so that 0 ≤ a(1) ≤ · · · ≤ a(n), and A(i) = {a(i), · · · , a(n)} with a(0) = 0.

Example 1 ([Grabisch, 1995; 1996; Marichal, 1998]) Let A, B and C be three students evaluated with respect to three subjects: mathematics (M), physics (P) and literature (L). Students' grades are summarized in Table 1. Using Choquet

integral with a fuzzy measure μ, the global grade for each student is computed as follows:

• student A: Chμ(A) = Chμ(18, 16, 10) = 10∗μMPL+ (16 - 10) ∗ μMP + (18 - 16) ∗ μM ,
• student B: Chμ(B) = Chμ(10, 12, 18) = 10∗μMPL+ (12 - 10) ∗ μPL + (18 - 12) ∗ μL,
• student C: Chμ(C) = Chμ(14, 15, 15) = 14∗μMPL+ (15 - 14) ∗ μPL,

where μX, μXY and μXYZ with X, Y, Z ∈ {M, P, L} denote the values of the fuzzy measure μ for the corresponding set of subjects.

The school is more scientifically than literary oriented and it gives the same importance to mathematics and physics. Moreover the school wants to favor well equilibrated students without weak grades so we should have: C preferred to A and A preferred to B, i.e. C ≻ A ≻ B 1 . As indicated before, the fuzzy measure μ models interaction between subjects. Since mathematics and physics have the same importance and they are more important than literature we have μM = μP , μM > μL and μP > μL. Moreover both mathematics and physics are scientific subjects, and thus are considered close, while literature is not. So the interaction between mathematics (resp. physics) and literature is higher than the interaction between mathematics and physics. Thus μML = μPL > μMP . This gives the following set of constraints on μ: {μM = μP , μM > μL, μP > μL, μML = μPL, μML > μMP , μPL > μMP }.

In addition we consider the constraints Chμ(C) > Chμ(A) and Chμ(A) > Chμ(B) corresponding to the preference order between students A, B and C. Table 2 gives an example of measure μ satisfying all these constraints [Marichal, 1998]. Using the discrete Choquet integral w.r.t. μ given

Let us consider another student D having 15 in physics, 15 is mathematics and 12 in literature. Using discrete Choquet integral with respect to μ given in Table 2 we get Chμ(D) = 13.5. Then we have the following ordering C ≻ A ≻ B ≻ D. Let us now use another fuzzy measure μ′ which is equal to μ except for μPL and μML. Instead we have μ′PL = μ′ML = 0.8. We can check that μ′ satisfies the set of constraints on μ. Using discrete Choquet integral with respect to μ′ we have C ≻ A ≻ D ≻ B. So we still have C ≻ A ≻ B but the ordering over B and D is reversed. This shows that Choquet integral is sensitive with respect to the fuzzy measure μ and may give arbitrary order over alternatives that are not explicitly stated in the set of constraints.

4 A qualitative ranking approach

Since a scale more refined than S is needed to rank-order the alternatives, we use the interval [0, 1] to encode this ordering. Indeed S would only offer a finite number of levels for discriminating alternatives. For this purpose, we write our constraints in terms of a possibility distribution π, which is a function from the set of alternatives U to [0, 1], and provides a complete pre-order between alternatives on the basis of their possibility degrees.

1 Note that there is no weighted arithmetic mean that gives this order over A, B and C [Marichal, 1998].

4.1 General principle

In the possibility theory framework, an elementary preference between alternatives u ≻ u′ is encoded by the constraint π(u) > π(u′ ). Generally those constraints induce partial pre-orders on the set of interpretations. Our aim is to combine all these partial pre-orders and compute a total pre-order consistent with the set of constraints. In possibility theory we distinguish two completion principles called minimal and maximal specificity principles that respectively generate the largest and the smallest possibility distributions which satisfies the set of constraints. The following defines the notion of specificity between possibility distributions:

Definition 3 (Minimal/Maximal specificity principle) Let

π1 and π2 be two possibility distributions over U. π1 is less specific than π2, denoted π1 ≥ π2, iff

π belongs to the set of the least (resp. most) specific possibility distributions among a set of possibility distributions P if there is no π′ in P such that π′ > π (resp. π > π′ ), i.e. π′ ≥ π holds but π ≥ π′ does not (resp. π ≥ π′ holds but π′ ≥ π does not).

Indeed the minimal specificity principle gives the highest possible degree to alternatives, while the maximal specificity principle gives the lowest possible degree to alternatives. The choice of using minimal or maximal specificity principle depends on the application. The minimal specificity principle used in the following amounts to consider that an alternative is as good as permitted by the constraints. Thus, an unconstrained alternative is good by default.

Before we present our approach it is worth noticing that possibility distributions are purely qualitative here, although they are encoded on the interval [0, 1], and can be represented by a well ordered partition (E1, · · · , Ek) on U such that:

• E1 ∪ · · · ∪ Ek = U with Ei ∩ Ej = ∅ for i ̸= j,
• ∀u, u′ ∈ Ei, π(u) = π(u′ ),
• ∀u, u′ ∈ U, if u ∈ Ei and u′ ∈ Ej with i < j then π(u) > π(u′ ).

4.2 Minimal specificity principle-based algorithm

An elementary preference has generally the following form:

For example given three criteria X, Y and Z, a relative importance constraint of X over Y is written as:

A set of constraints of the form (1) can be written in a compact form as a set of the following constraints:

where U1 and U2 are subsets of U.

We may also have equality constraints i.e.

For example given three criteria X, Y , and Z, stating that X and Y have the same importance is written as:

Algorithm 1 computes the unique least specific possibility distribution satisfying a set of constraints C of the form (2) (constraints (1) being a special case of constraints (2)) and (3) obtained from generic constraints and/or examples.

Let C = {Ci : i = 1, · · · ,m}. From C we define

where L(Ci) = U1 and R(Ci) = U2 for a constraint Ci : min{π(u)|u ∈ U1} >max{π(u′ )|u′ ∈ U2}. Let EQ = {π(uj ) = π(ul )}.

Algorithm 1 [Benferhat et al., 2001] is a generalization of

Algorithm 1: The least specific possibility distribution.

begin

k = 0; while U is not empty do - k ← k +1 ; - Ek = {u|∀(L(Ci), R(Ci)) ∈ LC , u ̸∈ R(Ci)} ; α = true while α = true do α = false; for π(uj ) = π(ul ) in EQ such that uj ̸∈ Ek or ul ̸∈ Ek do α = true, Ek = Ek\{uj , ul} if Ek = ∅ then Stop (inconsistent statements); - U = U\Ek ; - From C remove (L(Ci), R(Ci)) such that L(Ci)∩ Ek ̸= ∅ ; - From EQ remove π(uj ) = π(ul ) s.t. uj ∈ Ek. end return π = (E1, · · · , Ek)

the possibilistic counterpart of Pearl's algorithm for system Z [Pearl, 1990].

Example 2 Let us consider two subjects "mathematics" and "literature" that are evaluated on a scale a > b > c with "a" for good, "b" for medium and "c" for bad. Thus a student having "ac" is good in mathematics and bad in literature. Pareto ordering forces to have π(xy) > π(x′ y′ ) as soon as x > x′ and y ≥ y′ or x ≥ x′ and y > y′ for x, y, x′ , y′ ranging in {a, b, c}. Pareto principle generates the following set of constraints:

C = {min{π(aa)} > max{π(ab), π(ba), π(ca)}, min{π(aa), π(ab)} > max{π(ac), π(bb), π(bc), π(cb), π(cc)}, min{π(ac), π(ba)} > max{π(bc), π(cc)}, min{π(bb)} > max{π(cb), π(cc), π(bc)}, min{π(ba)} > max{π(ca), π(bb), π(cb)}, min{π(bc), π(cb), π(ca)} > max{π(cc)},

min{π(ca)} > max{π(cb)}}.

The application of the minimal specificity principle leads to π = ({aa}, {ab, ba}, {ac, bb, ca}, {bc, cb}, {cc}). Note that letting π(ac) = π(ca) > π(bb) or the converse would lead to express more constraints than what is only specified by Pareto constraints. In fact, it may look a little surprising to get π(ac) = π(bb) = π(ca). However this is justified by the fact that the minimal specificity principle gives to each alternative the highest possible rank (i.e., possibility degree). The alternatives ac, bb and ca cannot have the highest possibility degree since following Pareto ordering, they are strictly less preferred than aa, ab and ba respectively. To ensure that we associate the highest possibility degree to these alternatives, the minimal specificity principle keeps the three pairs of evaluations at the same level, and they are ranked immediately below ab and ba.

The following example extends Example 2 with relative importance constraints:

Example 3 (Example 2 continued)

Suppose that mathematics is more important than literature. This is translated by the following relative importance constraint: π(xy) > π(yx) for x > y. The instantiation of this constraint provides a new set of constraints:

C ′ = {π(ab) > π(ba), π(ac) > π(ca), π(bc) > π(cb)}. Let us now apply Algorithm 1 on C ∪ C ′ , we get: π′ = ({aa}, {ab}, {ba, ac}, {ca, bb}, {bc}, {cb}, {cc}).

4.3 Ordering queries

We may need to compare specific alternatives without computing the whole complete pre-order generated by Algo. 1. An ordering query over two distinct alternatives u1 and u2 consists in checking whether u1 ≻ u2 , u2 ≻ u1 , u1 = u2 , or not, given a set of generic constraints and examples. Algo. 2 gives a way to answer such queries. The idea is to compute the set of alternatives that are less preferred to u1 w.r.t. generic constraints and examples. Then we compute the deductive closure of this set w.r.t. generic constraints and examples. If u2 belongs to this set or some alternative in this set is preferred to u2 following Pareto principle then u1 ≻ u2 . Let u ≻P u′ stand for u is preferred to u′ following Pareto principle, and C be the set of generic constraints and examples.

Example 4 (Example 2 continued) Let us compare the alternatives ac and cb. We have U1 = {ac} and U2 = {ca}. We have ca ≻P cb then ac is strictly preferred to cb.

Note that Algorithm 2 returns that ac and bb are incomparable while ac ≻ bb w.r.t. Algorithm 1. Indeed, ac is put in the same stratum as ba (that dominates bb in the sense of Pareto). This is the effect of minimal specifity principle that puts any alternatives as high as possible. Thus, using Algorithm 2, one can distinguish the alternatives that are ranked only by virtue of the constraints, from those that also require the application of a default principle (here the minimal specificity principle) to be rank-ordered by application of Algorithm 1. Note that if we are only interested in comparing two alternatives, we may stop Algorithm 1 as soon as the two alternatives are ranked (without in general computing the whole pre-order). Note Algorithm 2: Ordering queries

begin

if u1 ≻P u2 then return u1 ≻ u2 if u2 ≻P u1 then return u2 ≻ u1 - U1 = {u1 } ∪ {u|u1 = u is derived from EQ} if u2 ∈ U1 then return u1 = u2 - U2 = {u|u′ ∈ U1, u′ ≻ u is derived from C ∪ EQ} - X = ∅ while U2 ̸= X do - X = U2 - U2 = U2 ∪ {v|u ∈ X, u ≻ v is derived from C ∪ EQ} if ∃u ∈ U2 s.t. u ≻P u2 or u = u2 then return u1 ≻ u2 - U1 = {u2 } ∪ {u|u2 = u is derived from EQ} if u1 ∈ U1 then u1 = u2 - U2 = {u|u′ ∈ U1, u′ ≻ u is derived from C ∪ EQ} X = ∅ while U2 ̸= X do - X = U2 - U2 = U2 ∪ {v|u ∈ X, u ≻ v is derived from C ∪ EQ} if ∃u ∈ U2 s.t. u ≻P u1 or u = u1 then return u2 ≻ u1 if neither u1 ≻ u2 , nor u2 ≻ u1 , nor u1 = u2 hold then u1 and u2 are incomparable.

end

also that any strict preference or equality returned by Algorithm 2 is consistent with Algorithm 1.

5 Comparison with Choquet integral

In contrast to Choquet integral, which is sensitive to the numerical values of criteria and coefficients of the fuzzy measure whose adjustment is not obvious, our approach relies on qualitative values of criteria. This qualitative aspect makes that the approach is general, i.e. independent of the values of criteria, which provides more robust results compared to Choquet integral. In fact, constraints over coefficients in a Choquet integral, as well as ranking over specific alternatives can be encoded in our framework by means of generic constraints and examples respectively. Then the application of Algorithm 1 gives a complete pre-order on U which satisfies all generic constraints and examples. In order to make the comparison precise, we encode each inequality between fuzzy measure coefficients in terms of relative importance constraints. Recall that Choquet integral expression writes:

Let us consider Example 1, and use a qualitative scale S = {a, b, c, d, e, f} (with a > b > c > d > e > f) to encode students' grades 18, 16, 15, 14, 12 and 10 respectively given in Table 1. Let x, y, z be students' grades in mathematics, physics and literature respectively so x, y, z ∈ {a, b, c, d, e, f}. We encode the constraints on μ namely μM > μL, μP > μL, μML > μMP , μPL > μMP and μP = μM by means of generic constraints on π.

1. M is more important than L:

At first sight we encode this constraint by:

However this encoding, apparently natural, is not adequate here since it doesn't recover the ranking on U induced by Choquet integral. Let us consider the following alternatives dfe, efd, ead and dae. Following equation (4) we have π(dfe) > π(efd) and π(dae) > π(ead). However following Choquet integral we have Chμ(dfe) = 12.7, Chμ(efd) = 12.4, Chμ(dae) = 14.8 and Chμ(ead) = 15.6. So we have well dfe preferred to efd but dae is not preferred to ead. This means that constraint (4) is too weak to encode μM > μL. The reason is that the constraint μM > μL is more requirying than what it appears. Thus y should be constrained rather than to take any value in S.

Let mpl and m′ p′ l′ two vectors. Note that Chμ(mpl) > Chμ(m′ p′ l′) reduces into μM > μL when Chμ(mpl) = p + (l - p) ∗ μML + (m- l) ∗ μM > Chμ(m′p′ l′ ) = p′ + (m′ - p′ ) ∗ μML + (l′ - m′ ) ∗ μL. This supposes p ≤ l < m and p′ ≤ m′ < l′ . We put p = p′ = y, l = m′ = z and m = l′ = x. Thus μM > μL is encoded in our framework by:

• ii) P is more important than L:

The same reasoning can be made for μP > μL. It is encoded by:π(xyz) > π(xzy) for y > z ≥ x. (6)

• iii) The interaction between M and L is higher than the interaction between P and M:

The inequality μML > μMP is equivalent to the following inequality between the two Choquet integrals Chμ(mpl) = p + (l - p) ∗ μML + (m - l) ∗ μM > Chμ(m′p′ l′ ) = l′ + (p′ - l′) ∗ μMP + (m′ - p′ ) ∗ μM . This supposes p < l ≤ m and l′ < p′ ≤ m′ . Letting p = l′ = y, l = p′ = z and m = m′ = x, then μML > μMP is encoded by:

• iv) The interaction between P and L is higher than the interaction between P and M:

Similarily μPL > μMP is encoded by:

1. M and P have the same importance:

• vi) As previously said, we suppose that Pareto ordering holds. Namely

for x ≥ x′ , y ≥ y′ , z ≥ z′ , (x > x′ or y > y′ or z > z′ ).

• vii) Lastly C ≻ A ≻ B is encoded by:

In sum we have the following set of generic constraints and examples:

Applying Algorithm 1 on C gives a possibility distribution with 26 strata π = (E1, · · · ,E26) with E1 = {aaa}, E2 = {baa, aba}, E3 = {caa, aca, aab}, · · ·, E26 = {fff}. The alternatives dcc, abf and fea belong to E12, E13 and E15 respectively. Using Choquet integral we get 77 different levels: Chμ(aaa) = 18, Chμ(aba) = Chμ(baa) = 17.8, Chμ(aca) = Chμ(caa) = Chμ(fea) = 13.6, · · ·, Chμ(fff) = 10. Note also that in our approach, cce belongs to E14, i.e. we have C ≻ A ≻ D ≻ B.

6 Related works

This work is based on an idea first presented in [Dubois et al., 2005]. This proposal has been extended in two main directions. First, an algorithm is proposed that directly provides the ranking between any two alternatives (without having to compute the whole complete preorder on the set of all the alternatives). Besides, a comparative discussion on an example suggests that the proposed approach may be more robust, more flexible, and is more transparent to the user (who can control precisely what is expressed by means of the constraints) than an aggregation-based method, which moreover requires the use of a numerical scale in order to have a sufficiently discriminative scale.

Our approach focussed on the particular case where examples and generic constraints are consistent together in order to perform the comparison with the Choquet integral example, which is also based on this hypothesis. However we can deal as well with examples that contradict generic constraints. An algorithm has been proposed in [Dubois et al., 2005] where examples are considered as exceptions. This algorithm computes the complete pre-order associated to generic constraints, which is then modified in order to satisfy the examples provided that Pareto principle is not violated.

The proposed approach may look a bit similar to a topological sorting procedure that computes a linear ordering of the nodes of a directed acyclic graph (DAG). An edge from a node A to a node B means that A should be strictly preferred to B. Then several complete strict orders are generated satisfying all constraints given in the graph. Using our approach on such constraints we generate a complete pre-order that is such that if we would change equalities between alternatives into strict preferences then we would obtain one of the strict orders agreeing with the topological sort.

The recent years have seen the development of important works in Artificial Intelligence on the representations of preferences (e.g., [Wilson, 2004]). However, they start with a set of local conditional preferences and apply a ceteris paribus principle for completing the specifications. In our case, constraints are of a different nature. Indeed they refer to complete specifications of alternatives either in a generic way or by referring to particular situations.

7 Conclusion

Starting from possibility theory as a framework for representing constraints satisfaction, a qualitative method has been proposed for building a complete preorder that agrees with a set of constraints in a qualitative way. The approach is fairly general, and agrees with the way humans state their preferences in a granular manner, either in terms of generic rules or by means of examples. This is an extrapolation task not to be confused with learning. It would amount to e.g. learn importance relations between criteria from a sufficient number of examples of preferences between complete alternatives.

A topic of interest for further research would be to make a general comparison of the approach with multiple criteria aggregation techniques such as Choquet or Sugeno integrals, and to study to what extent it is possible to extract constraints underlying the way these aggregations handle the assessment of the relative importance of criteria.