A Model for Accountable Ordinal Sorting

1 Introduction

While algorithmic automated decisions or recommendations are nowadays pervasive, there is a growing demand of institutions and citizens to make these recommendations transparent and trustworthy, while system designers seek persuasive recommendations [Tintarev, 2007]. The recent regulation adopted by the European Parliament (known as the General Data Protection Regulation, GDPR) goes further by adding a "right to explanation". According to [Goodman and Flaxman, 2016] "the GDPR's requirements could require a complete overhaul of standard and widely used algorithmic techniques". We interpret this requirement in the strong sense of accountability, its litmus test being the ability of the recipient of the recommendation to defend it before other, skeptical, stakeholders of the decision (whereas trust requires the recommendation to be consistently accurate, but eventually asks for delegation of the decision to the system; transparency simply provides access to the underlying algorithm without concern for technical literacy [Burell, 2016]; and persuasiveness is hardly transferable: someone persuaded by a recommendation may not be a good persuader).

Our aim in this paper is thus to build an accountable, ordinal, multicriteria classifier, mapping a candidate object to a recommendation consisting in one or more categories among a predefined, ordered collection of these. In a multicriteria decision aiding (MCDA) context, the only indisputable relation between objects is the Pareto dominance, occurring when an object outperforms another on all criteria. As the situation is seldom so clear, the rules permitting the comparison of objects need to be enriched, taking into account the knowledge and values of the decision-maker, collected under the label preference information, which is also considered as an input of the classifier. We also consider an additional output, an explanation aimed at the decision-maker, supporting the recommendation and enabling the accountability sought for. In order to reach this goal of accountability, we make two important assumptions about the recommender system. These design principles are as follows:

No jargon. A first step in a MCDA process is to collect decision-maker's preferences information. In order to accurately represent the specific decision process, we opt for an indirect elicitation [Dias et al., 2002]: the decision-maker is never asked any questions about artifacts of the model (e.g. weights). Instead she should express preferences directly in the language of the actual decision situation, i.e. providing direct assignments of typical examples, reference objects, to categories.

No arbitrariness. MCDA usually proceeds by representing the reasoning of the decision-maker with a formal parametric model, describing a specific stance. The values of the preference parameters are often fitted during an elicitation process, up to a certain point. While many methods proceed by picking a specific, so-called representative value of the parameters, we opt for a robust approach (to the lack of preference information) [Vincke, 1999; Greco et al., 2008], formulating a -possibly partial - recommendation that cannot be refuted by any judgment function consistent with the preference information.

On top of these principles, we make three further assumptions about the MCDA model, proceeding from the willingness to keep the model accessible to human reasoning.

No compensation. This assumption deals with the interpretation of collected data -the evaluation of objects on various criteria. We assume they are always used comparatively, in a purely ordinal manner: on a given criterion, an alternative is either as good as another one, or strictly worse. Hence, only the set of criteria for which an alternative is better is important, regardless of the specific values, and being very good on some criterion cannot compensate for low performance on others. This feature enables the algorithm to proceed without performing any algebraic computation, which makes it particularly suited for explanation. It is shared with established non-compensatory ordinal sorting models used in the field of MCDA (eg. NCS) [Bouyssou and Marchant, 2007]. Moreover, the use of a 2-valued comparison (≥,<) is similar to [Bouyssou, 1986] rather than [Fishburn, 1976] who proposes a 3-valued one (<, =, >).

No values. At the heart of the recommender system is a preference structure encoding the comparison of alternatives. There are two main families of structures: those based on value [Keeney and Raiffa, 1976], and those based on outranking relations [Roy, 1991]. We opt for the latter, as they eschew the construction of a scoring function. An outranking relation naturally provides four outcomes when comparing two alternatives: preference for the former, for the latter, indifference, or incomparability; also, it does not enforce transitivity of preference.

No frontiers. In MCDA, most classifiers link the preference structure and the recommendation of a class by introducing an explicit frontier between classes, defining the limit of each class (a single value for value-based models, a limiting profile for outranking-based ones, e.g. [Leroy et al., 2011]). We do without this construct, as for instance models based on Logical Analysis of Data (LAD) techniques [Crama et al., 1988] which output classification rules. We shall use simple rules permitting to classify a new object by comparing it to a set of already classified reference objects (see Sect.2.3).

The general philosophy of these principles must be clear to the reader: accountability should exclude in principle the use of any model artifact that the decision-maker may not properly handle, but at the same time provide enough understanding of the model so as to allow the decision-maker to defend the recommendation as if it was her own. Following this, our approach is to enforce these principles by design, and to investigate how far we can get with the resulting sorting model. This approach differs from the recent work of [Ribeiro et al., 2016] which adopts a model-agnostic approach, and builds explanations adapted to virtually any classifier. They obtain extremely promising results in terms of trust. As expected, the explanation cannot be fully faithful to the model (they are "locally" faithful though). It also differs from [Datta et al., 2016] which seeks to extract how influential are input parameters, but keeping a black-box access to the model. While for the trust requirement these approaches are sufficient, our notion of accountability requires to get to grips with the model.

The rest of this paper is as follows. We propose a model implementing and formalizing the different principles, decomposing it in a learning phase (Section 2) and a recommendation phase (Section 3). We provide formal explanations of the recommendation in most cases, in the form of argument schemes tailored to represent the specific rules of the model. Section 4 introduces some insights on the description of the sorting problem through an argumentation system. Section 5 concludes the paper, by putting its findings into perspective.

2 Formal Description

In this section, we define a recommender system following the design principles and assumptions, and describe some of its properties.

2.1 The Recommender System

We consider a multicriteria ordinal sorting problem : a collection of objects are evaluated on a set of criteria N. We note B := {0, 1}, so that elements of BN are at the same time vectors with binary coordinates, and subsets of N, partially ordered by inclusion. The maximal element of BN is the unanimous coalition N, also denoted (1, . . . , 1). The minimal element of BN is the empty coalition ∅, also denoted (0, . . . , 0). Each criterion i ∈ N maps an object to a performance value in a totally ordered set Xi, the higher the better. Consequently, each object is described by a performance vector in the partially ordered set X = ∏i∈N Xi. The objects are to be assigned to some class chosen among an ordered set K = {k1 ≺ · · · ≺ kp}, so that assignment to a class with a high index is desirable.

Formally, let us describe the recommender system as a function mapping a pair 〈z, P〉 to a pair 〈K, E〉, where:

• The object z ∈ X is a candidate for sorting;
• P denotes preference information collected from the decision-maker consisting of typical classification examples, a collection of reference objects X⋆ ⊂ X, and their assigned categories Class : X⋆ → K. For syntactic reasons, we represent it by a set of object-assignment pairs P ⊂ X × K.

• K ⊂ K is the recommendation, concerning the classes that could be assigned to the candidate (see Sect. 3);
• E is an explanation yet unspecified, supporting the recommendation K (see for instance [Labreuche et al., 2012; Belahcene et al., 2017]), and addressed by Sect. 3.

2.2 The Reasoning of the Decision-Maker

A non-compensatory outranking relation can be represented by a Boolean composite function:

where the observation function ON maps a pair of objects to its concordance set, and the consistent judgment of the decision-maker, based on these concordance sets, is represented by the judgment function φ mapping a concordance set to a truth value.

Antecedents of 1 by φ, called true points in the language of the LAD [Crama et al., 1988], represent sufficient coalitions of criteria, while antecedents of 0 by φ are false points or insufficient coalitions of criteria. φ is supposed nondecreasing, meaning that a superset of a sufficient coalition of criteria is also sufficient, and a subset of an insufficient coalition is also insufficient. Compatibility of the outranking relation S to the Pareto dominance imposes that a unanimous support of criteria is always sufficient, so φ(N) = 1. Conversely, φ(∅) = 0 must hold, so the relation S is not reduced to generalized indifference. Finally, we define the set of any possible judgment function :

2.3 Learning From the Assignment Examples

To assign a new object to a category, we shall use the following classification rules:

• (R1) an object cannot outrank any object assigned to a strictly better class;
• (R2) an object outranks objects assigned to a strictly worse class;
• (R3) objects in the same class can be in any position with respect to outranking.

To account for that, we first denote ≿P the complete preorder between reference objects induced by P:

We consider the strict enforcement of the model rules for reference objects:

• (R1) : ∀x⋆ , y⋆ ∈ X⋆ , x⋆ ≻P y⋆ ⇒Not (y⋆Sφx⋆ );
• (R2) : ∀x⋆ , y⋆ ∈ X⋆ , x⋆ ≻P y⋆ ⇒ x⋆Sφy⋆ .

Hence, the assignment of reference objects expressed by P places upper (by (R1)) and lower (by (R2)) bounds upon the outranking relation between reference objects. so that:

These constraints transfer to the judgment functions. Each pair (x⋆ , y⋆ ) is mapped by the observation function ON to a coalition of criteria. The observed coalitions ON(X⋆ ×X⋆ ) serve as a learning set for the judgment function φ. They are sorted between three sets, yielding necessary conditions on φ:

• insufficient coalitions ON(≺P) should be mapped to 0;
• sufficient coalitions ON(≻P) should be mapped to 1;
• ON(∼P), which images by φ are not constrained.

Consequently, we define the set Φ(P) of judgment functions compatible to the preference information P:

Example 2. (ex. 1 continued) In the following table, we detail all the relevant observed coalitions. Sufficient coalitions appear in the upper right side, boldfaced, while insufficient coalitions are in the lower left side. N stands for unanimity, which is self-explanatory.

2.4 Consistency of Judgment

The set Φ(P) is empty if, and only if, Pareto dominance is contradicted (∃ x⋆ , y⋆ ∈ X⋆ , ∀i ∈ N, x⋆i ≥ y⋆ i and Class(x⋆ ) < Class(y⋆ )), or some coalition of criteria M ∈ BN observed as being sufficient is weaker (for inclusion) than some coalition M ′ ∈ BN observed as being insufficient. In such a case, we call the preference information P inconsistent; otherwise, it is consistent and Φ(P) is a partially defined Boolean function [Crama et al., 1988]. Combining the constraints on the judgment functions expressed by ̂Φ and by P, we can compute the true points of Φ(P). They are the antecedents of 1 common to every judgment function φ ∈ Φ(P), and represent the coalitions established as sufficient, by the virtue of being at least as strong as an observed sufficient coalition.

Conversely, the false points are the antecedents of zero common to every φ ∈ Φ(P) and represent the coalitions established as insufficient.

Proposition 1 details three manners to express inconsistency:

Proposition 1. For any P ⊂ X × K, the three following conditions are equivalent and characterize inconsistency:

1. Absence of compatible judgment function: Φ(P) = ∅
2. Conflicting constraints: TP ∩FP ̸= ∅
3. Explicit contradiction: ∃t ∈ ON(≻P), ∃f ∈ ON(≺P) : t ⊆ f

Example 3. (ex. 2 continued) Coalitions are sorted according to the observations, and monotonicity:

ON (≺P ) = {c, d, ac, bd, cd, acd, bcd}

FP = {∅, a, b, c, d, ac, ad, bc, bd, acd, bcd}

There is no dispute, as TP ∩ FP = ∅, but the coalition ab is left undecided.

3 Recommendations and Explanations

In the previous section, we saw how the decision-maker interprets pairwise comparisons between reference objects belonging to different classes as sufficient or insufficient coalitions of criteria. Here comes a new candidate, z ∈ X. It gauges every reference object in X⋆ , yielding |P| observations -→o (z,P) := ⋃x⋆∈X⋆ ON(z, x⋆ ), and is also evaluated by every reference object, yielding |P| other observations ←-o (z,P) := ⋃x⋆∈X⋆ ON(x⋆ , z). Each of these 2|P| observations is interpreted as a sufficient, insufficient or undecided coalition of criteria.

Example 4. (ex. 3 continued) The following table augments the one presented in example 2 with the coalitions resulting from comparisons between the reference objects A1, A2, B1, B2, C1, C2 and the candidates X, Y .

A1 A2 B1 B2 C1 C2 X Y

A1 - - abc abd abc abd N N

A2 - - N abd N abd N acd

B1 d bd - - abd abd N ad

B2 acd ac - - abc abd acd acd

C1 d d acd bd - - acd ad

C2 cd c c bcd - - cd cd

Non-bracketed coalitions have already been sorted according to the preference information: boldfaced coalitions are those previously established as sufficient, the others are insufficient. Bracketed coalitions are yet undecided. ∀z ∈ {X, Y }, -→o (z,P) appears in the corresponding line, and ←-o (z, P) in the appropriate column.

In this section, we specify the mapping between these observations and the output of the classifier system, the recommendation K(z,P) ⊂ K and an explanation E(k,P) supporting it.

3.1 Possible Assignments

As defined by the works of [Greco et al., 2010] about necessary and possible preference relations, the definition of possible assignments is closely related to the notion of consistency of an assignment with respect to the corpus of preference information. Defining, as we did in Section 2, Φ(P) as the set of preference parameters compatible to P, and assuming it is not empty:

• necessary assignments are yielded by every possible completion of these preference parameters;
• possible assignments are yielded by some possible completion of these preference parameters;
• impossible assignments are yielded by no possible completion of these preference parameters;

These sets of assignments are concisely described referring

A possible assignment is in ̂K(z,P), an impossible one is not. When ̂K(z,P) boils down to a singleton, then it is a necessary assignment for z.

This definition of possible assignment is straightforward to implement, simply iterating through the set of possible assignments classes k ∈ K, updating the preference information P ′ ← P ∪ {(z, k)}, and checking the consistency of P ′ . Unfortunately, it is a tricky notion when it comes to explaining. The actual unveiling of a Boolean judgment function compatible to the assignment is not very appealing, as it introduces at the same time elements of jargon -describing the judgment of the decision-maker as the partition of coalitions of criteria between sufficient and insufficient- and arbitrariness, as some coalitions may very well be undecided and should remain so. Consequently, we adopt the following principle: "Everything is possible, unless proven otherwise".

Doing so shifts the burden of proof towards impossibility, focusing on the exhibition of constraints restricting the set ̂K(z,P). We aim at explaining these constraints thanks to statements of the form [premises : conclusions]scheme. We define several argument schemes, as formalized by [Walton, 1996] in order to capture stereotypical patterns of human reasoning. These schemes specify the nature and conditions imposed to both premises and conclusions, yielding to valid arguments. We are looking for complete explanations, so we must ensure the validity of the implication premises ⇒ conclusions, and provide grounded sets of statements, such that any premise is either the conclusion of another argument, or directly referencing the assumed available information (pairwise comparisons between the reference objects or the candidate, based on criteria or assignment).

In order to make apparent the cause of impossibility, we consider the potential consequences of assigning a candidate to a class through the additional (in)sufficient coalitions conditional to the assignment of the candidate z to the class k:

∆TP(z, k) := TP∪{(z,k)}\TP; ∆FP(z, k) := FP∪{(z,k)}\FP We rewrite the impossibility of assigning the candidate z to the class k using the conflicting constraints characterization of inconsistency (see Prop. 1). We consider three potential sources of impossibility, sorted by evidence: ̂K(z,P) = ⋂i∈{1,2,3} Ki(z, P) where:

• K1(z, P) := {k ∈ K : TP ∩∆FP(z, k) = ∅} highlights conflicts between established sufficient coalitions, and the assignment of z;
• K2(z, P) := {k ∈ K : ∆TP(z, k)∩FP = ∅} highlights conflicts between established insufficient coalitions, and the assignment of z;
• K3(z,P) := {k ∈ K : ∆TP(z, k) ∩ ∆FP(z, k) = ∅} takes into account the least obvious situation where some assignment of z may be self-contradictory, without conflicting with any previously acknowledged information.

The next section details the impossibilities captured by the set K1(z,P), and proposes a supporting explanation E1(z,P), while the other cases are briefly presented in section 3.3.

3.2 Assignments Contradicting Previously Established Sufficient Coalitions

In this section, we focus on the set K1(z,P) := {k ∈ K : TP ∩∆FP(z, k) = ∅}. As seen in the previous section this set provides a range of possible assignments for the candidate z, and partially implements the model described by the manifesto exposed in the introduction. We first describe K1(z, P) as an intersection of constraints, for which we provide a description based on arguments. We prove K1(z,P) is an interval of K, and provide a short, yet complete, explanation accounting for this recommendation.

For increased readability, we introduce notations for particular sets of classes. For k ∈ K, let K≾ k (resp. K≿ k) the interval of classes not greater (resp. not lower) than k.

By construction, the recommended set K1(z, P) is built in order to reject some impossible assignments. To illustrate and understand its behavior, we make up a situation that specifically triggers this rejection flag. Suppose we know that:

1. the coalition of criteria T ∈ BN is already known to be sufficient, and
2. the candidate z ∈ X is at least as good as the reference object x⋆ ∈ X⋆ , assigned to class k ∈ K, for all criteria in T .

Then, z outranks x⋆ and cannot be assigned to a class strictly worse than k by application of (R1). This constraint is captured by the set K1(z, P), as the assignment of z to any class k ≺ k would lead to conclude that the coalition of criteria ON(z, x⋆ ) is insufficient, so that the coalition of criteria T would belong to both sets ∆FP(z, k) and TP. Consequently, k /∈ K1(z,P).

If we replace the assumption (2) by:

• (2') the reference object x⋆ ∈ X⋆ , assigned to class k ∈ K, is at least as good as the candidate z ∈ X for all criteria in T .

then x⋆ ∈ X⋆ outranks z and z cannot be assigned to a class strictly better than k, as

Reciprocally, any assignment k0 /∈ K1(z,P) results in a non-empty intersection TP ∩ ∆FP(z, k0), which involves at least one sufficient coalition T ∈ TP, as in assumption (1), and one stronger, insufficient coalition resulting either from the observations -→o (z, P), as in assumption (2), or from ←-o (z,P), as in (2').

A statement of type (1) needs to be backed by evidence, so we introduce two argument schemes:

Definition 1. For any reference objects a⋆ , b⋆ ∈ X⋆ and any coalition of criteria T ∈ BN , we say the tuple [a⋆ , b⋆ : T]T instantiates the argument scheme SUFFICIENT COALITION(P) if, and only if, T ⊇ ON(a⋆ , b⋆ ) and a⋆ ≻P b⋆ . We also say the tuple [∅ : N]1 instantiates the argument scheme WEAK DOMINANCE.

Proposition 2 (Argumentative structure of the sufficient coalitions).

The sufficient coalitions are exactly the conclusions of the arguments instantiating the SUFFICIENT COALITION(P) scheme.

In order to account for the atoms of reasoning (2) and (2') and present them to the recipient of the recommendation, we define the corresponding argument schemes.

Definition 2. For any coalition of criteria T ∈ BN , any reference object x⋆ ∈ X⋆ and any class c ∈ K, we say that:

• the tuple [T, x⋆ : K≿ c ]T /-→o instantiates the argument scheme OUTRANKING(z, P) if, and only if, T ∈ TP and ∀i ∈ T, zi ≥ x⋆i and class(x⋆ ) = c.
• the tuple [T, x⋆ : K≾ c ]T /←-o instantiates the argument scheme OUTRANKED(z,P) if, and only if, T ∈ TP and ∀i ∈ T, x⋆i ≥ zi and class(x⋆ ) = c

Proposition 3 (Argumentative structure of the recommendation).

Proposition 3 is a concise rewording of the necessary and sufficient conditions for a given class not to belong to the set K1(z,P) detailed previously. As a corollary, it shows that K1(z, P) is an interval of K. Consequently, K1(z, P) can be completely described by a pair (k1, k1) such that:

• K1(z, P) = K≿ k1 ∩ K≾ k1
• the lower bound k1 is maximal, as there is no class strictly better than k1 which is supported by an argument instantiating the OUTRANKING(z,P) scheme. It is trivial if k1 = minK (either when the set OUTRANKING(z,P) is empty, or when it does not support a stronger outcome), in which case it does not need any explanation. If k1 ≻ minK, then it admits at least one explanation E1 composed of an argument [T, x⋆ : K≿ k1 ]T /-→o ∈ OUTRANKING backed by an argument [a⋆ , b⋆ : T]T ∈ SUFFICIENT COALITION;
• the upper bound k1 is minimal, as there is no class strictly worse than k1 which is supported by an argument instantiating the OUTRANKED(z,P) scheme. It is trivial if k1 = maxK, in which case it does not need any explanation. If k1 ≺ maxK, then it admits at least one explanation E1 composed of an argument [T ′ , x⋆ : K≾ k1 ]T /←-o ∈ OUTRANKED backed by an argument [a⋆ , b⋆ : T ′ ]T ∈ SUFFICIENT COALITION.

Finally, the recommended interval K1(z,P) is supported by an explanation E1 in the form of a pair (E1,E1), where E1 and E1 can be either the empty set or a pair of arguments. Taken together, all these 0, 2 or 4 arguments prove that any assignment k ∈ K \ K1(z, P) should be rejected as "impossible". Such explanation is not necessarily unique, and we denote by ̂ E1(z, P) the set of suitable explanations.

Example 5. (ex. 4 continued)

Using the table presented in Example 4, the set K1 can be interpreted as "a candidate cannot be assigned a class laying strictly on the right of, nor a class strictly above, a case containing a boldfaced coalition": Consequently,

• { K1(X, P) = {⋆, ⋆⋆} E1(X, P) ∋ (∅, {[∅ : N]1, [N, B1 : ≾ ⋆⋆]T /←-o }) X cannot be ranked higher than ⋆⋆, because B1 is rated ⋆⋆ and dominates X.
• { K1(Y, P) = {⋆⋆, ⋆ ⋆ ⋆} E1(Y, P) ∋ ({[A1, C1 : abc]T , [abc, B1 : ≿ ⋆⋆]T /-→o }, ∅)
• ̂ Y cannot be ranked lower than ⋆⋆, because it outranks B1. Indeed, Y compares to B1 the same way as A1 to C1: it is at least as good on the sufficient coalition of criteria abc.

3.3 Other Impossible Assignments

The set K2(z,P) is defined symmetrically from K1(z,P) w.r.t. sufficient and insufficient coalitions. Assignments not in K2(z,P) result from the collision of a coalition of criteria known to be insufficient, and the observation of a candidate object resulting in an even weaker coalition, so outranking is excluded, and all the classes strictly above or below (depending on the direction of observation) the one of the reference object are therefore forbidden. Mutatis mutandis, we can define the argument schemes INSUFFICIENT COALITION(P), WEAKLY DOMINATED, NOT OUTRANKING(z, P), NOT OUTRANKED(z, P) and obtain the same structural results, leading to define similar explanations for the lower and upper bounds of the interval K2(z, P).

Example 6. (ex. 4 continued)

Using the table presented in Ex. 4, the set K2 interprets the insufficient coalitions of the table, those not boldfaced nor parenthesized. A candidate cannot be assigned a class strictly below, nor strictly on the left, of such cases. For instance, ON(B2,X) = acd ∈ FP (e.g. because ON(C1 ≺P B1) = acd), so X is not outranked by B2 and should be at least assigned the same class (⋆⋆), and ON(X,B2) = bd ∈ FP (e.g. because it is weaker than bcd = ON(C2 ≺P B2)), so X does not outrank B2 and should not be assigned a strictly better class (⋆⋆). In terms of preference, objects X and B2 are incomparable, and thus should be assigned the same class. Finally, K2(X, P) = {⋆⋆}.

The set K3(z,P) excludes inconsistent judgments on yet undecided coalitions of criteria. There is no guarantee that K3(z,P) has an interval structure. We omit this case due to space limitations.

4 An argumentative Perspective

Along this paper, we proposed the construction of explanations supporting results of a multi-criteria sorting problem, as combinations of arguments schemes. Each instantiation of one of the six previous main schemes (see Def. 1, 2 and their symmetrical forms) provides one type of argument. These arguments may be conflicting, and two different relations can be distinguished:

Conflicting coalitions: we have evidence indicating that a given coalition is potentially at the same time sufficient and insufficient (i.e. there are two coalitions t ⊆ f such that [a⋆ , b⋆ : t]T and [c⋆ , d⋆ : f]F). This situation represents an explicit contradiction corresponding to an inconsistency situation (see Sec. 2.4). Such conflicts are not illustrated through the previous examples, however inconsistencies are classical situations within decision problems, as it concerns a human decision-maker.

Conflicting classification: it may occur that, for some candidate, arguments based on the outranking relation point towards an empty interval of possible assignments. This situation corresponds to the fact that the sets K1(z,P) and K2(z,P) are disjoint, which may happen when either is empty, or when the lower bound of one exceed the upper bound of the other.

Example 7. (ex. 4 cont.) Y andA2 are incomparable, Y and B2 are incomparable, yet A2 is preferred to B2. In particular, A2 (⋆⋆⋆) does not outrank Y and Y does not outrank B2 (⋆⋆) so K2(Y, P) = ∅.

The impossibility to provide any recommendation is clearly critical from the point of view of decision aiding. These unfortunate situations cannot be ruled out in the general case, as they may stem from Condorcet paradoxes (failures of transitivity) concerning the necessary outranking relation or the necessary not-outranking relation (see e.g. [K¨oksalan et al., 2009] for a discussion).

The argumentative treatment for our multi-criteria ordinal sorting problem is thus to construct arguments pro and against each possible assignment (of the reference object and the candidate), and to determine among conflicting arguments the acceptable ones. This can be done by taking two different perspectives. One way is to rely on the work of [Dung, 1995] - the next question being to identify which semantics are appropriate in our situation. This is close in spirit to an approach presented in [Amgoud and Serrurier, 2007] for classification in unordered classes (however in our context the relation between arguments would be symmetric [Coste-Marquis et al., 2005]). Another perspective is to consider the construction of the argumentation system as a dialogue game and to rely on critical questions [Walton, 1996; Ouerdane et al., 2008] to evaluate the arguments. This perspective has the advantage to keep the decision-maker in the loop, which is often essential in a decision situation [Labreuche et al., 2015]. Both approaches look promising and are made possible thanks to the modeling presented in this paper.

5 Conclusion

We have presented a fully accountable multi-criteria ordinal sorting model, based on several design principles and assumptions. The strength of the model is that it solely relies on a simple set of classification rules, which means that each recommendation can be justified by instantiating and combining these rules-nothing else. Several argument schemes have been proposed for that purpose. Interestingly, some of these schemes have a flavour of analogical reasoning, which was studied in the context of classification [Hug et al., 2016]. Now the simplicity of our model comes at a price: there are different situations where inconsistency might occur, and the model is not equipped yet to handle such situations. Facing this issue we can take two stances. The first one is to relax some of our design assumptions. For instance, we may decide that it is actually acceptable for the model to use a frontier between classes (allowing to eschew the Condorcet paradox). This would require original explanation techniques to maintain the desired accountability. Another avenue is to handle the inconsistencies thanks to defeasible and non-monotonic reasoning techniques [Brewka et al., 2008]. Our discussion in Section 4 points to formal argumentation as a natural and promising opportunity for future research.