Potentially all pairwise rankings of all possible alternatives

Last updated

Potentially All Pairwise RanKings of all possible Alternatives (PAPRIKA) is a method for multi-criteria decision making (MCDM) or conjoint analysis, [1] [2] [3] as implemented by decision-making software and conjoint analysis products 1000minds and MeenyMo. [4] [5] [6]

Contents

The PAPRIKA method is based on users expressing their preferences with respect to the relative importance of the criteria or attributes of interest for the decision or choice at hand by pairwise comparing (ranking) alternatives.

In MCDM applications, PAPRIKA is used by decision-makers to determine weights on the criteria for the decision being made, representing their relative importance. Depending on the application, these weights are used to rank, prioritize or choose between alternatives.

In conjoint analysis applications, PAPRIKA is used with consumers or other stakeholders to estimate 'part-worth utilities' (i.e. weights) representing the relative importance of the attributes characterizing products or other objects of interest (i.e., choice modelling, conjoint analysis and discrete choice). [2] [3]

Applications

The PAPRIKA method is implemented by decision-making software and conjoint analysis products 1000minds and MeenyMo. [4] [5] [6]

Examples of areas in which the method is used for multi-criteria decision making or conjoint analysis include (see also 1000minds applications):

Additive multi-attribute value models

The PAPRIKA method specifically applies to additive multi-attribute value models with performance categories [69] – also known as 'points', 'scoring', 'point-count' or 'linear' systems or models. The following explanations are mostly couched in terms of multi-criteria decision making. Analogous explanations in terms of conjoint analysis are possible but not presented here.

As the name implies, additive multi-attribute value models with performance categories – hereinafter referred to simply as 'value models' – consist of multiple criteria (or 'attributes'), with two or more performance categories (or 'levels') within each criterion, that are combined additively.

Each category within each criterion is worth a certain number of points that is intended to reflect both the relative importance ('weight') of the criterion and its degree of achievement. For each alternative being considered, the point values are summed across the criteria to get a total score – hence, these are additive value models – by which the alternatives are prioritized or ranked (or otherwise classified) relative to each other.

Thus, a value model (or 'points system') is simply a schedule of criteria (and categories) and point values for the decision problem at hand; for an example, see Table 1 in the sub-section below. This 'points system' representation is equivalent to a more traditional approach involving normalized criterion weights and 'single-criterion value functions' to represent the relative importance of the criteria and to combine values overall (see weighted sum model). The unweighted points system representation is easier to use and helps inform the explanation of the PAPRIKA method below.

An example application of a points system

An example application of a points system is ranking candidates applying for a job.

Imagine that 'Maartje', 'Michelle' and 'Paulien' are three job candidates to be ranked using the value model in Table 1 below. Suppose they are assessed on the five criteria (see Table 1) like this:

Table 1: Example of a value model (points system) for ranking job candidates

CriterionCategoryPoints
Educationpoor0
good8
very good20
excellent40
Experience< 2 years0
2–5 years3
> 5 years10
Referencespoor0
good27
Social skillspoor0
good10
Enthusiasmpoor0
good13

Summing the point values in Table 1 corresponding to the descriptions for Maartje, Michelle and Paulien gives their total scores:

Clearly, Paulien has the highest total score. Therefore, according to the value model (and how Maartje, Michelle and Paulien were assessed) Paulien is the best candidate for the job. (Though, clearly, relative to other candidates who could potentially have applied, Paulien is not as good as the best hypothetically-possible candidate – who would score a 'perfect' 40 + 10 + 27 + 10 + 13 = 100 points.)

In general terms, having specified the criteria and categories for a given value model, the challenge is to derive point values that accurately reflect the relative importance of the criteria and categories to the decision-maker. Deriving valid and reliable point values is arguably the most difficult task when creating a value model. The PAPRIKA method does this based on decision-makers' preferences as expressed using pairwise rankings of alternatives.

Overview of the PAPRIKA method

As mentioned at the start of the article, PAPRIKA is a (partial) acronym for 'Potentially All Pairwise RanKings of all possible Alternatives'. The following explanation should make clear this name's derivation.

The PAPRIKA method pertains to value models for ranking particular alternatives that are known to decision-makers (e.g. as in the job candidates example above) and also to models for ranking potentially all hypothetically possible alternatives in a pool that is changing over time (e.g. patients presenting for medical care). The following explanation is centered on this second type of application because it is more general.

PAPRIKA is based on the fundamental principle that an overall ranking of all possible alternatives representable by a given value model – i.e. all possible combinations of the categories on the criteria – is defined when all pairwise rankings of the alternatives vis-à-vis each other are known (and provided the rankings are consistent).

(As an analogy, suppose you wanted to rank everyone living in a given city from the youngest to the oldest. If you knew how each person was pairwise ranked relative to everyone else with respect to their ages – i.e. for each possible pair of individuals, you identified who is the younger of the two individuals or that they're the same age – then you could produce an overall ranking of the city's population from the youngest to the oldest.)

However, depending on the number of criteria and categories included in the value model, the number of pairwise rankings of all possible alternatives is potentially in the millions or even billions. Of course, though, many of these pairwise rankings are automatically resolved due to one alternative in the pair having a higher category for at least one criterion and none lower for the other criteria than for the other alternative – known as 'dominated pairs'.

But this still leaves potentially millions or billions of 'undominated pairs' – pairs of alternatives where one has a higher ranked category for at least one criterion and a lower ranked category for at least one other criterion than the other alternative – and hence a judgment is required for the alternatives to be pairwise ranked. With reference to the example of ranking job candidates in the previous section, an example of an undominated pair (of candidates) would be where one person in the pair is, say, highly educated but inexperienced whereas the other person is uneducated but highly experienced, and so a judgement is required to pairwise rank this (undominated) pair.

For n possible alternatives, there are n(n−1)/2 pairwise rankings. For example, for a value model with eight criteria and four categories within each criterion, and hence 48 = 65,536 possible alternatives, there are 65,536 x 65,535 / 2 = 2,147,450,880 pairwise rankings. Even after eliminating the 99,934,464 dominated pairs, there are still 2,047,516,416 undominated pairs to be ranked. [1] Clearly, performing anywhere near this number of pairwise rankings – more than two billion! – is humanly impossible without a special method.

The PAPRIKA method resolves this 'impossibility' problem by ensuring that the number of pairwise rankings that decision-makers need to perform is kept to a minimum – i.e. only a small fraction of the potentially millions or billions of undominated pairs – so that the burden on decision-makers is minimized and the method is practicable. PAPRIKA keeps the number of pairwise rankings performed by decision-makers to a minimum by, for each undominated pair explicitly ranked by decision-makers, identifying (and eliminating) all undominated pairs implicitly ranked as corollaries of this and other explicitly ranked pairs. Fundamental to the efficiency of the method is application of the transitivity property of additive value models, as illustrated in the simple demonstration later below.

The PAPRIKA method begins with the decision-maker pairwise ranking undominated pairs defined on just two criteria at-a-time (where, in effect, all other criteria's categories are pairwise identical). Again with reference to the example of ranking job candidates, an example of such a pairwise-ranking question is: "Who would you prefer to hire, someone whose education is poor but he or she has 5 years or moreexperience or another person whose education is excellent but he or she has less than 2 yearsexperience, all else being equal?" (see Figure 1).

Figure 1: Example of a pairwise-ranking question (a screenshot from 1000minds)

1000minds trade-off.png

Each time the decision-maker ranks a pair (such as the example above), all undominated pairs implicitly ranked as corollaries are identified and discarded. After having completed ranking undominated pairs defined on just two criteria at-a-time, this is followed, if the decision-maker chooses to continue (she can stop at any time), by pairs with successively more criteria (i.e. three criteria, then four, then five, etc.), until potentially all undominated pairs are ranked.

Thus, Potentially All Pairwise RanKings of all possible Alternatives (hence the PAPRIKA acronym) are identified as either: (1) dominated pairs (given), or (2) undominated pairs explicitly ranked by the decision-maker, or (3) undominated pairs implicitly ranked as corollaries. From the explicitly ranked pairs, point values (weights) are obtained via linear programming; although multiple solutions to the linear program are possible, the resulting point values all reproduce the same overall ranking of alternatives.

Simulations of PAPRIKA's use reveal that if the decision-maker stops after having ranked undominated pairs defined on just two criteria at-a-time, the resulting overall ranking of all possible alternatives is very highly correlated with the decision-maker's 'true' overall ranking obtained if all undominated pairs (involving more than two criteria) were ranked. [1]

Therefore, for most practical purposes decision-makers are unlikely to need to rank pairs defined on more than two criteria, thereby reducing the burden on decision-makers. For example, approximately 95 explicit pairwise rankings are required for the value model referred to above with eight criteria and four categories each (and 2,047,516,416 undominated pairs to be ranked); 25 pairwise rankings for a model with five criteria and three categories each; and so on. [1] The real-world applications of PAPRIKA referred to earlier suggest that decision-makers are able to rank comfortably more than 50 and up to at least 100 pairs, and relatively quickly, and that this is sufficient for most applications.

Theoretical antecedents

The PAPRIKA method's closest theoretical antecedent is Pairwise Trade-off Analysis, [70] a precursor to Adaptive Conjoint Analysis in marketing research. [71] Like the PAPRIKA method, Pairwise Trade-off Analysis is based on the idea that undominated pairs that are explicitly ranked by the decision-maker can be used to implicitly rank other undominated pairs. Pairwise Trade-off Analysis was abandoned in the late 1970s, however, because it lacked a method for systematically identifying implicitly ranked pairs.

The ZAPROS method (from Russian for 'Closed Procedure Near References Situations') was also proposed; [72] however, with respect to pairwise ranking all undominated pairs defined on two criteria "it is not efficient to try to obtain full information". [73] As explained in the present article, the PAPRIKA method overcomes this efficiency problem.

A simple demonstration of the PAPRIKA method

The PAPRIKA method can be easily demonstrated via the simple example of determining the point values (weights) on the criteria for a value model with just three criteria – denoted by 'a', 'b' and 'c' – and two categories within each criterion – '1' and '2', where 2 is the higher ranked category. [1]

This value model's six-point values (two for each criterion) can be represented by the variables a1, a2, b1, b2, c1, c2 (a2 > a1, b2 > b1, c2 > c1), and the eight possible alternatives (23 = 8) as ordered triples of the categories on the criteria (abc): 222, 221, 212, 122, 211, 121, 112, 111. These eight alternatives and their total score equations – derived by simply adding up the variables corresponding to the point values (which are as yet unknown: to be determined by the method being demonstrated here) – are listed in Table 2.

Undominated pairs are represented as '221 vs (versus) 212' or, in terms of the total score equations, as 'a2 + b2 + c1 vs a2 + b1 + c2', etc. [Recall, as explained earlier, an 'undominated pair' is a pair of alternatives where one is characterized by a higher ranked category for at least one criterion and a lower ranked category for at least one other criterion than the other alternative, and hence a judgement is required for the alternatives to be pairwise ranked. Conversely, the alternatives in a 'dominated pair' (e.g. 121 vs 111 – corresponding to a1 + b2 + c1 vs a1 + b1 + c1) are inherently pairwise ranked due to one having a higher category for at least one criterion and none lower for the other criteria (and no matter what the point values are, given a2 > a1, b2 > b1 and c2 > c1, the pairwise ranking will always be the same).]

'Scoring' this model involves determining the values of the six-point value variables (a1, a2, b1, b2, c1, c2) so that the decision-maker's preferred ranking of the eight alternatives is realized.

For many readers, this simple value model can perhaps be made more concrete by considering an example to which most people can probably relate: a model for ranking job candidates consisting of the three criteria (for example) (a) education, (b) experience, and (c) references, each with two 'performance' categories, (1) poor or (2) good. (This is a simplified version of the illustrative value model in Table 1 earlier in the article.)

Accordingly, each of this model's eight possible alternatives can be thought of as being a 'type' (or profile) of candidate who might ever, hypothetically, apply. For example, '222' denotes a candidate who is good on all three criteria; '221' is a candidate who is good on education and experience but poor on references; '212' a third who is good on education, poor on experience, and good on references; etc.

Finally, with respect to undominated pairs, 221 vs 212, for example, represents candidate 221 who has good experience and poor references whereas 212 has the opposite characteristics (and they both have good education). Thus, which is the better candidate ultimately depends on the decision-maker's preferences with respect to the relative importance of experience vis-à-vis references.

Table 2: The eight possible alternatives and their total-score equations

AlternativeTotal-score equation
222a2 + b2 + c2
221a2 + b2 + c1
212a2 + b1 + c2
122a1 + b2 + c2
211a2 + b1 + c1
121a1 + b2 + c1
112a1 + b1 + c2
111a1 + b1 + c1

Identifying undominated pairs

The PAPRIKA method's first step is to identify the undominated pairs. With just eight alternatives this can be done by pairwise comparing all of them vis-à-vis each other and discarding dominated pairs.

This simple approach can be represented by the matrix in Figure 2, where the eight possible alternatives (in bold) are listed down the left-hand side and also along the top. Each alternative on the left-hand side is pairwise compared with each alternative along the top with respect to which of the two alternatives is higher ranked (i.e. in the present example, which candidate is more desirable for the job). The cells with hats (^) denote dominated pairs (where no judgement is required) and the empty cells are either the central diagonal (each alternative pairwise ranked against itself) or the inverse of the non-empty cells containing the undominated pairs (where a judgement is required).

Figure 2: Undominated pairs identified by pairwise comparing the eight possible alternatives (bolded)

vs222221212122112121211111
222^^^^^^^
221(i) b2 + c1 vs b1 + c2(ii) a2 + c1 vs a1 + c2(iv) a2 + b2 + c1 vs a1 + b1 + c2^^^
212(iii) a2 + b1 vs a1 + b2^(v) a2 + b1 + c2 vs a1 + b2 + c1^^
122^^(vi) a1 + b2 + c2 vs a2 + b1 + c1^
112(*i) b1 + c2 vs b2 + c1(*ii) a1 + c2 vs a2 + c1^
121(*iii) a1 + b2 vs a2 + b1^
211^
111

Figure 2 notes: ^ denotes dominated pairs. The undominated pairs are labelled with Roman numerals; the three with asterisks are duplicates of pairs (i)-(iii).

As summarized in Figure 2, there are nine undominated pairs (labelled with Roman numerals). However, three pairs are duplicates after any variables common to a pair are 'cancelled' (e.g. pair *i is a duplicate of pair i, etc.). Thus, there are six unique undominated pairs (without asterisks in Figure 2, and listed later below).

The cancellation of variables common to undominated pairs can be illustrated as follows. When comparing alternatives 121 and 112, for example, a1 can be subtracted from both sides of a1 + b2 + c1 vs a1 + b1 + c2. Similarly, when comparing 221 and 212, a2 can be subtracted from both sides of a2 + b2 + c1 vs a2 + b1 + c2. For both pairs this leaves the same 'cancelled' form: b2 + c1 vs b1 + c2.

Formally, these subtractions reflect the 'joint-factor' independence property of additive value models: [74] the ranking of undominated pairs (in uncancelled form) is independent of their tied rankings on one or more criteria. Notationally, undominated pairs in their cancelled forms, like b2 + c1 vs b1 + c2, are also representable as _21 vs _12 – i.e. where '_' signifies identical categories for the identified criterion.

In summary, here are the six undominated pairs for the value model:

(i) b2 + c1 vs b1 + c2
(ii) a2 + c1 vs a1 + c2
(iii) a2 + b1 vs a1 + b2
(iv) a2 + b2 + c1 vs a1 + b1 + c2
(v) a2 + b1 + c2 vs a1 + b2 + c1
(vi) a1 + b2 + c2 vs a2 + b1 + c1

The task is to pairwise rank these six undominated pairs, with the objective that the decision-maker is required to perform the fewest pairwise rankings possible (thereby minimizing the burden on the decision-maker).

Ranking undominated pairs and identifying implicitly ranked pairs

Undominated pairs with just two criteria are intrinsically the least cognitively difficult for the decision-maker to pairwise rank relative to pairs with more criteria. Thus, arbitrarily beginning here with pair (i) b2 + c1 vs b1 + c2, the decision-maker is asked: "Which alternative do you prefer, _21 or _12 (i.e. given they're identical on criterion a), or are you indifferent between them?" This choice, in other words, is between a candidate with good experience and poor references and another with poor experience and good references, all else the same.

Suppose the decision-maker answers: "I prefer _21 to _12" (i.e. good experience and poor references is preferred to poor experience and good references). This preference can be represented by '_21 ≻_12', which corresponds, in terms of total score equations, to b2 + c1 > b1 + c2 [where ≻ and '~' (used later) denote strict preference and indifference respectively, corresponding to the usual relations '>' and '=' for the total score equations].

Central to the PAPRIKA method is the identification of all undominated pairs implicitly ranked as corollaries of the explicitly ranked pairs. Thus, given a2 > a1 (i.e. good education≻poor education), it is clear that (i) b2 + c1 > b1 + c2 (as above) implies pair (iv) (see Figure 2) is ranked as a2 + b2 + c1 > a1 + b1 + c2. This result reflects the transitivity property of (additive) value models. Specifically, 221≻121 (by dominance) and 121≻112 (i.e. pair i _21≻_12, as above) implies (iv) 221≻112; equivalently, 212≻112 and 221≻212 implies 221≻112.

Next, corresponding to pair (ii) a2 + c1 vs a1 + c2, suppose the decision-maker is asked: "Which alternative do you prefer, 1_2 or 2_1 (given they're identical on criterion b), or are you indifferent between them?" This choice, in other words, is between a candidate with poor education and good references and another with good education and poor references, all else the same.

Suppose the decision-maker answers: "I prefer 1_2 to 2_1" (i.e. poor education and good references is preferred to good education and poor references). This preference corresponds to a1 + c2 > a2 + c1. Also, given b2 > b1 (good experience≻poor experience), this preference/inequality implies pair (vi) is ranked as a1 + b2 + c2 > a2 + b1 + c1.

Furthermore, the two explicitly ranked pairs (i) b2 + c1 > b1 + c2 and (ii) a1 + c2 > a2 + c1 imply pair (iii) is ranked as a1 + b2 > a2 + b1. This result can easily be seen by adding the corresponding sides of the inequalities for pairs (i) and (ii) and cancelling common variables. Again, this result reflects the transitivity property: (i) 121≻112 and (ii) 112≻211 implies (iii) 121≻211; equivalently, 122≻221 and 221≻212 implies 122≻212.

As a result of two explicit pairwise comparisons – i.e. explicitly performed by the decision-maker – five of the six undominated pairs have been ranked. The decision-maker may cease ranking whenever she likes (before all undominated pairs are ranked), but let's suppose she continues and ranks the remaining pair (v) as a2 + b1 + c2 > a1 + b2 + c1 (i.e. in response to an analogous question to the two spelled out above).

Thus, all six undominated pairs have been ranked as a result of the decision-maker explicitly ranking just three:

(i) b2 + c1 > b1 + c2
(ii) a1 + c2 > a2 + c1
(v) a2 + b1 + c2 > a1 + b2 + c1

The overall ranking of alternatives and point values

Because the three pairwise rankings above are consistent – and alln (n−1)/2 = 28 pairwise rankings (n = 8) for this simple value model are known – a complete overall ranking of all eight possible alternatives is defined (1st to 8th): 222, 122, 221, 212, 121, 112, 211, 111.

Simultaneously solving the three inequalities above (i, ii, v), subject to a2 > a1, b2 > b1 and c2 > c1, gives the point values (i.e. the 'points system'), reflecting the relative importance of the criteria to the decision-maker. For example, one solution is: a1 = 0, a2 = 2, b1 = 0, b2 = 4, c1 = 0 and c2 = 3 (or normalized so the 'best' alternative, 222, scores 100 points: a1 = 0, a2 = 22.2, b1 = 0, b2 = 44.4, c1 = 0 and c2 = 33.3).

Thus, in the context of the example of a value model for ranking job candidates, the most important criterion is revealed to be (good) experience (b, 4 points) followed by references (c, 3 points) and, least important, education (a, 2 points). Although multiple solutions to the three inequalities are possible, the resulting point values all reproduce the same overall ranking of alternatives as listed above and reproduced here with their total scores:

1st 222: 2 + 4 + 3 = 9 points (or 22.2 + 44.4 + 33.3 = 100 points normalized) – i.e. total score from adding the point values above.
2nd 122: 0 + 4 + 3 = 7 points (or 0 + 44.4 + 33.3 = 77.8 points normalized)
3rd 221: 2 + 4 + 0 = 6 points (or 22.2 + 44.4 + 0 = 66.7 points normalized)
4th 212: 2 + 0 + 3 = 5 points (or 22.2 + 0 + 33.3 = 55.6 points normalized)
5th 121: 0 + 4 + 0 = 4 points (or 0 + 44.4 + 0 = 44.4 points normalized)
6th 112: 0 + 0 + 3 = 3 points (or 0 + 0 + 33.3 = 33.3 points normalized)
7th 211: 2 + 0 + 0 = 2 points (or 22.2 + 0 + 0 = 22.2 points normalized)
8th 111: 0 + 0 + 0 = 0 points (or 0 + 0 + 0 = 0 points normalized)

Additional considerations

First, the decision-maker may decline to explicitly rank any given undominated pair (thereby excluding it) on the grounds that at least one of the alternatives considered corresponds to an impossible combination of the categories on the criteria. Also, if the decision-maker cannot decide how to explicitly rank a given pair, she may skip it – and the pair may eventually be implicitly ranked as a corollary of other explicitly ranked pairs (via transitivity).

Second, in order for all undominated pairs to be ranked, the decision-maker will usually be required to perform fewer pairwise rankings if some indicate indifference rather than strict preference. For example, if the decision-maker had ranked pair (i) above as _21~_12 (i.e. indifference) instead of _21≻_12 (as above), then she would have needed to rank only one more pair rather than two (i.e. just two explicitly ranked pairs in total). On the whole, indifferently ranked pairs generate more corollaries with respect to implicitly ranked pairs than pairs that are strictly ranked.

Finally, the order in which the decision-maker ranks the undominated pairs affects the number of rankings required. For example, if the decision-maker had ranked pair (iii) before pairs (i) and (ii) then it is easy to show that all three would have had to be explicitly ranked, as well as pair (v) (i.e. four explicitly ranked pairs in total). However, determining the optimal order is problematical as it depends on the rankings themselves, which are unknown beforehand.

Applying PAPRIKA to 'larger' value models

Of course, most real-world value models have more criteria and categories than the simple example above, which means they have many more undominated pairs. For example, the value model referred to earlier with eight criteria and four categories within each criterion (and 48 = 65,536 possible alternatives) has 2,047,516,416 undominated pairs in total (analogous to the nine identified in Figure 2), of which, excluding replicas, 402,100,560 are unique (analogous to the six in the example above). [1] (As mentioned earlier, for a model of this size the decision-maker is required to explicitly rank approximately 95 pairs defined on two criteria at-a-time, which most decision-makers are likely to be comfortable with.)

For such real-world value models, the simple pairwise-comparisons approach to identifying undominated pairs used in the previous sub-section (represented in Figure 2) is highly impractical. Likewise, identifying all pairs implicitly ranked as corollaries of the explicitly ranked pairs becomes increasingly intractable as the numbers of criteria and categories increase. The PAPRIKA method, therefore, relies on computationally efficient processes for identifying unique undominated pairs and implicitly ranked pairs respectively. The details of these processes are beyond the scope of this article, but are available elsewhere [1] and, as mentioned earlier, the PAPRIKA method is implemented by decision-making software products 1000minds and MeenyMo. [4] [5] [6]

See also

Related Research Articles

In mathematics, the symbolic method in invariant theory is an algorithm developed by Arthur Cayley, Siegfried Heinrich Aronhold, Alfred Clebsch, and Paul Gordan in the 19th century for computing invariants of algebraic forms. It is based on treating the form as if it were a power of a degree one form, which corresponds to embedding a symmetric power of a vector space into the symmetric elements of a tensor product of copies of it.

<span class="mw-page-title-main">Multiple-criteria decision analysis</span> Operations research that evaluates multiple conflicting criteria in decision making

Multiple-criteria decision-making (MCDM) or multiple-criteria decision analysis (MCDA) is a sub-discipline of operations research that explicitly evaluates multiple conflicting criteria in decision making. It is also known as multiple attribute utility theory, multiple attribute value theory, multiple attribute preference theory, and multi-objective decision analysis.

<span class="mw-page-title-main">Analytic hierarchy process</span> Structured technique for organizing and analyzing complex decisions

In the theory of decision making, the analytic hierarchy process (AHP), also analytical hierarchy process, is a structured technique for organizing and analyzing complex decisions, based on mathematics and psychology. It was developed by Thomas L. Saaty in the 1970s; Saaty partnered with Ernest Forman to develop Expert Choice software in 1983, and AHP has been extensively studied and refined since then. It represents an accurate approach to quantifying the weights of decision criteria. Individual experts’ experiences are utilized to estimate the relative magnitudes of factors through pair-wise comparisons. Each of the respondents compares the relative importance of each pair of items using a specially designed questionnaire. The relative importance of the criteria can be determined with the help of the AHP by comparing the criteria and, if applicable, the sub-criteria in pairs by experts or decision-makers. On this basis, the best alternative can be found.

Fuzzy set operations are a generalization of crisp set operations for fuzzy sets. There is in fact more than one possible generalization. The most widely used operations are called standard fuzzy set operations; they comprise: fuzzy complements, fuzzy intersections, and fuzzy unions.

Pairwise comparison generally is any process of comparing entities in pairs to judge which of each entity is preferred, or has a greater amount of some quantitative property, or whether or not the two entities are identical. The method of pairwise comparison is used in the scientific study of preferences, attitudes, voting systems, social choice, public choice, requirements engineering and multiagent AI systems. In psychology literature, it is often referred to as paired comparison.

<span class="mw-page-title-main">Independence of clones criterion</span> Property of electoral systems

In social choice theory, the independence of (irrelevant) clones criterion says that adding a clone, i.e. a new candidate very similar to an already-existing candidate, should not spoil the results. It can be considered a weak form of the independence of irrelevant alternatives (IIA) criterion that nevertheless is failed by a number of voting rules. A method that passes the criterion is said to be clone independent.

<i>Catene</i> (album) 1984 studio album by Mina

Catene is a double studio album by Italian singer Mina, released in October 1983 by PDU and distributed by EMI Italiana.

<i>Kyrie</i> (album) 1980 studio album by Mina

Kyrie is a double studio album by Italian singer Mina released on 27 November 1980 by PDU and distributed by EMI Italiana. Later the album was released in separate parts with subtitles "Vol. 1" and "Vol. 2". On this album Mina experiments with various genres, especially rock. The cover of the album features Mina's son Massimiliano Pani dressed as a hockey player.

Decision-making software is software for computer applications that help individuals and organisations make choices and take decisions, typically by ranking, prioritizing or choosing from a number of options.

In decision theory, the weighted sum model (WSM), also called weighted linear combination (WLC) or simple additive weighting (SAW), is the best known and simplest multi-criteria decision analysis (MCDA) / multi-criteria decision making method for evaluating a number of alternatives in terms of a number of decision criteria.

The weighted product model (WPM) is a popular multi-criteria decision analysis (MCDA) / multi-criteria decision making (MCDM) method. It is similar to the weighted sum model (WSM) in that it produces a simple score, but has the very important advantage of overcoming the issue of 'adding apples and pears' i.e. adding together quantities measured in different units. While there are various ways of normalizing the data beforehand, the results of the weighted sum model differ according to which normalization is chosen. The weighted product approach does not require any normalization because it uses multiplication instead of addition to aggregate the data.

The decision-making paradox is a phenomenon related to decision-making and the quest for determining reliable decision-making methods. It was first described by Triantaphyllou, and has been recognized in the related literature as a fundamental paradox in multi-criteria decision analysis (MCDA), multi-criteria decision making (MCDM) and decision analysis since then.

In decision-making, a rank reversal is a change in the rank ordering of the preferability of alternative possible decisions when, for example, the method of choosing changes or the set of other available alternatives changes. The issue of rank reversals lies at the heart of many debates in decision-making and multi-criteria decision-making, in particular.

In multiple criteria decision aiding (MCDA), multicriteria classification involves problems where a finite set of alternative actions should be assigned into a predefined set of preferentially ordered categories (classes). For example, credit analysts classify loan applications into risk categories, customers rate products and classify them into attractiveness groups, candidates for a job position are evaluated and their applications are approved or rejected, technical systems are prioritized for inspection on the basis of their failure risk, clinicians classify patients according to the extent to which they have a complex disease or not, etc.

<span class="mw-page-title-main">2012 FIBA World Olympic Qualifying Tournament for Men</span> International basketball competition

The 2012 FIBA World Olympic Qualifying Tournament was a men's basketball tournament that consisted of 12 national teams, where the top three teams earned a place in the 2012 Olympics basketball tournament. It was held on 2–8 July 2012 in Caracas, Venezuela.

<span class="mw-page-title-main">1000minds</span>

1000minds is a web application for decision-making and conjoint analysis supplied by 1000minds Ltd since 2003.

The VIKOR method is a multi-criteria decision making (MCDM) method. It was originally developed by Serafim Opricovic in 1979 to solve decision problems with conflicting and noncommensurable criteria. It assumes that compromise is acceptable for conflict resolution and that the decision maker wants a solution that is the closest to the ideal, so the alternatives are evaluated according to all established criteria. VIKOR then ranks alternatives and determines the solution named compromise that is the closest to the ideal.

<span class="mw-page-title-main">SegReg</span>

In statistics and data analysis, the application software SegReg is a free and user-friendly tool for linear segmented regression analysis to determine the breakpoint where the relation between the dependent variable and the independent variable changes abruptly.

<i>Italiana</i> (album) 1982 studio album by Mina

Italiana is a double studio album by Italian singer Mina, released in November 1982 by PDU and distributed by EMI Italiana.

References

  1. 1 2 3 4 5 6 7 Hansen, Paul; Ombler, Franz (2008). "A new method for scoring additive multi-attribute value models using pairwise rankings of alternatives". Journal of Multi-Criteria Decision Analysis. 15 (3–4): 87–107. doi:10.1002/mcda.428.
  2. 1 2 3 Smith, K. F.; Fennessy, P. F. (2011). "The use of conjoint analysis to determine the relative importance of specific traits as selection criteria for the improvement of perennial pasture species in Australia". Crop and Pasture Science. 62 (4): 355–65. doi:10.1071/CP10320.
  3. 1 2 3 Isma', Salim Al; ili, N.A.; Li, Mengxiang; Shen, Jun; He, Qiang (2016). "Cloud computing adoption decision modelling for SMEs: a conjoint analysis". International Journal of Web and Grid Services. 12 (3): 296. doi:10.1504/ijwgs.2016.079157.
  4. 1 2 3 Weistroffer, H. Roland; Li, Yan (2016). "Multiple Criteria Decision Analysis Software". Multiple Criteria Decision Analysis. International Series in Operations Research & Management Science. Vol. 233. pp. 1301–1341. doi:10.1007/978-1-4939-3094-4_29. ISBN   978-1-4939-3093-7.
  5. 1 2 3 Oleson, S. (2016), "Decision analysis software survey", OR/MS Today, vol. 43, no. 5
  6. 1 2 3 Amoyal, J. (2018), "Decision analysis software survey", OR/MS Today, vol. 45, no. 5, doi:10.1287/orms.2018.05.13, S2CID   642562
  7. ↑ Hansen, Paul; Hendry, Alison; Naden, Ray; Ombler, Franz; Stewart, Ralph (2012). "A new process for creating points systems for prioritising patients for elective health services". Clinical Governance. 17 (3): 200–209. doi:10.1108/14777271211251318.
  8. ↑ Taylor, William J.; Laking, George (2010). "Value for money – recasting the problem in terms of dynamic access prioritisation". Disability & Rehabilitation. 32 (12): 1020–1027. doi:10.3109/09638281003775535. PMID   20380596. S2CID   30265587.
  9. ↑ Gwynne-Jones, David P.; Iosua, Ella E.; Stout, Kirsten M. (1 May 2016). "Rationing for Total Hip and Knee Arthroplasty Using the New Zealand Orthopaedic Association Score: Effectiveness and Comparison With Patient-Reported Scores". The Journal of Arthroplasty. 31 (5): 957–962. doi:10.1016/j.arth.2015.11.022. ISSN   0883-5403. PMID   26944014.
  10. ↑ Fitzgerald, Avril; Spady, Barbara Conner; DeCoster, Carolyn; Naden, Ray; Hawker, Gillian A.; Noseworthy, Thomas (October 2009). "WCWL Rheumatology Priority Referral Score Reliability and Validity Testing" (PDF). Arthritis & Rheumatism. 60 (Suppl 10): 54. doi:10.1002/art.27423. PMID   21162114.
  11. ↑ Fitzgerald, Avril; de Coster, Carolyn; McMillan, Stewart; Naden, Ray; Armstrong, Fraser; Barber, Alison; Cunning, Les; Conner-Spady, Barbara; Hawker, Gillian; Lacaille, Diane; Lane, Carolyn; Mosher, Dianne; Rankin, Jim; Sholter, Dalton; Noseworthy, Tom (2011). "Relative urgency for referral from primary care to rheumatologists: The Priority Referral Score". Arthritis Care & Research. 63 (2): 231–239. doi:10.1002/acr.20366. ISSN   2151-464X. PMID   20890984. S2CID   205221379.
  12. ↑ White, Douglas; Solanki, Kamal; Quincey, Vicki; Minett, Andrew; Tam, Gordon; Doube, Alan; Naden, Ray (2015). "Development of a Multidimensional Additive Points System for Determining Access to Rheumatology Services". Journal of Clinical Rheumatology. 21 (5): 239–243. doi:10.1097/RHU.0000000000000274. ISSN   1076-1608. PMID   26203827. S2CID   32974482.
  13. ↑ Noseworthy, T; De Coster, C; Naden, R (2009). Priority-setting tools for improving access to medical specialists (PDF). 6th Health Technology Assessment International Annual Meeting. Annals, Academy of Medicine, Singapore. Vol. 38. Singapore. p. S78. Archived from the original (PDF) on 3 March 2016.
  14. ↑ Golan, Ofra; Hansen, Paul; Kaplan, Giora; Tal, Orna (2011). "Health technology prioritization: Which criteria for prioritizing new technologies and what are their relative weights?". Health Policy. 102 (2–3): 126–35. doi:10.1016/j.healthpol.2010.10.012. PMID   21071107.
  15. ↑ Golan, Ofra G; Hansen, Paul (2012). "Which health technologies should be funded? A prioritization framework based explicitly on value for money". Israel Journal of Health Policy Research. 1 (1): 44. doi: 10.1186/2045-4015-1-44 . PMC   3541977 . PMID   23181391.
  16. ↑ Shmueli, Amir (30 April 2017). "Do the equity-efficiency preferences of the Israeli Basket Committee match those of Israeli health policy makers?". Israel Journal of Health Policy Research. 6 (1): 20. doi: 10.1186/s13584-017-0145-4 . PMC   5410368 . PMID   28469840.
  17. ↑ Shmueli, Amir; Golan, Ofra; Paolucci, Francesco; Mentzakis, Emmanouil (1 April 2017). "Efficiency and equity considerations in the preferences of health policy-makers in Israel". Israel Journal of Health Policy Research. 6 (1): 18. doi: 10.1186/s13584-017-0142-7 . PMC   5376275 . PMID   28373904.
  18. ↑ Sullivan, Trudy; Hansen, Paul (1 April 2017). "Determining Criteria and Weights for Prioritizing Health Technologies Based on the Preferences of the General Population: A New Zealand Pilot Study". Value in Health. 20 (4): 679–686. doi: 10.1016/j.jval.2016.12.008 . ISSN   1098-3015. PMID   28408011.
  19. ↑ Martelli, Nicolas; Hansen, Paul; van den Brink, Hélène; Boudard, Aurélie; Cordonnier, Anne-Laure; Devaux, Capucine; Pineau, Judith; Prognon, Patrice; Borget, Isabelle (1 February 2016). "Combining multi-criteria decision analysis and mini-health technology assessment: A funding decision-support tool for medical devices in a university hospital setting". Journal of Biomedical Informatics. 59: 201–208. doi: 10.1016/j.jbi.2015.12.002 . PMID   26705065.
  20. ↑ Lasorsa, I.; Abis, G.; Podda, B.; Accardo, A. (2015). "Multi-criteria decision analysis to redesign an Italian Clinical Engineering Service under specific needs and regulation requirements". World Congress on Medical Physics and Biomedical Engineering, June 7–12, 2015, Toronto, Canada. IFMBE Proceedings. Vol. 51. pp. 1562–1565. doi:10.1007/978-3-319-19387-8_380. ISBN   978-3-319-19386-1.
  21. ↑ Aletaha, Daniel; Neogi, Tuhina; Silman, Alan J.; Funovits, Julia; Felson, David T.; Bingham, Clifton O.; Birnbaum, Neal S.; Burmester, Gerd R.; Bykerk, Vivian P.; Cohen, Marc D.; Combe, Bernard; Costenbader, Karen H.; Dougados, Maxime; Emery, Paul; Ferraccioli, Gianfranco; Hazes, Johanna M. W.; Hobbs, Kathryn; Huizinga, Tom W. J.; Kavanaugh, Arthur; Kay, Jonathan; Kvien, Tore K.; Laing, Timothy; Mease, Philip; Ménard, Henri A.; Moreland, Larry W.; Naden, Raymond L.; Pincus, Theodore; Smolen, Josef S.; Stanislawska-Biernat, Ewa; Symmons, Deborah; Tak, Paul P.; Upchurch, Katherine S.; Vencovský, Jiří; Wolfe, Frederick; Hawker, Gillian (2010). "2010 Rheumatoid arthritis classification criteria: An American College of Rheumatology/European League Against Rheumatism collaborative initiative". Arthritis & Rheumatism. 62 (9): 2569–2581. doi: 10.1002/art.27584 . hdl:2027.42/78045. ISSN   0004-3591. PMID   20872595. S2CID   23564760.
  22. ↑ Neogi, Tuhina; Aletaha, Daniel; Silman, Alan J.; Naden, Raymond L.; Felson, David T.; Aggarwal, Rohit; Bingham, Clifton O.; Birnbaum, Neal S.; Burmester, Gerd R.; Bykerk, Vivian P.; Cohen, Marc D.; Combe, Bernard; Costenbader, Karen H.; Dougados, Maxime; Emery, Paul; Ferraccioli, Gianfranco; Hazes, Johanna M. W.; Hobbs, Kathryn; Huizinga, Tom W. J.; Kavanaugh, Arthur; Kay, Jonathan; Khanna, Dinesh; Kvien, Tore K.; Laing, Timothy; Liao, Katherine; Mease, Philip; Ménard, Henri A.; Moreland, Larry W.; Nair, Raj; Pincus, Theodore (2010). "The 2010 American College of Rheumatology/European League Against Rheumatism classification criteria for rheumatoid arthritis: Phase 2 methodological report". Arthritis & Rheumatism. 62 (9): 2582–2591. doi:10.1002/art.27580. PMC   3077961 . PMID   20872596.
  23. ↑ Aletaha, Daniel (2015), "Classification of rheumatoid arthritis", in Emery, Paul (ed.), Atlas of Rheumatoid Arthritis, vol. 1, Springer Healthcare, pp. 3–21, doi:10.1007/978-1-907673-91-7_1, ISBN   978-1-907673-90-0
  24. ↑ Van Den Hoogen, F.; Khanna, D.; Fransen, J.; Johnson, S. R.; Baron, M.; Tyndall, A.; Matucci-Cerinic, M.; Naden, R. P.; Medsger, T. A.; Carreira, P. E.; Riemekasten, G.; Clements, P. J.; Denton, C. P.; Distler, O.; Allanore, Y.; Furst, D. E.; Gabrielli, A.; Mayes, M. D.; Van Laar, J. M.; Seibold, J. R.; Czirjak, L.; Steen, V. D.; Inanc, M.; Kowal-Bielecka, O.; Müller-Ladner, U.; Valentini, G.; Veale, D. J.; Vonk, M. C.; Walker, U. A.; et al. (2013). "2013 Classification Criteria for Systemic Sclerosis: An American College of Rheumatology/European League Against Rheumatism Collaborative Initiative". Arthritis & Rheumatism. 65 (11): 2737–2747. doi:10.1002/art.38098. PMC   3930146 . PMID   24122180.
  25. ↑ Van Den Hoogen, F.; Khanna, D.; Fransen, J.; Johnson, S. R.; Baron, M.; Tyndall, A.; Matucci-Cerinic, M.; Naden, R. P.; Medsger, T. A.; Carreira, P. E.; Riemekasten, G.; Clements, P. J.; Denton, C. P.; Distler, O.; Allanore, Y.; Furst, D. E.; Gabrielli, A.; Mayes, M. D.; Van Laar, J. M.; Seibold, J. R.; Czirjak, L.; Steen, V. D.; Inanc, M.; Kowal-Bielecka, O.; Muller-Ladner, U.; Valentini, G.; Veale, D. J.; Vonk, M. C.; Walker, U. A.; et al. (2013). "2013 classification criteria for systemic sclerosis: An American college of rheumatology/European league against rheumatism collaborative initiative". Annals of the Rheumatic Diseases. 72 (11): 1747–55. doi:10.1136/annrheumdis-2013-204424. hdl:2027.42/100304. PMC   3930146 . PMID   24092682. S2CID   44940902.
  26. ↑ Johnson, S. R.; Naden, R. P.; Fransen, J.; Van Den Hoogen, F.; Pope, J. E.; Baron, M.; Tyndall, A.; Matucci-Cerinic, M.; Denton, C. P.; Distler, O.; Gabrielli, A.; Van Laar, J. M.; Mayes, M.; Steen, V.; Seibold, J. R.; Clements, P.; Medsger, T. A.; Carreira, P. E.; Riemekasten, G.; Chung, L.; Fessler, B. J.; Merkel, P. A.; Silver, R.; Varga, J.; Allanore, Y.; Mueller-Ladner, U.; Vonk, M. C.; Walker, U. A.; Cappelli, S.; Khanna, D. (2014). "Multicriteria decision analysis methods with 1000Minds for developing systemic sclerosis classification criteria". Journal of Clinical Epidemiology. 67 (6): 706–14. doi:10.1016/j.jclinepi.2013.12.009. PMC   4134523 . PMID   24721558.
  27. ↑ Pope, Janet E.; Johnson, Sindhu R. (2015). "New Classification Criteria for Systemic Sclerosis (Scleroderma)". Rheumatic Disease Clinics of North America. 41 (3): 383–398. doi:10.1016/j.rdc.2015.04.003. ISSN   0889-857X. PMID   26210125.
  28. ↑ Johnson, Sindhu R. (2015). "New ACR EULAR Guidelines for Systemic Sclerosis Classification". Current Rheumatology Reports. 17 (5): 32. doi:10.1007/s11926-015-0506-3. ISSN   1523-3774. PMID   25874345. S2CID   19695361.
  29. ↑ Neogi, Tuhina; Jansen, Tim L. Th. A.; Dalbeth, Nicola; Fransen, Jaap; Schumacher, H. Ralph; Berendsen, Dianne; Brown, Melanie; Choi, Hyon; Edwards, N. Lawrence; Janssens, Hein J. E. M.; Lioté, Frédéric; Naden, Raymond P.; Nuki, George; Ogdie, Alexis; Perez-Ruiz, Fernando; Saag, Kenneth; Singh, Jasvinder A.; Sundy, John S.; Tausche, Anne-Kathrin; Vaquez-Mellado, Janitzia; Yarows, Steven A.; Taylor, William J. (2015). "2015 Gout Classification Criteria: An American College of Rheumatology/European League Against Rheumatism Collaborative Initiative". Arthritis & Rheumatology. 67 (10): 2557–2568. doi:10.1002/art.39254. ISSN   2326-5191. PMC   4566153 . PMID   26352873.
  30. ↑ Vargas-Santos, Ana Beatriz; Taylor, William J.; Neogi, Tuhina (24 June 2016). "Gout Classification Criteria: Update and Implications". Current Rheumatology Reports. 18 (7): 46. doi:10.1007/s11926-016-0594-8. PMC   4981244 . PMID   27342957.
  31. ↑ Haar, Nienke M. ter; Annink, Kim V.; Al-Mayouf, Sulaiman M.; Amaryan, Gayane; Anton, Jordi; Barron, Karyl S.; Benseler, Susanne M.; Brogan, Paul A.; Cantarini, Luca; Cattalini, Marco; Cochino, Alexis-Virgil; Benedetti, Fabrizio De; Dedeoglu, Fatma; Jesus, Adriana A. De; Alberighi, Ornella Della Casa; Demirkaya, Erkan; Dolezalova, Pavla; Durrant, Karen L.; Fabio, Giovanna; Gallizzi, Romina; Goldbach-Mansky, Raphaela; Hachulla, Eric; Hentgen, Veronique; Herlin, Troels; Hofer, Michaël; Hoffman, Hal M.; Insalaco, Antonella; Jansson, Annette F.; Kallinich, Tilmann; Koné-Paut, Isabelle; Kozlova, Anna; Kuemmerle-Deschner, Jasmin B.; Lachmann, Helen J.; Laxer, Ronald M.; Martini, Alberto; Nielsen, Susan; Nikishina, Irina; Ombrello, Amanda K.; Ozen, Seza; Papadopoulou-Alataki, Efimia; Quartier, Pierre; Rigante, Donato; Russo, Ricardo; Simon, Anna; Trachana, Maria; Uziel, Yosef; Ravelli, Angelo; Gattorno, Marco; Frenkel, Joost (1 May 2017). "Development of the autoinflammatory disease damage index (ADDI)". Annals of the Rheumatic Diseases. 76 (5): 821–830. doi:10.1136/annrheumdis-2016-210092. ISSN   0003-4967. PMC   4597180 . PMID   27811147.
  32. ↑ Kuemmerle-Deschner, Jasmin B.; Ozen, Seza; Tyrrell, Pascal N.; Kone-Paut, Isabelle; Goldbach-Mansky, Raphaela; Lachmann, Helen; Blank, Norbert; Hoffman, Hal M.; Weissbarth-Riedel, Elisabeth; Hugle, Boris; Kallinich, Tilmann; Gattorno, Marco; Gul, Ahmet; Haar, Nienke Ter; Oswald, Marlen; Dedeoglu, Fatma; Cantarini, Luca; Benseler, Susanne M. (1 June 2017). "Diagnostic criteria for cryopyrin-associated periodic syndrome (CAPS)". Annals of the Rheumatic Diseases. 76 (6): 942–947. doi:10.1136/annrheumdis-2016-209686. hdl: 11365/997711 . ISSN   0003-4967. PMID   27707729. S2CID   23147525.
  33. ↑ de Lautour, Hugh; Taylor, William J.; Adebajo, Ade; Alten, Rieke; Burgos-Vargas, Ruben; Chapman, Peter; Cimmino, Marco A.; da Rocha Castelar Pinheiro, Geraldo; Day, Ric; Harrold, Leslie R.; Helliwell, Philip; Janssen, Matthijs; Kerr, Gail; Kavanaugh, Arthur; Khanna, Dinesh; Khanna, Puja P.; Lin, Chingtsai; Louthrenoo, Worawit; McCarthy, Geraldine; Vazquez-Mellado, Janitzia; Mikuls, Ted R.; Neogi, Tuhina; Ogdie, Alexis; Perez-Ruiz, Fernando; Schlesinger, Naomi; Ralph Schumacher, H.; Scirè, Carlo A.; Singh, Jasvinder A.; Sivera, Francisca; Slot, Ole; Stamp, Lisa K.; Tausche, Anne-Kathrin; Terkeltaub, Robert; Uhlig, Till; van de Laar, Mart; White, Douglas; Yamanaka, Hisashi; Zeng, Xuejun; Dalbeth, Nicola (May 2016). "Development of Preliminary Remission Criteria for Gout Using Delphi and 1000Minds Consensus Exercises". Arthritis Care & Research. 68 (5): 667–672. doi: 10.1002/acr.22741 . hdl: 2027.42/134851 . PMID   26414176. S2CID   13512657.
  34. ↑ Rider, Lisa G.; Aggarwal, Rohit; Pistorio, Angela; Bayat, Nastaran; Erman, Brian; Feldman, Brian M.; Huber, Adam M.; Cimaz, Rolando; Cuttica, Rubén J.; Oliveira, Sheila Knupp de; Lindsley, Carol B.; Pilkington, Clarissa A.; Punaro, Marilynn; Ravelli, Angelo; Reed, Ann M.; Rouster-Stevens, Kelly; Royen-Kerkhof, Annet van; Dressler, Frank; Magalhaes, Claudia Saad; Constantin, Tamás; Davidson, Joyce E.; Magnusson, Bo; Russo, Ricardo; Villa, Luca; Rinaldi, Mariangela; Rockette, Howard; Lachenbruch, Peter A.; Miller, Frederick W.; Vencovsky, Jiri; Ruperto, Nicolino (1 May 2017). "2016 American College of Rheumatology/European League Against Rheumatism Criteria for Minimal, Moderate, and Major Clinical Response in Juvenile Dermatomyositis". Annals of the Rheumatic Diseases. 76 (5): 782–791. doi:10.1136/annrheumdis-2017-211401. ISSN   0003-4967. PMC   5517365 . PMID   28385804.
  35. ↑ Avila, M. L.; Brandão, L. R.; Williams, S.; Montoya, M. I.; Stinson, J.; Kiss, A.; Feldman, B. M. (December 2016). "Development of CAPTSure - a new index for the assessment of pediatric postthrombotic syndrome". Journal of Thrombosis and Haemostasis. 14 (12): 2376–2385. doi: 10.1111/jth.13530 . PMID   27709837. S2CID   3451087.
  36. ↑ Aringer, M; Dörner, T; Leuchten, N; Johnson, S R (31 May 2016). "Toward new criteria for systemic lupus erythematosus—a standpoint". Lupus. 25 (8): 805–811. doi: 10.1177/0961203316644338 . PMID   27252256. S2CID   2719174.
  37. ↑ Shiboski, Caroline H.; Shiboski, Stephen C.; Seror, Raphaèle; Criswell, Lindsey A.; Labetoulle, Marc; Lietman, Thomas M.; Rasmussen, Astrid; Scofield, Hal; Vitali, Claudio; Bowman, Simon J.; Mariette, Xavier; Group, the International Sjögren's Syndrome Criteria Working (1 January 2017). "2016 American College of Rheumatology/European League Against Rheumatism classification criteria for primary Sjögren9s syndrome". Annals of the Rheumatic Diseases. 76 (1): 9–16. doi:10.1136/annrheumdis-2016-210571. ISSN   0003-4967. PMID   27789466. S2CID   25716098.{{cite journal}}: |last12= has generic name (help)
  38. ↑ Shiboski, Caroline H.; Shiboski, Stephen C.; Seror, Raphaèle; Criswell, Lindsey A.; Labetoulle, Marc; Lietman, Thomas M.; Rasmussen, Astrid; Scofield, Hal; Vitali, Claudio; Bowman, Simon J.; Mariette, Xavier (January 2017). "2016 American College of Rheumatology/European League Against Rheumatism Classification Criteria for Primary Sjögren's Syndrome: A Consensus and Data-Driven Methodology Involving Three International Patient Cohorts". Arthritis & Rheumatology. 69 (1): 35–45. doi:10.1002/art.39859. PMC   5650478 . PMID   27785888.
  39. ↑ Miloslavsky, Eli M.; Naden, Ray P.; Bijlsma, Johannes W. J.; Brogan, Paul A.; Brown, E. Sherwood; Brunetta, Paul; Buttgereit, Frank; Choi, Hyon K.; DiCaire, Jean-Francois; Gelfand, Jeffrey M.; Heaney, Liam G.; Lightstone, Liz; Lu, Na; Murrell, Dedee F.; Petri, Michelle; Rosenbaum, James T.; Saag, Kenneth S.; Urowitz, Murray B.; Winthrop, Kevin L.; Stone, John H. (1 March 2017). "Development of a Glucocorticoid Toxicity Index (GTI) using multicriteria decision analysis" (PDF). Annals of the Rheumatic Diseases. 76 (3): 543–546. doi:10.1136/annrheumdis-2016-210002. hdl:10044/1/39268. ISSN   0003-4967. PMID   27474764. S2CID   206852414.
  40. ↑ Pinto, Daniel; Danilovich, Margaret K.; Hansen, Paul; Finn, Daniel J.; Chang, Rowland W.; Holl, Jane L.; Heinemann, Allen W.; Bockenholt, Ulf (1 June 2017). "Qualitative Development of a Discrete Choice Experiment for Physical Activity Interventions to Improve Knee Osteoarthritis". Archives of Physical Medicine and Rehabilitation. 98 (6): 1210–1216.e1. doi:10.1016/j.apmr.2016.11.024. ISSN   0003-9993. PMID   28034720.
  41. ↑ Griffin, D. R.; Dickenson, E. J.; O'Donnell, J.; Agricola, R.; Awan, T.; Beck, M.; Clohisy, J. C.; Dijkstra, H. P.; Falvey, E.; Gimpel, M.; Hinman, R. S.; Hölmich, P.; Kassarjian, A.; Martin, H. D.; Martin, R.; Mather, R. C.; Philippon, M. J.; Reiman, M. P.; Takla, A.; Thorborg, K.; Walker, S.; Weir, A.; Bennell, K. L. (1 October 2016). "The Warwick Agreement on femoroacetabular impingement syndrome (FAI syndrome): an international consensus statement". Br J Sports Med. 50 (19): 1169–1176. doi: 10.1136/bjsports-2016-096743 . ISSN   0306-3674. PMID   27629403. S2CID   1017216.
  42. ↑ Aggarwal, Rohit; Rider, Lisa G.; Ruperto, Nicolino; Bayat, Nastaran; Erman, Brian; Feldman, Brian M.; Oddis, Chester V.; Amato, Anthony A.; Chinoy, Hector; Cooper, Robert G.; Dastmalchi, Maryam; Fiorentino, David; Isenberg, David; Katz, James D.; Mammen, Andrew; Visser, Marianne de; Ytterberg, Steven R.; Lundberg, Ingrid E.; Chung, Lorinda; Danko, Katalin; Torre, Ignacio García-De la; Song, Yeong Wook; Villa, Luca; Rinaldi, Mariangela; Rockette, Howard; Lachenbruch, Peter A.; Miller, Frederick W.; Vencovsky, Jiri (1 May 2017). "2016 American College of Rheumatology/European League Against Rheumatism criteria for minimal, moderate, and major clinical response in adult dermatomyositis and polymyositis". Annals of the Rheumatic Diseases. 76 (5): 792–801. doi:10.1136/annrheumdis-2017-211400. ISSN   0003-4967. PMC   5496443 . PMID   28385805.
  43. ↑ Rider, Lisa G.; Aggarwal, Rohit; Pistorio, Angela; Bayat, Nastaran; Erman, Brian; Feldman, Brian M.; Huber, Adam M.; Cimaz, Rolando; Cuttica, Rubén J.; de Oliveira, Sheila Knupp; Lindsley, Carol B.; Pilkington, Clarissa A.; Punaro, Marilynn; Ravelli, Angelo; Reed, Ann M.; Rouster-Stevens, Kelly; van Royen-Kerkhof, Annet; Dressler, Frank; Magalhaes, Claudia Saad; Constantin, Tamás; Davidson, Joyce E.; Magnusson, Bo; Russo, Ricardo; Villa, Luca; Rinaldi, Mariangela; Rockette, Howard; Lachenbruch, Peter A.; Miller, Frederick W.; Vencovsky, Jiri; Ruperto, Nicolino (May 2017). "2016 American College of Rheumatology/European League Against Rheumatism Criteria for Minimal, Moderate, and Major Clinical Response in Juvenile Dermatomyositis: An International Myositis Assessment and Clinical Studies Group/Paediatric Rheumatology Inter". Arthritis & Rheumatology. 69 (5): 911–923. doi:10.1002/art.40060. PMC   5577002 . PMID   28382778.
  44. ↑ Tacconelli, Evelina; Carrara, Elena; Savoldi, Alessia; Harbarth, Stephan; Mendelson, Marc; Monnet, Dominique L; Pulcini, Céline; Kahlmeter, Gunnar; Kluytmans, Jan; Carmeli, Yehuda; Ouellette, Marc; Outterson, Kevin; Patel, Jean; Cavaleri, Marco; Cox, Edward M; Houchens, Chris R; Grayson, M Lindsay; Hansen, Paul; Singh, Nalini; Theuretzbacher, Ursula; Magrini, Nicola; Aboderin, Aaron Oladipo; Al-Abri, Seif Salem; Awang Jalil, Nordiah; Benzonana, Nur; Bhattacharya, Sanjay; Brink, Adrian John; Burkert, Francesco Robert; Cars, Otto; et al. (2017). "Discovery, research, and development of new antibiotics: The WHO priority list of antibiotic-resistant bacteria and tuberculosis". The Lancet Infectious Diseases. 18 (3): 318–327. doi:10.1016/S1473-3099(17)30753-3. PMID   29276051.
  45. ↑ Wijland, Roel; Hansen, Paul; Gardezi, Fatima (15 March 2016). "Mobile nudging: Youth engagement with banking apps". Journal of Financial Services Marketing. 21 (1): 51–63. doi:10.1057/fsm.2016.1. S2CID   167856107.
  46. ↑ Lee, Pui Yee; Lusk, Karen; Mirosa, Miranda; Oey, Indrawati (2015). "An attribute prioritization-based segmentation of the Chinese consumer market for fruit juice". Food Quality and Preference. 46: 1–8. doi:10.1016/j.foodqual.2015.06.016. hdl:10523/10826. ISSN   0950-3293.
  47. ↑ Boyd, Philip; Law, Cliff; Doney, Scott (2011). "A Climate Change Atlas for the Ocean" (PDF). Oceanography. 24 (2): 13–6. doi:10.5670/oceanog.2011.42.
  48. ↑ Chhun, Sophal; Thorsnes, Paul; Moller, Henrik (2013). "Preferences for Management of Near-Shore Marine Ecosystems: A Choice Experiment in New Zealand". Resources. 2 (3): 406–438. doi: 10.3390/resources2030406 .
  49. ↑ Chhun, Sophal; Kahui, Viktoria; Moller, Henrik; Thorsnes, Paul (2015). "Advancing Marine Policy Toward Ecosystem-Based Management by Eliciting Public Preferences". Marine Resource Economics. 30 (3): 261–275. doi:10.1086/681052. hdl:10523/7118. ISSN   0738-1360. S2CID   154046715.
  50. ↑ Graff, P.; McIntyre, S. (2014). "Using ecological attributes as criteria for the selection of plant species under three restoration scenarios". Austral Ecology. 39 (8): 907–917. Bibcode:2014AusEc..39..907G. doi:10.1111/aec.12156. hdl: 11336/4243 .
  51. ↑ Crozier, G. K. D.; Schulte-Hostedde, A. I. (2014). "Towards Improving the Ethics of Ecological Research". Science and Engineering Ethics. 21 (3): 577–94. doi:10.1007/s11948-014-9558-4. PMC   4430594 . PMID   24903671.
  52. ↑ de Olde, Evelien M.; Moller, Henrik; Marchand, Fleur; McDowell, Richard W.; MacLeod, Catriona J.; Sautier, Marion; Halloy, Stephan; Barber, Andrew; Benge, Jayson; Bockstaller, Christian; Bokkers, Eddie A. M.; de Boer, Imke J. M.; Legun, Katharine A.; Le Quellec, Isabelle; Merfield, Charles; Oudshoorn, Frank W.; Reid, John; Schader, Christian; Szymanski, Erika; Sørensen, Claus A. G.; Whitehead, Jay; Manhire, Jon (11 May 2016). "When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture". Environment, Development and Sustainability. 19 (4): 1327–1342. doi:10.1007/s10668-016-9803-x. hdl:10523/7113. S2CID   62803197.
  53. ↑ Byrne, T. J.; Amer, P. R.; Fennessy, P. F.; Hansen, P.; Wickham, B. W. (2011). "A preference-based approach to deriving breeding objectives: Applied to sheep breeding". Animal. 6 (5): 778–88. doi: 10.1017/S1751731111002060 . PMID   22558925. S2CID   206336898.
  54. ↑ Nielsen, H. M.; Amer, P. R.; Byrne, T. J. (2013). "Approaches to formulating practical breeding objectives for animal production systems". Acta Agriculturae Scandinavica, Section A. 64 (1): 2–12. doi:10.1080/09064702.2013.827237. ISSN   0906-4702. S2CID   84895619.
  55. ↑ Martin-Collado, D.; Byrne, T.J.; Amer, P.R.; Santos, B.F.S.; Axford, M.; Pryce, J.E. (2015). "Analyzing the heterogeneity of farmers' preferences for improvements in dairy cow traits using farmer typologies". Journal of Dairy Science. 98 (6): 4148–61. doi: 10.3168/jds.2014-9194 . ISSN   0022-0302. PMID   25864048.
  56. ↑ Slagboom, M.; Kargo, M.; Edwards, D.; Sørensen, A. C.; Thomasen, J. R.; Hjortø, L. (2 July 2016). "Herd characteristics influence farmers' preferences for trait improvements in Danish Red and Danish Jersey cows" (PDF). Acta Agriculturae Scandinavica, Section A. 66 (3): 177–182. doi:10.1080/09064702.2016.1277550. ISSN   0906-4702. S2CID   53575561.
  57. ↑ Slagboom, M.; Kargo, M.; Edwards, D.; Sørensen, A.C.; Thomasen, J.R.; Hjortø, L. (1 December 2016). "Organic dairy farmers put more emphasis on production traits than conventional farmers". Journal of Dairy Science. 99 (12): 9845–9856. doi: 10.3168/jds.2016-11346 . ISSN   0022-0302. PMID   27692711.
  58. ↑ Byrne, T.J.; Santos, B.F.S.; Amer, P.R.; Martin-Collado, D.; Pryce, J.E.; Axford, M. (1 October 2016). "New breeding objectives and selection indices for the Australian dairy industry". Journal of Dairy Science. 99 (10): 8146–8167. doi: 10.3168/jds.2015-10747 . ISSN   0022-0302. PMID   27522425.
  59. ↑ Smith, K. F.; Fennessy, P. F. (2014). "Utilizing Conjoint Analysis to Develop Breeding Objectives for the Improvement of Pasture Species for Contrasting Environments when the Relative Values of Individual Traits Are Difficult to Assess". Sustainable Agriculture Research. 3 (2): 44. doi: 10.5539/sar.v3n2p44 .
  60. ↑ Smith, K. F.; Ludemann, C.; Lewis, C. D.; Malcolm, B.; Banks, R. G.; Jacobs, J. L.; Fennessy, P. F.; Spangenberg, G. C. (2014). "Estimating the value of genetic gain in perennial pastures with emphasis on temperate species". Crop and Pasture Science. 65 (11): 1230. doi:10.1071/CP13384. ISSN   1836-0947. S2CID   84123507.
  61. ↑ Christofferson, Andrew (2007), "Housing choice in Dunedin" (PDF), City Planning, District Plan Monitoring Series, Research Report, vol. 2007/1
  62. ↑ Moura, Filipe; Cambra, Paulo; Gonçalves, Alexandre B. (2016). "Measuring walkability for distinct pedestrian groups with a participatory assessment method: A case study in Lisbon". Landscape and Urban Planning. 157: 282–296. doi:10.1016/j.landurbplan.2016.07.002.
  63. ↑ Chang, Shoou-Yuh; Gronwald, Frank (1 May 2016). "A Multi-criteria Evaluation of the Methods for Recycling Scrap Tires". The Journal of Solid Waste Technology and Management. 42 (2): 145–156. doi:10.5276/JSWTM.2016.145.
  64. ↑ Mancini, Adriano; Frontoni, Emanuele; Zingaretti, Primo (December 2015). "Embedded Multisensor System for Safe Point-to-Point Navigation of Impaired Users". IEEE Transactions on Intelligent Transportation Systems. 16 (6): 3543–3555. doi:10.1109/TITS.2015.2489261. S2CID   8231525.
  65. ↑ Smith, Christie (2009), "Revealing monetary policy preferences" Archived 8 May 2014 at the Wayback Machine , Reserve Bank of New Zealand Discussion Paper Series, DP2009/01;
  66. ↑ Au, Joey; Coleman, Andrew; Sullivan, Trudy (2015). A Practical Approach to Well-being Based Policy Development: What Do New Zealanders Want from Their Retirement Income Policies?. Working Papers. Vol. WP 15/14. New Zealand Treasury. ISBN   978-0-478-43678-5. Archived from the original on 5 March 2018.
  67. ↑ Hansen, P.; Kergozou, N.; Knowles, S.; Thorsnes, P. (2014). "Developing Countries in Need: Which Characteristics Appeal most to People when Donating Money?" (PDF). The Journal of Development Studies. 50 (11): 1494–1509. doi:10.1080/00220388.2014.925542. hdl:10523/4276. S2CID   154738603.
  68. ↑ Cunningham, Harry; Knowles, Stephen; Hansen, Paul (12 March 2017). "Bilateral foreign aid: how important is aid effectiveness to people for choosing countries to support?". Applied Economics Letters. 24 (5): 306–310. doi:10.1080/13504851.2016.1184372. hdl:10523/6393. ISSN   1350-4851. S2CID   29778171.
  69. ↑ Belton, V and Stewart, TJ, Multiple Criteria Decision Analysis: An Integrated Approach, Kluwer: Boston, 2002, pp. 166–168.
  70. ↑ Johnson, Richard M. (1976). "Beyond conjoint measurement: A method of pairwise trade-off analysis". Advances in Consumer Research. 3: 353–8.
  71. ↑ Green, P. E.; Krieger, A. M.; Wind, Y. (2001). "Thirty Years of Conjoint Analysis: Reflections and Prospects". Interfaces. 31 (3_supplement): S56. CiteSeerX   10.1.1.130.2548 . doi:10.1287/inte.31.3s.56.9676.
  72. ↑ Larichev, O.I.; Moshkovich, H.M. (1995). "ZAPROS-LM – A method and system for ordering multiattribute alternatives". European Journal of Operational Research. 82 (3): 503–521. doi:10.1016/0377-2217(93)E0143-L.
  73. ↑ Moshkovich, Helen M; Mechitov, Alexander I; Olson, David L (2002). "Ordinal judgments in multiattribute decision analysis". European Journal of Operational Research. 137 (3): 625–641. doi:10.1016/S0377-2217(01)00106-0.
  74. ↑ Krantz, D. H. (1972). "Measurement Structures and Psychological Laws". Science. 175 (4029): 1427–35. Bibcode:1972Sci...175.1427K. doi:10.1126/science.175.4029.1427. PMID   17842276. S2CID   29113793.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.