STABILITY OF RANKING-DEPENDENT PAIR-WISE COMPARISON PATTERNS IN THE ANALYTIC HIERARCHY PROCESS

In the Analytic Hierarchy Process (AHP), obtaining pair-wise comparisons (PCs) for a large number of objects is labor-intensive. To simplify this, researchers use incomplete PC methods that leverage rough ordinal rankings of the objects. This paper investigates the stability of three such methods—Best-worst, Top 2 (Best-Second Best), and the max difference method—to determine which is most robust against expert errors.

The authors conducted a simulation-based experiment to compare the stability of these three methods. They defined stability as the maximum deviation of the calculated priority vector from the true priority vector when judgments are perturbed by simulated errors. To ensure a fair comparison, they standardized the number of comparisons across all three methods. Crucially, they introduced a generalized fluctuation formula that models expert error as a function of the ordinal distance between compared objects, allowing them to test scenarios ranging from error being independent of rank distance to error being strongly dependent on it.

The study reveals that there is no single universally superior method; rather, the optimal choice depends on the nature of the expert's error. When expert errors are assumed to be independent of the rank distance between objects, the Top 2 method demonstrates the highest stability. However, as the model shifts to assume that errors are strongly influenced by the ordinal distance between objects—where comparing more distant objects is more reliable—the max difference method becomes the most stable. The authors suggest that the max difference method is particularly effective for mitigating errors in complex, uncertain environments.

By identifying the conditions under which specific incomplete PC patterns are most stable, this research provides a framework for decision-makers to reduce the cognitive burden on experts without sacrificing the credibility of the final results. It highlights the importance of considering the cognitive aspects of the comparison process, such as the anchoring bias, when designing decision support systems.