On the Metric Properties of IR Evaluation Measures Based on Ranking Axioms

Abstract: The axiomatic analysis of IR evaluation metrics has contributed to a better understanding of their properties. Some works have modelled the effectiveness of retrieval measures with axioms that capture desirable properties on the set of ranked lists of documents. Recently, it has been shown that three of these axioms lead to some orderings. This work formally explores the metric properties of the set of rankings, endowed with these orderings. Based on lattice theory, the possible metrics and pseudo-metrics, defined on these structures, are determined. It is found that, when the relevant documents are prioritized, precision, recall, RBP and DCG are metrics on the set of rankings, however they are pseudo-metrics when the swapping of documents is considered.