Query performance prediction and effectiveness evaluation without relevance judgments: Two sides of the same coin

Mizzaro, Stefano; Mothe, Josiane; Roitero, Kevin; Ullah, Md Zia

Abstract

Some methods have been developed for automatic effectiveness evaluation without relevance judgments. We propose to use those methods, and their combination based on a machine learning approach, for query performance prediction. Moreover, since predicting average precision as it is usually done in query performance prediction literature is sensitive to the reference system that is chosen, we focus on predicting the average of average precision values over several systems. Results of an extensive experimental evaluation on ten TREC collections show that our proposed methods outperform state-of-the-art query performance predictors.

Citation

Mizzaro, S., Mothe, J., Roitero, K., & Ullah, M. Z. (2018, July). Query performance prediction and effectiveness evaluation without relevance judgments: Two sides of the same coin. Presented at SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA

• Presentation Conference Type: Conference Paper (published)
• Conference Name: SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval
• Start Date: Jul 8, 2018
• End Date: Jul 12, 2018
• Online Publication Date: Jun 27, 2018
• Publication Date: 2018-06
• Deposit Date: Mar 13, 2023
• Publisher: Association for Computing Machinery (ACM)
• Pages: 1233-1236
• Book Title: SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval
• ISBN: 9781450356572
• DOI: https://doi.org/10.1145/3209978.3210146