Sébastien Déjean
Studying the variability of system setting effectiveness by data analytics and visualization
Déjean, Sébastien; Mothe, Josiane; Ullah, Md Zia
Abstract
Search engines differ from their modules and parameters; defining the optimal system setting is challenging the more because of the complexity of a retrieval stream. The main goal of this study is to determine which are the most important system components and parameters in system setting, thus which ones should be tuned as the first priority. We carry out an extensive analysis of 20, 000 different system settings applied to three TREC ad-hoc collections. Our analysis includes zooming in and out the data using various data analysis methods such as ANOVA, CART, and data visualization. We found that the query expansion model is the most significant component that changes the system effectiveness, consistently across collections. Zooming in the queries, we show that the most significant component changes to the retrieval model when considering easy queries only. The results of our study are directly re-usable for the system designers and for system tuning.
Citation
Déjean, S., Mothe, J., & Ullah, M. Z. (2019). Studying the variability of system setting effectiveness by data analytics and visualization. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9--12, 2019, Proceedings (62-74). https://doi.org/10.1007/978-3-030-28577-7_3
Conference Name | Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association (CLEF 2019) |
---|---|
Conference Location | Lugano, Switzerland |
Start Date | Sep 9, 2019 |
End Date | Sep 12, 2019 |
Online Publication Date | Aug 3, 2019 |
Publication Date | 2019 |
Deposit Date | Mar 13, 2023 |
Publisher | Springer |
Pages | 62-74 |
Series Title | Lecture Notes in Computer Science |
Series Number | 11696 |
Series ISSN | 1611-3349 |
Book Title | Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9--12, 2019, Proceedings |
ISBN | 978-3-030-28576-0 |
DOI | https://doi.org/10.1007/978-3-030-28577-7_3 |
Keywords | Information retrieval, Data analytics, Retrieval system modules, Effectiveness |
You might also like
Defining an Optimal Configuration Set for Selective Search Strategy - A Risk-Sensitive Approach
(2021)
Conference Proceeding
Forward and backward feature selection for query performance prediction
(2020)
Conference Proceeding
Information nutritional label and word embedding to estimate information check-worthiness
(2019)
Conference Proceeding
Query performance prediction focused on summarized letor features
(2018)
Conference Proceeding