Query Subtopic Mining Exploiting Word Embedding for Search Result Diversification

Ullah, Md Zia; Chy, Abu Nowshed; Aono, Masaki

Abstract

Understanding the users’ search intents through mining query subtopic is a challenging task and a prerequisite step for search diversification. This paper proposes mining query subtopic by exploiting the word embedding and short-text similarity measure. We extract candidate subtopic from multiple sources and introduce a new way of ranking based on a new novelty estimation that faithfully represents the possible search intents of the query. To estimate the subtopic relevance, we introduce new semantic features based on word embedding and bipartite graph based ranking. To estimate the novelty of a subtopic, we propose a method by combining the contextual and categorical similarities. Experimental results on NTCIR subtopic mining datasets turn out that our proposed approach outperforms the baselines, known previous methods, and the official participants of the subtopic mining tasks.

Citation

Ullah, M. Z., Chy, A. N., & Aono, M. (2016). Query Subtopic Mining Exploiting Word Embedding for Search Result Diversification. In Information Retrieval Technology: 12th Asia Information Retrieval Societies Conference, AIRS 2016, Beijing, China, November 30 – December 2, 2016, Proceedings (308-314). https://doi.org/10.1007/978-3-319-48051-0_24

• Conference Name: 12th Asia Information Retrieval Societies Conference
• Conference Location: Beijing, China
• Start Date: Nov 30, 2016
• End Date: Dec 2, 2016
• Online Publication Date: Oct 15, 2016
• Publication Date: 2016
• Deposit Date: Mar 13, 2023
• Publisher: Springer
• Pages: 308-314
• Series Title: Lecture Notes in Computer Science
• Series ISSN: 1611-3349
• Book Title: Information Retrieval Technology: 12th Asia Information Retrieval Societies Conference, AIRS 2016, Beijing, China, November 30 – December 2, 2016, Proceedings
• DOI: https://doi.org/10.1007/978-3-319-48051-0_24
• Keywords: Subtopic mining, Word embedding, Diversification, Novelty