The use of bibliography enriched features for automatic citation screening

Olorisade, Babatunde Kazeem; Brereton, Pearl; Andras, Peter

Abstract

Context
Citation screening (also called study selection) is a phase of systematic review process that has attracted a growing interest on the use of text mining (TM) methods to support it to reduce time and effort. Search results are usually imbalanced between the relevant and the irrelevant classes of returned citations. Class imbalance among other factors has been a persistent problem that impairs the performance of TM models, particularly in the context of automatic citation screening for systematic reviews. This has often caused the performance of classification models using the basic title and abstract data to ordinarily fall short of expectations.

Objective
In this study, we explore the effects of using full bibliography data in addition to title and abstract on text classification performance for automatic citation screening.

Methods
We experiment with binary and Word2vec feature representations and SVM models using 4 software engineering (SE) and 15 medical review datasets. We build and compare 3 types of models (binary-non-linear, Word2vec-linear and Word2vec-non-linear kernels) with each dataset using the two feature sets.

Results
The bibliography enriched data exhibited consistent improved performance in terms of recall, work saved over sampling (WSS) and Matthews correlation coefficient (MCC) in 3 of the 4 SE datasets that are fairly large in size. For the medical datasets, the results vary, however in the majority of cases the performance is the same or better.

Conclusion
Inclusion of the bibliography data provides the potential of improving the performance of the models but to date results are inconclusive.

Citation

Olorisade, B. K., Brereton, P., & Andras, P. (2019). The use of bibliography enriched features for automatic citation screening. Journal of Biomedical Informatics, 94, Article 103202. https://doi.org/10.1016/j.jbi.2019.103202

• Journal Article Type: Article
• Acceptance Date: May 3, 2019
• Online Publication Date: May 7, 2019
• Publication Date: 2019-06
• Deposit Date: Nov 8, 2021
• Journal: Journal of biomedical informatics
• Print ISSN: 1532-0464
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 94
• Article Number: 103202
• DOI: https://doi.org/10.1016/j.jbi.2019.103202
• Keywords: Computing methodologies, Citation screening automation, Systematic reviews, Text mining, Feature enrichment
• Public URL: http://researchrepository.napier.ac.uk/Output/2808924