SMK-means: An Improved Mini Batch K-means Algorithm Based on Mapreduce with Big Data

Xiao, Bo; Wang, Zhen; Liu, Qi; Liu, Xiaodong

Abstract

In recent years, the rapid development of big data technology has also been favored by more and more scholars. Massive data storage and calculation problems have also been solved. At the same time, outlier detection problems in mass data have also come along with it. Therefore, more research work has been devoted to the problem of outlier detection in big data. However, the existing available methods have high computation time, the improved algorithm of outlier detection is presented, which has higher performance to detect outlier. In this paper, an improved algorithm is proposed. The SMK-means is a fusion algorithm which is achieved by Mini Batch K-means based on simulated annealing algorithm for anomalous detection of massive household electricity data, which can give the number of clusters and reduce the number of iterations and improve the accuracy of clustering. In this paper, several experiments are performed to compare and analyze multiple performances of the algorithm. Through analysis, we know that the proposed algorithm is superior to the existing algorithms.

Citation

Xiao, B., Wang, Z., Liu, Q., & Liu, X. (2018). SMK-means: An Improved Mini Batch K-means Algorithm Based on Mapreduce with Big Data. Computers, Materials & Continua, 56(3), 365-379. https://doi.org/10.3970/cmc.2018.01830

• Journal Article Type: Article
• Acceptance Date: Jun 6, 2018
• Publication Date: Dec 1, 2018
• Deposit Date: Jun 22, 2018
• Publicly Available Date: Dec 1, 2018
• Journal: Computers, Materials & Continua
• Print ISSN: 1546-2218
• Publisher: Tech Science Press
• Peer Reviewed: Peer Reviewed
• Volume: 56
• Issue: 3
• Pages: 365-379
• DOI: https://doi.org/10.3970/cmc.2018.01830
• Keywords: Big data, outlier detection, SMK-means, Mini Batch K-means, simulated annealing.
• Public URL: http://researchrepository.napier.ac.uk/Output/1233588
• Contract Date: Jun 21, 2018