Estimation Accuracy on Execution Time of Run-Time Tasks in a Heterogeneous Distributed Environment

Liu, Qi; Cai, Weidong; Jin, Dandan; Shen, Jian; Fu, Zhangjie; Liu, Xiaodong; Linge, Nigel

doi:10.3390/s16091386

Estimation Accuracy on Execution Time of Run-Time Tasks in a Heterogeneous Distributed Environment

Liu, Qi; Cai, Weidong; Jin, Dandan; Shen, Jian; Fu, Zhangjie; Liu, Xiaodong; Linge, Nigel

Authors

Qi Liu

Weidong Cai

Dandan Jin

Jian Shen

Zhangjie Fu

Prof Xiaodong Liu X.Liu@napier.ac.uk
Professor

Nigel Linge

Abstract

Distributed Computing has achieved tremendous development since cloud computing was proposed in 2006, and played a vital role promoting rapid growth of data collecting and analysis models, e.g., Internet of things, Cyber-Physical Systems, Big Data Analytics, etc. Hadoop has become a data convergence platform for sensor networks. As one of the core components, MapReduce facilitates allocating, processing and mining of collected large-scale data, where speculative execution strategies help solve straggler problems. However, there is still no efficient solution for accurate estimation on execution time of run-time tasks, which can affect task allocation and distribution in MapReduce. In this paper, task execution data have been collected and employed for the estimation. A two-phase regression (TPR) method is proposed to predict the finishing time of each task accurately. Detailed data of each task have drawn interests with detailed analysis report being made. According to the results, the prediction accuracy of concurrent tasks’ execution time can be improved, in particular for some regular jobs.

Citation

Liu, Q., Cai, W., Jin, D., Shen, J., Fu, Z., Liu, X., & Linge, N. (2016). Estimation Accuracy on Execution Time of Run-Time Tasks in a Heterogeneous Distributed Environment. Sensors, 16(9), 1386. https://doi.org/10.3390/s16091386

Journal Article Type	Article
Acceptance Date	Aug 25, 2016
Online Publication Date	Aug 30, 2016
Publication Date	Oct 31, 2016
Deposit Date	Jan 30, 2017
Publicly Available Date	Feb 1, 2017
Journal	Sensors
Publisher	MDPI
Peer Reviewed	Peer Reviewed
Volume	16
Issue	9
Pages	1386
DOI	https://doi.org/10.3390/s16091386
Keywords	cloud computing; data convergence; MapReduce; data analysis; speculative execution
Public URL	http://researchrepository.napier.ac.uk/Output/451221

Files

Estimation Accuracy on Execution Time of Run-Time Tasks in a Heterogeneous Distributed Environment (2.9 Mb)
PDF

Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/

Copyright Statement
© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license http://creativecommons.org/licenses/by/4.0/)