Universiti Teknologi Malaysia Institutional Repository

Big data tasks execution time analysis using machine learning techniques

Shabbir, A. and Abu Bakar, K. and Radzi, R. Z. R. M. and Siraj, M. (2019) Big data tasks execution time analysis using machine learning techniques. In: 9th International Conference on Industrial Engineering and Operations Management, IEOM 2019, 5-7 March 2019, JW Marriott Hotel Bangkok, Thailand.


Official URL: http://www.ieomsociety.org/ieom2019/papers/665.pdf...


Big data and its analysis are in the focus of current era. The volume of data production is tremendous and a significant part of delivered data is not utilized because of the limited assets to store and process them efficiently. The world acclaimed platform that can efficiently deal with the gigantic amount of data in a cost effective manner is Hadoop MapReduce. In order to effectively utilize any computational platform, information about the components affecting its performance is necessary. Similarly, Hadoop MapReduce's performance can be enhanced by identifying those factors that can affect its performance. Some researchers provided some schemes for improving total task completion time of big data tasks on Hadoop MapReduce by suitable selection and scheduling of processing units i.e. mappers. However, reducers are still underexplored for its effect on the total execution time. This paper aimed at evaluation of reducer's impact on total execution time of big data tasks on Hadoop MapReduce by employing machine learning techniques. The evaluation has been carried out both analytically and experimentally by changing different number of reducers across various types and length of tasks. The results clearly depicts the dependence of total MapReduce task execution time on the number of reducers.

Item Type:Conference or Workshop Item (Paper)
Uncontrolled Keywords:big data, hadoop mapreduce, machine learning
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
ID Code:91088
Deposited By: Narimah Nawil
Deposited On:31 May 2021 21:21
Last Modified:31 May 2021 21:21

Repository Staff Only: item control page