Improving the mapreduce performance by optimizing the structure of mapreduce computation in hadoop

Abstract

MapReduce is a framework used for processing huge datasets with newlineparallel and distributed computing environments. Hadoop, an open-source newlinewidely used implementation of MapReduce can be set up on massive newlinecomputing clusters. Hadoop automatically parallelizes a job into multiple newlinetasks and handles the execution of tasks on large commodity clusters. Each newlineMapReduce job in Hadoop has two major dependent phases, namely, Map newlineand Reduce. A single task failure can cause the failure of the complete job newlineand unpredictable job running times. Task and node failures cause a newlinesubstantial impact on Hadoop s performance. MapReduce handles the task newlinefailures by restarting the failed task and recomputing all the input data from newlinescratch, regardless of how much data was processed earlier. This increases the newlineturn-around time of MapReduce jobs newline