Effective outlier detection in data mining using big data analysis

Abstract

newline Information technologies are growing rapidly in recent days such that the number of databases dimensions and complexity are also increasing Hence it is essential to examine the large amount of information in the data mining Data mining deals with finding the hidden patterns on trivial and interesting knowledge from different types of data An outlier is an anomaly that differs from other types of data points in the dataset The outlier detection and future prediction are the significant issues in data mining It is the process of determining outliers depending up on the behavior and transmission of data It is also used to determine the abnormal data with unpredictable features Commonly the outlier is the opinion of data that can be removed or considered separately in regression model to improve the accuracy The existing techniques have some of the drawbacks such as increased complexity size and unable to handle variety of datasets In order to overcome these issues two novel techniques namely Label Based Outlier Detection using Support Vector Data Description LBOD SVDD and Optimized Moving Average OMA are proposed with the use of SPARK Hadoop architecture in this work In this first phase of this research the new approach is developed and it is termed as Label Based Outlier Detection using Support Vector Data Description LBOD SVDD and Optimized Moving Average OMA This approach is mainly designed to overcome these issues with the help of Spark Hadoop find architecture The purpose of this research area is to determine the outliers and to find the demand of power in future The power demand dataset is used for this purpose This approach is used to classify LBOD SVDD for outlier detection and OMA for future prediction process newline newline