Novel Techniques for Concept Drift Detection and Handling

Abstract

In today s world, learning in the presence of dynamic environments, where continous change and development are evident, is a challenging task. Recent advances in technology have witnessed an increase in the number of real world applications which include spam filtering, fraud detection, weather forecasting, sensors, smart cities, health monitoring etc. Data generated from such sources in form of streams tends to evolve with the course of time. The predictive models which are trained using such data tend to become obselete with time, resulting into poor adaptabilty to the underlying drifting distributions. In terms of machine learning and data mining, the change in the statistical properties of data is known as concept drift. Such changes causes degradation in the performance of the learning systems since the models that were built on the old data are no longer consistent with the new data. To address the problem of concept drift, efficient learning models which can monitor the evolving distributions and update themselves regularly are required. These models should detect the drifts and handle them timely by using adaptive learning techniques, to overcome the deteriorating performance. Various learning methods which include single learners as well as ensemble based modelling which utilize drift detectors, are used in literature to handle evolving data streams. This thesis proposes three techniques for concept drift detection and handling. First one, a hybrid diversity based ensemble approach, called Ensemble Based Online Diversified Drift Detection (En-ODDD), combines explicit drift detection and adaptive techniques deal with drifting distributions. In second approach,