Distributed Association Rule Mining

Abstract

Association rules are used to discover interesting patterns of discrete data located in transactional database or another information repository. In many organizations, data is not stored in centralized location but dispersed at various geographical locations. The aim of newlineDistributed Association Rule Mining (DARM) is to determine interesting patterns from the newlinedatasets that are spread over various geographical sites. While determining interesting patterns,DARM focuses on minimizing the execution time and communication costs between datasets newlineat different locations. The existing work in DARM includes algorithms such as Count newlineDistribution Algorithm (CDA), Fast Distributed Mining Algorithms (FDM) and Optimized Distributed Association Mining (ODAM). The experiments conducted showed that the ODAM algorithm performed better than CDA and FDM algorithms and was considered as a benchmark for the proposed algorithms. newlineThe first contribution is a new proposed architecture to the existing DARM framework. The existing architecture handles the local and global data using local and global modules, whereas the proposed architecture handles incremental data along with local and global data. The proposed architecture added modules namely the Database (DB) processing module and the incremental module along with local ARM and global module. The second contribution is an enhanced DARM algorithm called Transaction Reduction Enhanced Distributed Association Rule Mining (TR-EDARM) for the ARM module of proposed architecture. The algorithm reduced the number of transactions at each pass which causes a considerable reduction in the execution time. TR-EDRAM also improved the communication cost by exchanging the locally newlinelarge frequent itemsets in the distributed environment. The third contribution is a new algorithm called Incremental ODAM (IODAM) for incremental module of the proposed architecture.