Content Extraction from Web Pages Using Data Mining
Loading...
Date
item.page.authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Data mining is an interdisciplinary sub field of computer science and statistics with an overall goal to extract information from a large data set and transform the information into a comprehensible structure for further use. It plays an important role in today s world, because volume of data is increasing day by day. Data mining is the process of extracting useful information from this large amount of data.
newlineThis research work is mainly related to web mining. It is an application of data mining and is mainly focused on data present in World Wide Web. The size of web is large and is growing rapidly. The contents of web pages are more complex than any other documents. It may contain audio, video, hyper links, advertisements, copyright and privacy notices, etc and is called noises. These noises are unimportant in the information extraction process. Web pages may also contain duplicate contents. Noises as well as duplicate contents are removed during the information extraction process.
newlineData mining and web mining are interrelated. Web mining is an application of data mining to extract information from web documents. Two different approaches have been proposed in web mining. One is process centric view and the other is data centric view. Process centric view is a sequence of tasks ordered and the second approach is defined in terms of web data used. Three important dimensions of web mining are web content mining, web structure mining and web usage mining.
newlineIn this research work, text summarization is done by various approaches. A web page may contain noises like header, footer, copyright information, advertisements, audio, video, redundant sentences, etc. Identifying and removal of these noises is a complex task. Some existing methods are there for web content extraction, but they are not up to the mark. In the proposed method, feature extraction, unsupervised clustering and paraphrase identifications are used for getting summarized document. Paraphrase identification uses both syntax and semantic analysis