A framework for extracting relevant web pages from www using web mining

Abstract

The key characteristics of the World Wide Web are its large size and the lack of a centralized control over its contents. These are also the most important source of problems for locating information. In fact, the traditional Information Retrieval methods become hopelessly slow especially when confronted with the extremely large volume of the Web and its speed of change. Therefore, the coverage of the web by the modern search engines is relatively small compared to the size of web. Moreover, the quality of the web pages downloaded by the search engines is very much skewed in the sense that the interesting pages are scarce in comparison with the rest of the content. Current search engines employ crawlers to collect pages from the WWW which are then stored in the index. This index is then matched with the user submitted query to present the list of documents to the user. However, the list of documents presented to the user is so huge and sometimes irrelevant that it is actually posing the problem of information overkill. Relevancy is a very subjective term in a sense that a document which may be relevant for one user for the same query may be irrelevant for some other user situated at the geographically different location. The other major issue is user perceived latency occurring because of the complex nature of the web coupled with high network traffic prevalent over the net. Therefore, response time that the user experiences in getting the results for his query from the search engine is very large. According to the famous 8 sec rule , users tend to divert themselves from the website if they do not get what they are looking for within this time frame. This work focuses on the above mentioned issues and provides the solution for them by applying Web Prefetching over World Wide Web. Web Prefetching is the technique where users web request is fulfilled even before the user actually makes it.