EMPLOYING SERENDIPITY IN RECOMMENDER SYSTEMS TO OVERCOME THE POPULARITY BIAS AND LONG TAIL PROBLEM

Abstract

Assisting users to aid in decision making while doing e-commerce purchases is a primary task of a traditional Recommender System. But often there is a necessity to give equitable importance to the products somehow ignored by the traditional Recommender Systems. Drawing less number of ratings from the users of the system should not be the reason to make the item fall into the long tail list of non-popular Items. So, to overcome such Recommender System issues of ‘Long Tail’ and ‘Popularity Bias’, we have proposed a new Serendipitous Recommender System using k nearest neighbor approach coupled with the Bhattacharyya coefficient to recommend not only novel but also at the same time, the relevant set of Items out of the long tail list of non-popular Items. The thrust on serendipity is assisting the traditional recommender systems to narrow down on the abundance of recommendations with special weightage and emphasis on waiting-to-be-recommended ‘long tail’ items. Further, it also paves the way for moving from the overlooked ‘accuracy’ aspect of recommender systems to the highly fruitful and rightful aspect of ‘user satisfaction’. As the serendipitous recommender systems inculcate the refreshing ‘novelty’ component, the inherent traditional recommender systems’ issues of ‘long tail problem’, ‘popularity bias’, ‘cold start problem’, ‘over specialization issue’, ‘matthew effect’, etc. are overcome. Hence, we investigate and analyze the effectiveness of three different serendipitous recommender system algorithms, TANGENT, KFN and an already published NOVEL SERENDIPITOUS ALGORITHM on a prominent ‘novelty score’ metric. The detailed and rigorous analysis suggest that all the three algorithms are able to surpass the 50 % novelty score benchmark, with the overall novelty scores of 55.57 % for the TANGENT algorithm, 79.39 % for the KFN algorithm and 83.03 % for the NOVEL SERENDIPITOUS ALGORITHM. The results vindicate the overall supremacy and efficacy of NOVEL SERENDIPITOUS ALGORITHM over the other two serendipitous algorithms.