Lexicon Based Multi Category Classification Using Text Mining For NLP of Punjabi Poetry

Abstract

Analysis of poetic text is very challenging from computational linguistic perspective. Computational analysis of literary arts, especially poetry, is a very difficult task for classification. For library recommendation system, poetries can be classified on various metrics such as poet, time period, sentiments and subject matter. In this work, content-based Punjabi poetry classifier was developed using Weka toolset. Four different categories were manually populated with 2034 poems (NAFE, LIPA, RORE, PHSP categories consists of 505, 399, 529 and 601 numbers of poetries, respectively. These poetries were passed to various pre-processing sub-phases such as tokenization, noise removal, stop word removal, special symbol removal. A total of 31398 tokens were extracted, after removal of noise, and weighed using term frequency (TF) and term frequency-inverse document frequency (TF-IDF) weighting scheme. Extracted tokens were further reduced by ranking using gain ratio. These ranked tokens were weighed using TF and TF-IDF. Depending upon the elements of poetry, five different linguistic features were experimented to develop classifier using machine learning algorithms using both scenarios. These five different features were divided into textual ones and poetic ones. Naive Bayes, Support Vector Machine, newlineHyper pipes, and K-nearest neighbour algorithms experimented with textual features (lexical, syntactic and semantic) as well as poetic features (orthographic and phonemic). Selection of these four algorithms was based on small experimentation with 240 poetries. newlineInitially, 10 different algorithms experimented on small dataset of 240 poetries. The aim of this small experimentation was to find the suitable machine learning algorithm for Punjabi poetry dataset. Selection of these 10 algorithms was based on machine learning algorithms being used in poetry classification problem.