Transliteration Between English and Other Indian Languages A Machine Learning Based Approach

Abstract

Machine Translation involves conversion of text in one language to equivalent text in another language while retaining the semantics of the original text. However, there are some classes of words that need to be retained as such with same phonetics in the changed script. Machine Transliteration is used in Machine Translation of such words in a sentence e.g, named entities. In this work, we have created a Machine Transliteration system for transliteration of English lexical words to Hindi and Telugu and Hindi lexical words to Telugu. newlineAn important pre-requisite for creating a machine learning system for transliteration is a parallel database providing the ground truth for training. A phonetically rich parallel database dubbed DB2 has been constructed using phonetic symbols given in Oxford s Advanced Learner s Dictionary for British pronunciations. DB2 has 2200 words. In this database, an attempt is made to capture the diversity of letter and grapheme pronunciations with easily pronounceable English words. Transliteration systems are trained using this database for training. The training and testing data was segmented into letters and graphemes using three different segmentation methods, each yielding a separate transliteration model for English to Hindi and English to Telugu. It was observed that systems trained using the smaller DB2 and tested on a large database (DB1) gave comparable results to the systems trained on a larger database (DB1) and tested on the smaller database (DB2). newlineIn Hindi and Telegu, some of the letters can be considered as phonetically similar letters and can be substituted directly to give a phonetically similar transliteration. Therefore, in this work, a Modified Edit Distance (MED) based evaluation system was developed to evaluate the trained transliteration system. For transliterating Hindi text to Telugu, a UTF-8 based conversion is used. In this method the gaps between the pronunciation of Hindi and Telugu are also considered and transliteration is performed considering these gaps. newlineThe transliteration system developed for English to Hindi and Telugu can be trained for transliterating English words to other Indian languages viz., Gujarati, Marathi, Bengali and Kannada. newline newline