Extraction of Multiword Expressions from Hindi Text Document

Abstract

Multiword expressions (MWEs) are a significant challenge in many fields of newlinelanguage technology. Multiword extraction from random text data has grown in newlinepopularity among the NLP community. This topic of research is strongly connected to statistical analysis and artificial intelligence. This thesis presents a detailed literature assessment and numerous strategies for building an automated newlineMultiword extraction system. The overall contribution of the thesis has been newlinedivided into six parts. newlineIn this study, a method of Hindi MWEs has been proposed, and the significance of boundary threshold calculations in this study. The main objective of this dissertation work is to develop a generalized mechanism for the extraction of Hindi multiword expressions. The primary goal of this research is to build an approach for extracting Hindi MWEs using syntactical and statistical idiosyncrasy newline(i.e., the structure of linguistic patterns and association) and context connection newlinebetween their constituent words. Various combination strategies of different newlineclassifiers based on these properties may be applied to develop a multi word extraction mechanism. Hence, creating a best-performing combination strategy is also an objective of this dissertation. newlineThere are various hurdles in designing a method using these properties. In statistical filtering, calculating the boundary threshold is a challenging task. newlineAnother issue is to combine multiple filters since different combination strategies newlinemay be possible. Thus, recognizing the best combination strategy is also a challenge. In the Hybrid method, Semantic Similarity has been used. The study developed a web application using the Flask framework to automatically extract the Hindi MWEs using the Association based and Hybrid methods. newlineThe methods, evaluation results, and findings in each contribution have been presented in different chapters. The proposed technique is evaluated using the HDTB Treebank and TDIL dataset, which is freely available.