Augmenting the Effectiveness of Detecting Cyber Threats using Big Data Analytics

Abstract

With ever increasing cyber threats by the day, organizations face significant challenges in safeguarding their networks and data. The increasing complexity of cyber-attacks, including sophisticated techniques like advanced persistent threats (APTs), underscores the need for innovative and robust cyber threat detection methods. Traditional approaches, which often rely on either machine learning or search engine algorithms, have limitations in adaptability, accuracy, and the ability to detect emerging and unknown threats. This research aims to address these limitations by developing a novel hybrid cyber threat detection framework that utilizes the strengths of machine learning algorithms combined with customized search engine methodologies to enhance detection accuracy and scalability. newlineThe motivation for this research arises from the gap in integrating comprehensive, real-time detection mechanisms that can handle high-dimensional data streams and adapt to evolving threat patterns, particularly on platforms like social media, where vast amounts of data are generated continuously. With the surge in social media and the prevalence of cyber threats on these platforms, traditional detection methods struggle to keep up, necessitating the need for a dynamic and adaptive approach. Furthermore, existing intrusion detection systems (IDS) often face challenges with scalability, false positives, and timely adaptability, emphasizing the demand for a hybrid solution that can simultaneously achieve high detection rates and maintain efficiency. newlineThe primary objective of this research is to create an advanced cyber threat detection framework that incorporates feature extraction techniques to effectively analyze high-dimensional data. This research is divided into three distinct phases. In the first phase, feature extraction techniques from social media platforms are explored to enhance cyber threat detection in real time.