Development of multi modal detection system for campus environments using visual and audio based solutions

Abstract

The development of intelligent systems tailored for campus environments demands robust newlinesolutions capable of addressing unique challenges, such as dense pedestrian traffic, newlinemixed movement of cyclists and vehicles, and complex, unstructured road configurations. newlineTo ensure safety and navigational efficiency in such settings, this research newlineproposes a comprehensive multimodal detection framework that integrates visual and newlineaudio-based sensing capabilities. The system combines three key modules: visual object newlinerecognition, lane boundary segmentation, and emergency siren classification, offering newlinea holistic approach to real-time situational awareness. newlineAt the core of the visual processing module is the YOLOv5s architecture, selected newlinefor its efficiency on edge devices. The model was trained on a custom campus-specific newlinedataset to detect pedestrians, vehicles, cyclists, and environmental obstacles. Performance newlineevaluation using metrics such as precision (0.851), recall (0.831), and mean newlineAverage Precision (mAPat0.5 of 0.843) demonstrated the model s reliability in minimizing newlinefalse positives and false negatives in dynamic campus conditions. newlineTo complement this, the lane detection component employs the U-Net segmentation newlinemodel, optimized through rigorous hyperparameter tuning and validated using five-fold newlinecross-validation. This model achieved a training accuracy of 99.93%, a validation accuracy newlineof 99.81%, and a test accuracy of 99.78%, along with an IoU score of 0.8861 and newlineF1-score of 0.9396 indicating strong generalization to real-world lane visibility conditions, newlineincluding curved or partially marked paths. newlineFor emergency response scenarios, the framework incorporates audio classification newlineusing machine learning methods applied to temporal and spectral features. Classifiers newlinesuch as Support Vector Machine (SVM), Random Forest (RF), and a stacked ensemble newlinewere evaluated. The SVM and ensemble models achieved the highest classification newlineaccuracy of 99.5%. Data augmentation techniques simulating background noise and newlineDoppler effects enhanced