Investigations on affective computing to improve classroom engagement analysis in higher education by deep learning

Abstract

The learning and teaching experience can be improved by using approaches that are not obtrusive to perform a comprehensive student engagement analysis throughout the classroom. In these modern times, when courses are conducted online, it is vital to accurately measure the levels of participation that each individual student has. It is crucial and essential to provide assistance to educators so that they may annotate and comprehend the signifcant learning rate of the students. A system that can perceive data and transpire it into information automates the learning and teaching experience in a classroom. In this study, videos are collected from online and ofand#64258;ine classes that have one single student per frame or many students per frame and are analysed for emotions and behavioural engagement through a multimodal system. newlineLarge amounts of video data processing call for an increase in the hardware resources newlineas well as the time required for processing images. This is particularly true in a newlineclassroom setting, where there are a large number of frames to analyse each and every minute in order to handle classroom involvement detection. Hierarchical Video newlineSummarization is used as a preliminary step on the videos to detect important frames newlinethat have the sum of all the information in the local neighborhood. These key frames newlineserve as important information units that provide details of facial emotions and behavioural aspects. The local maxima estimation based on the frst derivative provides summative information about the local neighborhood. The key frames serve newlineas an input for face detection and emotional analysis. In this research, the method newlinecan perform video summarization on a varied category of videos and with different newlineresolutions. Face detection in a temporal environment have not been trivial. Though there are methods that can identify multiple faces with varied sizes in a frame, it is still a current research topic to address false localization of faces in a frame.