Deep learning basedframeworksfor Automaticmedicalvideosummarization

Abstract

quotThe increasing use of medical video recordings in clinical practice and surgical education newlinepresents significant opportunities for knowledge extraction and decision support. However, newlineeffectively analyzing these videos remains a considerable challenge due to their extended dura newlinetion, visual redundancy, and complex semantics. A single surgical procedure may span several newlinehours, generating large video files that are difficult to browse, store, or manually review. Med newlineical videos differ significantly from general-purpose visual content due to their limited visual newlinevariation. Subtle changes, such as slight movements of surgical tools or minor tissue responses, newlineare often clinically significant. However, these changes are typically difficult to detect without newlinespecialized contextual understanding. This presents unique challenges for automated analysis, newlineas critical events may be visually indistinct yet carry important medical implications. The lack newlineof clear scene transitions and the high visual similarity across frames make automatic summa newlinerization particularly difficult. Additionally, the scarcity of annotated datasets, due to the cost newlineand expertise required for labeling, further limits the effectiveness of supervised learning ap newlineproaches. This thesis addresses these challenges through a series of novel, context-aware video newlinesummarization frameworks designed for medical and dynamic procedural content. newlineThe first contribution introduces an unsupervised, resolution-aware summarization frame newlinework, combiningCNN-basedmulti-layerfeature extraction (MobileNetV3, GoogleNet, DenseNet newline161, ResNet50, EfficientNet-B7), spatial pyramid pooling (SPP), and Bi-LSTM-based tempo newlineral modeling. This approach effectively captures multi-scale spatio-temporal features and iden newlinetifies key events across diverse medical datasets. Among the architectures explored, EfficientNet newlineB7 stands out with the highest precision of 87.2% for Endoscapes-CVS201 dataset and the newlinehighest recall of 88.3% for JIGSAWS, achieving an overall F1-score of 87.1%.