Content-based retrieval of video segments from minimally invasive surgery videos using deep convolutional video descriptors and iterative query refinement

Deepak R. Chittajallu; Arslan Basharat; Paul Tunison; Samantha Horvath; Katerina O. Wells; Steven G. Leeds; James W. Fleshman; Ganesh Sankaranarayanan; Andinet Enquobahrie

doi:10.1117/12.2509985

8 March 2019 Content-based retrieval of video segments from minimally invasive surgery videos using deep convolutional video descriptors and iterative query refinement

Deepak R. Chittajallu, Arslan Basharat, Paul Tunison, Samantha Horvath, Katerina O. Wells, Steven G. Leeds, James W. Fleshman, Ganesh Sankaranarayanan, Andinet Enquobahrie

Author Affiliations +

Proceedings Volume 10951, Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling; 109512Q (2019) https://doi.org/10.1117/12.2509985
Event: SPIE Medical Imaging, 2019, San Diego, California, United States

Abstract

Despite a strong evidence of the clinical and economic benefits of minimally invasive surgery (MIS) for many common surgical procedures, there is a gross underutilization of MIS in many US hospitals, potentially due to its steep learning curve. Intraoperative videos captured using a camera inserted into the body during MIS procedures are emerging as an invaluable resource for MIS education, skill assessment and quality assurance. However, these videos often have a duration of several hours and there is a pressing need for automated tools to help surgeons quickly find key semantic segments of interest within MIS videos. In this paper, we present a novel integrated approach for facilitating content-based retrieval of video segments that are semantically similar to a query video within a large collection of MIS videos. We use state-of-theart deep 3D convolutional neural network (CNN) models pre-trained on large public video classification datasets to extract spatiotemporal features from MIS video segments and employ an iterative query refinement (IQR) strategy where in a support vector machine (SVM) classifier trained online based on relevance feedback from the user is used to refine the search results iteratively. We show that our method outperforms the state-of-the-art on the SurgicalActions160 dataset containing 160 video clips of typical surgical actions in gynecologic MIS procedures.

Citation Download Citation

Deepak R. Chittajallu, Arslan Basharat, Paul Tunison, Samantha Horvath, Katerina O. Wells, Steven G. Leeds, James W. Fleshman, Ganesh Sankaranarayanan, and Andinet Enquobahrie "Content-based retrieval of video segments from minimally invasive surgery videos using deep convolutional video descriptors and iterative query refinement", Proc. SPIE 10951, Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, 109512Q (8 March 2019); https://doi.org/10.1117/12.2509985

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available