Efficiency gain of paired split-plot designs in MRMC ROC studies

Weijie Chen; Qi Gong; Brandon D. Gallas

doi:10.1117/12.2293741

7 March 2018 Efficiency gain of paired split-plot designs in MRMC ROC studies

Weijie Chen, Qi Gong, Brandon D. Gallas

Proceedings Volume 10577, Medical Imaging 2018: Image Perception, Observer Performance, and Technology Assessment; 105770F (2018) https://doi.org/10.1117/12.2293741
Event: SPIE Medical Imaging, 2018, Houston, Texas, United States

Abstract

The widely used multi-reader multi-case ROC study design for comparing imaging modalities is the fullycrossed (FC) design: every reader reads every case of both modalities. In this work, we investigate the paired split-plot (PSP) designs that allow for reduced cost and increased flexibility compared to the FC design. In the PSP design, patient images from two modalities are read by the same readers, thereby the readings are paired across modalities. However, within each modality, not every reader reads every case. Instead, both the readers and the cases are partitioned into a number of groups and each group of readers read their own group of cases - a split-plot design. Using the U-statistic based variance analysis for AUC (i.e., area under the ROC curve), we show analytically that, with a fixed number of readings per reader, substantial statistical efficiency can be gained by the PSP design as compared to the FC design. Equivalently, we show that the PSP design can achieve the same statistical power as the FC design with substantially reduced number of readings. However, the efficiency/power gain of the PSP design comes with the increased cost of collecting a larger number of truthverified patient cases than the FC design. This means that one can trade off between different sources of cost and choose a least burdensome design. We demonstrate the advantages of the PSP design with a real-world reader study for the comparison of full field digital mammography with screen-film mammography.

Conference Presentation

Citation Download Citation

Weijie Chen, Qi Gong, and Brandon D. Gallas "Efficiency gain of paired split-plot designs in MRMC ROC studies", Proc. SPIE 10577, Medical Imaging 2018: Image Perception, Observer Performance, and Technology Assessment, 105770F (7 March 2018); https://doi.org/10.1117/12.2293741

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available