We developed a deep learning (DL)-based framework, Surf-X, to estimate real-time 3D liver motion. Surf-X synergizes two imaging modalities, optical surface imaging and x-ray imaging, to track the 3D liver motion. By incorporating prior knowledge of motion learnt from patient-specific 4D-CTs, Surf-X progressively solves the liver motion in two steps: firstly from an optical surface image via learnt internal-external correlations; and secondly from directly-observed motion on an x-ray projection. Surf-X combines the complementary information from surface and x-ray imaging and solves liver motion more accurately and robustly than either modality alone, all at a temporal resolution of <100 milliseconds.
|