Fusion of Kinect depth data with trifocal disparity estimation for near real-time high quality depth maps generation

Guillaume Boisson; Paul Kerbiriou; Valter Drazic; Olivier Bureller; Neus Sabater; Arno Schubert

doi:10.1117/12.2036530

6 March 2014 Fusion of Kinect depth data with trifocal disparity estimation for near real-time high quality depth maps generation

Guillaume Boisson, Paul Kerbiriou, Valter Drazic, Olivier Bureller, Neus Sabater, Arno Schubert

Author Affiliations +

Proceedings Volume 9011, Stereoscopic Displays and Applications XXV; 90110J (2014) https://doi.org/10.1117/12.2036530
Event: IS&T/SPIE Electronic Imaging, 2014, San Francisco, California, United States

Abstract

Generating depth maps along with video streams is valuable for Cinema and Television production. Thanks to the improvements of depth acquisition systems, the challenge of fusion between depth sensing and disparity estimation is widely investigated in computer vision. This paper presents a new framework for generating depth maps from a rig made of a professional camera with two satellite cameras and a Kinect device. A new disparity-based calibration method is proposed so that registered Kinect depth samples become perfectly consistent with disparities estimated between rectified views. Also, a new hierarchical fusion approach is proposed for combining on the flow depth sensing and disparity estimation in order to circumvent their respective weaknesses. Depth is determined by minimizing a global energy criterion that takes into account the matching reliability and the consistency with the Kinect input. Thus generated depth maps are relevant both in uniform and textured areas, without holes due to occlusions or structured light shadows. Our GPU implementation reaches 20fps for generating quarter-pel accurate HD720p depth maps along with main view, which is close to real-time performances for video applications. The estimated depth is high quality and suitable for 3D reconstruction or virtual view synthesis.

Citation Download Citation

Guillaume Boisson, Paul Kerbiriou, Valter Drazic, Olivier Bureller, Neus Sabater, and Arno Schubert "Fusion of Kinect depth data with trifocal disparity estimation for near real-time high quality depth maps generation", Proc. SPIE 9011, Stereoscopic Displays and Applications XXV, 90110J (6 March 2014); https://doi.org/10.1117/12.2036530

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available