High-speed vision technology in which not just image capturing and recording but also image processing are executed simultaneously at high frame rates, exceeding video rates (30 Hz), has recently been considered an important technology for various applications, such as robotics, and man-machine interfaces. However, the image processing performed in conventional high-speed vision systems is mainly based on two-dimensional pattern recognition. In order to extend the possibilities of this technology, here we focus on real-time three-dimensional sensing at the speeds achievable by high-speed vision systems. Although a related approach for high-speed 3D sensing can achieve a frame rate of over 200 fps, there are disadvantages, including the need for multiple captured frames during sensing, a limited measurement range and low resolution. Our proposed real-time 3D sensing system consists of a projector and two cameras. By projecting a well-designed segmented pattern and using three-viewpoint epipolar constraints, the proposed system can obtain 3D points at high speed. The developed system robustly obtained a 3D shape at 450–500 fps in real time.