MHMC: Real-Time Hand Motion Capture Using Millimeter-Wave Radar

F Deng, H Zhou, P Xia, Z Wang… - 2022 IEEE 28th …, 2023 - ieeexplore.ieee.org
F Deng, H Zhou, P Xia, Z Wang, X Wang, XY Li
2022 IEEE 28th International Conference on Parallel and …, 2023ieeexplore.ieee.org
Hand motion capture is essential for emerged augmented reality (AR) and virtual reality (VR)
applications. We propose MHMC, a real-time hand motion capture system using millimeter-
wave (mmWave) radar with the advantages of lighting independence, privacy protection and
good user experience. When implementing such a system, we design a novel vertical neural
network architecture to infer complex hand motion from limited resolution data of mmWave
radar, a vision-based labeling method to acquire adequate labeled data, and effective loss …
Hand motion capture is essential for emerged augmented reality (AR) and virtual reality (VR) applications. We propose MHMC, a real-time hand motion capture system using millimeter-wave (mmWave) radar with the advantages of lighting independence, privacy protection and good user experience. When implementing such a system, we design a novel vertical neural network architecture to infer complex hand motion from limited resolution data of mmWave radar, a vision-based labeling method to acquire adequate labeled data, and effective loss functions to avoid abnormal output. The experimental results demonstrate that MHMC achieves similar accuracy as visionbased methods, along with fast processing speed, i.e., 7. 3ms per frame, for supporting real-time motion capture.
ieeexplore.ieee.org