Multi-rigid-body Approximation of Human Hands Enables Real-Time Physics Simulation with MANO Models

Accurate simulation of the human hand remains a significant challenge for creating realistic digital twins, demanding models that combine anatomical detail with computational efficiency. Bin Zhao, Yiwen Lu, and Haohua Zhu, from DexRobot Co. Ltd., alongside Xiao Li and Sheng Yi, now present a complete method for constructing multi-rigid-body approximations of hands that achieve precisely this balance. Their approach begins with motion capture data and builds a personalised hand model, converting it into a format suitable for real-time physics simulation.

The team overcomes a key technical hurdle by accurately translating complex hand movements into the constraints of a rigid-body system, enabling the creation of a digital hand that closely replicates human motion and successfully executes diverse manipulation tasks with sub-centimetre accuracy. This advancement promises to significantly improve the realism and functionality of digital twins for applications ranging from robotics to virtual reality.

Realistic Hand Models via Multi-Rigid-Bodies This research details a complete pipeline for creating realistic and computationally efficient hand models, crucial for applications like digital twins, teleoperation, and human-robot collaboration. The core innovation lies in approximating the human hand using a multi-rigid-body representation, carefully balancing visual fidelity with real-time performance. Scientists developed a mathematically grounded framework that accurately projects the unconstrained rotations of a detailed hand model onto the constrained movements of a simplified, rigid-body representation, utilizing closed-form solutions for single-axis joints and a corrected iterative method for more complex, two-axis joints. This ensures accurate kinematic behaviour and enables rigid-body physics simulation at rates exceeding 1000Hz, a significant advancement over traditional mesh-based methods. The method achieves sub-centimeter tracking error and high success rates in manipulation tasks, demonstrating a significant advancement in realistic hand simulation and facilitating applications in teleoperation, virtual training, and collaborative robotics.

Hand Model Conversion via Kinematic Constraints Scientists have developed a complete pipeline for constructing multi-rigid-body approximations of human hands, enabling realistic and efficient digital twin applications. The study begins with motion capture data, constructing a personalized hand model and converting it into a universally compatible format with anatomically consistent joint axes. A key achievement is accurately mapping the unconstrained rotations of the hand model to the kinematically constrained joint angles of the rigid-body model, utilizing closed-form solutions for single degree-of-freedom joints and an iterative method corrected by the Baker-Campbell-Hausdorff (BCH) formula for two degree-of-freedom joints. This BCH-corrected method converges rapidly, typically requiring only three to five iterations, and produces joint angles that closely approximate the original pose. Quantitative evaluation demonstrates sub-centimeter reconstruction error and successful grasp execution across diverse manipulation tasks, confirming the accuracy and effectiveness of the methodology. Hand Motion to Constrained Rigid Body Mapping Scientists have developed a complete pipeline for constructing multi-rigid-body approximations of human hands, enabling realistic and efficient digital twin applications. The work begins with capturing human hand motion and fitting a personalized hand model to establish precise correspondence between observed movement and a parametric representation. This allows for the creation of a segmented mesh, defining rigid links and anatomically consistent joint axes. The core achievement lies in accurately mapping the unconstrained rotations of the hand model to the kinematically constrained joint angles of the rigid-body model, utilizing closed-form solutions for single degree-of-freedom joints and an iterative method utilizing the Baker-Campbell-Hausdorff (BCH) formula to handle the non-linear interaction between rotation axes. Experiments demonstrate sub-centimeter reconstruction error, validating the accuracy of the approach, and successful grasp execution across diverse manipulation tasks.

Realistic Hand Simulation via Rigidity Reduction This work presents a complete pipeline for creating multi-rigid-body approximations of the human hand, balancing realistic visual appearance with the computational efficiency needed for real-time simulation. A key achievement is a mathematically grounded framework that accurately projects the unconstrained rotations of a detailed hand model onto the constrained movements of a simplified, rigid-body representation, utilizing closed-form solutions for single-axis joints and a corrected iterative method for more complex, two-axis joints. Experiments demonstrate the effectiveness of this approach, achieving sub-centimeter tracking error when replaying captured human hand movements in a digital twin environment, while simultaneously maintaining simulation rates exceeding 1000Hz. The authors acknowledge that the current method focuses on replicating captured motions and future research will explore learned projection methods tailored to individual hand kinematics. 👉 More information 🗞 Multi-Rigid-Body Approximation of Human Hands with Application to Digital Twin 🧠 ArXiv: https://arxiv.org/abs/2512.07359 Tags: