CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction

Lixin Yang
Xinyu Zhan
Kailin Li
Wenqiang Xu
Jiefeng Li
Cewu Lu

Shanghai Jiao Tong University

In ICCV 2021



Modeling the hand-object (HO) interaction not only requires estimation of the HO pose, but also pays attention to the contact due to their interaction. Significant progress has been made in estimating hand and object separately with deep learning methods, simultaneous HO pose estimation and contact modeling has not yet been fully explored. In this paper, we present an explicit contact representation namely Contact Potential Field (CPF), and a learning-fitting hybrid framework namely MIHO for Modeling the Interaction of Hand and Object. In CPF, we treat each contacting HO vertex pair as a spring-mass system. Hence the whole system forms a potential field with minimal elastic energy at the grasp position. Extensive experiments on the two commonly used benchmarks have demonstrated that our method can achieve state-of-the-art in several reconstruction metrics, and allow us to produce more physically plausible HO pose even when the ground-truth exhibits severe interpenetration or disjointedness.


The architecture of the hybrid model MIHO


Paper and Supplementary Material

Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, Cewu Lu
CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction
In ICCV, 2021.



This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.