[Title] HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video
[Keyword] Category-agnostic Hand-Object Reconstruction, SDF, NeRF
[Journal] CVPR, 2024
[arXiv] https://arxiv.org/abs/2311.18448
[Summary]
Most existing methods use a pre-scanned template to reconstruct the Hand-Object Interaction(HOI). In this paper, a novel method, called HOLD, for category-agnostic HOI reconstruction without any pre-scanned template is proposed. First, HOLD initializes the pose of hands and objects. Hand pose means MANO parameters, which are obtained by using the off-the-shelf model. Object pose means rotation and translation of the point clouds, which are obtained by Structure-from-Motion(SfM). Each initialized poses are fed into the input of HOLD-Net, which reconstructs a mesh using the Signed Distance Function(SDF) and the Marching Cubes algorithm, and predicts a color using a NeRF-based method. Reconstructed meshes are used for interaction prior, which is used for refining initialized poses. After that, refined poses are fed into HOLD-Net again, and it reconstructs the fully-refined mesh.
댓글남기기