CLUTCH：用于解锁文本条件的手部动作建模的上下文化语言模型

Hands play a crucial role in daily life, yet the modeling of natural hand motions remains underexplored. Existing methods for text-to-hand-motion generation or hand animation captioning rely on limited, studio-captured datasets, making them costly to scale to 'in-the-wild' settings. Furthermore, contemporary models and their training schemes struggle with capturing animation fidelity in text-motion alignment. To address these challenges, the '3D Hands in the Wild' (3D-HIW) dataset is introduced, containing 32K 3D hand-motion sequences with aligned text. Additionally, CLUTCH, an LLM-based hand animation system, is proposed, featuring two critical innovations: (a) SHIFT, a novel VQ-VAE architecture for tokenizing hand motion, and (b) a geometric refinement stage to enhance animation quality.