基于多模态大语言模型的零-shot人机交互检测无检测器依赖的交互识别

Zero-shot Human-object interaction (HOI) detection aims to locate humans and objects in images and recognize their interactions. While advances in open-vocabulary object detection provide promising solutions for object localization, interaction recognition (IR) remains challenging due to the combinatorial diversity of interactions. Existing methods, including two-stage approaches, tightly couple IR with specific detectors and rely on coarse-grained vision-language model (VLM) features, limiting generalization to unseen interactions. A decoupled framework is proposed that separates object detection from IR and leverages multi-modal large language models (MLLMs) for zero-shot IR. A deterministic generation method is introduced to enhance the accuracy and flexibility of interaction recognition.