学习选择视觉上下文示范

Multimodal Large Language Models (MLLMs) adapt to visual tasks through in-context learning (ICL), which is heavily reliant on the quality of demonstrations. The prevalent demonstration selection strategy employs unsupervised k-Nearest Neighbor (kNN) search. While this approach is straightforward, it proves sub-optimal for complex factual regression tasks, as it tends to select redundant examples that do not adequately represent the full output range of the task. This research reframes the selection process as a sequential decision-making problem and introduces Learning to Select Demonstrations (LSD), which trains a Reinforcement Learning agent to construct optimal demonstration sets. Utilizing a Dueling DQN with a query-centric Transformer Decoder, the agent learns a policy that maximizes the downstream performance of MLLMs.