📄 中文摘要
持续微调旨在将预训练的骨干网络顺序适应于新任务，同时保持对早期任务的性能，尽管这些任务的数据不再可用。现有方法主要分为输入适应和参数适应两类。输入适应方法依赖于在测试时检索最相关的提示，但需要不断学习一个检索函数，容易导致遗忘。参数适应方法则使用固定的输入嵌入函数，以实现无检索预测并避免遗忘，但牺牲了表示的适应性。为结合两者的优点，提出了一种新的参数适应方法，能够在测试时自适应地使用输入嵌入。该方法有效提高了模型在新任务上的表现，同时保持了对旧任务的记忆能力。
具有可证明准确性和无参数任务检索的持续微调
📄 English Summary
Continual Fine-Tuning with Provably Accurate and Parameter-Free Task Retrieval
Continual fine-tuning aims to adapt a pre-trained backbone to new tasks sequentially while preserving performance on earlier tasks whose data are no longer available. Existing approaches fall into two categories: input-adaptation and parameter-adaptation. Input-adaptation methods rely on retrieving the most relevant prompts at test time but require continuously learning a retrieval function, which is prone to forgetting. Parameter-adaptation methods use a fixed input embedding function to enable retrieval-free prediction and avoid forgetting, but sacrifice representation adaptability. To combine their strengths, a new parameter-adaptation method is proposed that enables adaptive use of input embeddings during test time, effectively enhancing the model's performance on new tasks while maintaining memory of previous tasks.