📄 中文摘要
大型语言模型的个性化通常通过提示工程或参数高效微调（如LoRA）实现。然而，书写风格难以通过单一提示精确捕捉，而LoRA微调则需要大量计算资源和基础设施。一种轻量级替代方案是，在解码时通过在Logit空间注入N-gram风格先验来引导冻结的大型语言模型。具体而言，训练一个N-gram模型，该模型能够捕获特定风格的文本特征，例如词汇选择、短语结构或句法偏好。在大型语言模型生成文本的每个解码步骤中，N-gram模型的概率分布被整合到大型语言模型的Logit输出中。这种整合通过调整N-gram模型对生成的下一个词的Logit值进行加权，从而在不修改大型语言模型底层参数的情况下，影响其生成文本的风格。通过这种方法，可以尝试将大型语言模型的输出风格与N-gram模型所学习的目标风格对齐。然而，这种方法的有效性受到N-gram模型捕获复杂风格特征能力的限制。N-gram模型擅长捕捉局部词序和频率模式，但对于更高级别的语义、篇章结构或长期连贯性等复杂风格属性，其表达能力较弱。例如，N-gram模型可能难以区分讽刺与严肃的语气，或者无法在长篇文本中保持一致的叙事风格。此外，Logit空间注入的强度和N-gram模型的选择对最终生成文本的风格控制效果至关重要，不当的参数设置可能导致生成文本的流畅性或语义连贯性下降。因此，尽管Logit空间注入提供了一种无需重新训练模型即可进行风格控制的轻量级方法，但其对于复杂风格的控制能力存在固有的局限性，特别是在需要深层语义理解和全局文本规划的场景中。
📄 English Summary
Limits of n-gram Style Control for LLMs via Logit-Space Injection
Personalization of large language models (LLMs) typically relies on prompt engineering or parameter-efficient fine-tuning like LoRA. However, writing style proves challenging to encapsulate in a single prompt, and LoRA fine-tuning demands substantial computational resources and infrastructure. A lightweight alternative involves steering a frozen LLM by injecting n-gram style priors into the logit space during decoding. Specifically, an n-gram model is trained to capture characteristic textual features of a particular style, including lexical choices, phrase structures, or syntactic preferences. At each decoding step of the LLM's text generation, the probability distribution from the n-gram model is integrated into the LLM's logit outputs. This integration involves weighting the logit values for the next word generated by the LLM based on the n-gram model's probabilities, thereby influencing the generated text's style without altering the LLM's underlying parameters. This approach attempts to align the LLM's output style with the target style learned by the n-gram model. Nevertheless, the efficacy of this method is constrained by the n-gram model's capacity to capture intricate style features. N-gram models excel at capturing local word sequences and frequency patterns but possess limited expressive power for complex stylistic attributes such as higher-level semantics, discourse structure, or long-term coherence. For instance, an n-gram model may struggle to differentiate between sarcastic and serious tones or maintain a consistent narrative style in extended texts. Furthermore, the strength of logit space injection and the choice of the n-gram model are crucial for the effectiveness of style control in the generated text; inappropriate parameter settings can lead to reduced fluency or semantic coherence. Consequently, while logit space injection offers a lightweight method for style control without retraining the model, it presents inherent limitations in controlling complex styles, especially in scenarios demanding deep semantic understanding and global text planning.