大型语言模型中的能量溢出

This study reinterprets the final Large Language Model (LLM) softmax classifier as an Energy-Based Model (EBM), decomposing the sequence-to-sequence probability chain into multiple interacting EBMs during inference. This principled approach allows for tracking 'energy spills' during decoding, which are empirically shown to correlate with factual errors, biases, and failures. Similar to the work of Orgad et al. (2025), the method localizes the exact answer token and subsequently tests for hallucinations. Crucially, this is achieved without requiring trained probe classifiers or activation ablations. Instead, two completely training-free metrics are introduced, derived directly from output logits: spilled energy, which captures the discrepancy between energy values across the decoding process.