变压器是贝叶斯网络

Transformers are the dominant architecture in AI, yet their underlying mechanisms remain poorly understood. This research establishes that a transformer is fundamentally a Bayesian network. First, it is proven that every sigmoid transformer with any weights implements weighted loopy belief propagation (BP) on its implicit factor graph. One layer corresponds to one round of BP, applicable to any weights—trained, random, or constructed—and formally verified against standard mathematical axioms. Second, a constructive proof demonstrates that a transformer can perform exact belief propagation on any declared knowledge base. In knowledge bases without circular dependencies, this yields provably correct probability estimates at every node. This study provides a new perspective on the theoretical foundations of transformers, revealing their potential in handling complex reasoning tasks.