通过逐层监督工程化可验证的变换器模块化

Transformers exhibit resistance to surgical control, where ablating an attention head deemed critical for specific tasks results in minimal behavioral changes due to distributed redundancy compensating for the damage. This Hydra effect obscures interpretability, as components can be identified through correlation but their causal roles remain unpredictable and uncontrollable. Architectural interventions are demonstrated to reveal hidden modularity. The proposed approach integrates dual-stream processing that separates token and contextual representations, per-layer supervision providing independent gradient signals at each depth, and gated attention that regularizes towards discrete activation patterns. Models trained with per-layer supervision exhibit ablation effects that are 5 to 23 times more pronounced.