一种语言，两种书写：探究大型语言模型概念表示中的书写不变性

This study investigates whether the features learned by Sparse Autoencoders (SAEs) represent abstract meaning or are tied to the way text is written. Using Serbian digraphia as a controlled testbed, where Serbian can be written in both Latin and Cyrillic scripts with a near-perfect character mapping, it allows for variation in orthography while keeping meaning constant. Notably, these scripts are tokenized completely differently, sharing no tokens. Analyzing SAE feature activations across the Gemma model family (270M-27B parameters) reveals that identical sentences in different Serbian scripts activate highly overlapping features, significantly exceeding random baselines. Remarkably, changing the script results in less representational difference, indicating a certain robustness of SAEs in capturing abstract meaning.