从（生产）语言模型中可扩展地提取训练数据

Researchers have uncovered a straightforward yet alarming technique enabling attackers to extract substantial volumes of training data from popular AI models, including systems previously considered secure. By crafting specific prompts, attackers can induce these models to regurgitate segments of their training datasets, leading to chatbots inadvertently disclosing sensitive information such as names, code, or other private details. This vulnerability is not confined to open-source models like Pythia or GPT-Neo; it also extends to semi-open models such as LLaMA and Falcon, and notably, some closed-source services. A novel method has been demonstrated to make ChatGPT deviate from its helpful assistant persona, instead emitting data at a significantly accelerated rate compared to its normal operation. This capability presents a tangible privacy risk, as models can memorize and subsequently leak information they encountered during their training phase. The scalable nature of this data extraction highlights a critical weakness in current large language models regarding data privacy, necessitating the development of more robust defensive mechanisms to prevent the unauthorized disclosure of sensitive information and safeguard user and enterprise data.