📄 中文摘要
系统提示是对语言模型的概率性建议,而非防火墙、访问控制列表或政策引擎。它仅仅是文本,由模型根据上下文窗口、训练数据及当前用户输入进行评估。将系统提示视为安全控制的常见误解可能导致严重后果。系统提示在安全控制方面的失败主要体现在三个方面:提示注入、上下文理解不足以及对用户意图的错误解读。这些因素使得依赖系统提示来确保安全性变得极其危险。
📄 English Summary
Why Your System Prompt Is Not a Security Control
A system prompt serves as a probabilistic suggestion to a language model, not as a firewall, access control list, or policy engine. It is merely text evaluated by the model against its context window, training data, and current user input. The misconception of treating system prompts as security controls can lead to significant risks. System prompts fail as security controls in three main ways: prompt injection, insufficient context understanding, and misinterpretation of user intent. These factors render reliance on system prompts for security highly dangerous.
Powered by Cloudflare Workers + Payload CMS + Claude 3.5
数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等