五个模型，467个动作，一个赢家——我们在真实代码生成中比较LLM的经验

Five AI models were tested on the same task 467 times, with each run producing a complete deployable website, including HTML, CSS, JavaScript, and assets. The central question was whether cheaper models could match Claude Sonnet's capabilities in production code generation. The findings indicate that cheaper models do not reach Claude Sonnet's level, but the longer analysis reveals more interesting insights. The models tested spanned a cost range of 15 times, including Claude Sonnet 4.6, assumed to be the gold standard, and Claude Haiku 4.5. This research provides an in-depth understanding of the performance of different models in real-world applications.