我追踪了一个月在 LLM API 上花费的每一个 Token——我学到了什么

📄 中文摘要

在使用 LLM API(如 OpenAI、Anthropic、Gemini 等)时,开发者可能会无意识地消耗大量 Token。作者在一个月内详细记录了所有项目中使用的 Token,结果令人惊讶。通过对多个项目的监控,发现系统提示是一个潜在的隐患。例如,某个应用在每次请求中发送了 4000 个 Token 的系统提示,这些 Token 虽然是付费的,但用户并未直接受益。此发现为开发者提供了优化 Token 使用的思路,帮助更好地控制成本。

📄 English Summary

I Tracked Every Token I Spent on LLM APIs for a Month — Here's What I Learned

Developers using LLM APIs like OpenAI, Anthropic, and Gemini may be unknowingly wasting a significant number of tokens. The author meticulously tracked every token spent across various projects over a month, leading to surprising insights. Monitoring multiple projects revealed that system prompts can be a hidden cost. For instance, one application was sending a 4,000-token system prompt with every request, resulting in charges for tokens that provided no direct benefit to users. This finding offers developers a pathway to optimize token usage and better manage expenses.

Powered by Cloudflare Workers + Payload CMS + Claude 3.5

数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等