我比较了五种跟踪 LLM API 成本的方法(只有一种能实时工作)
📄 中文摘要
在按令牌计费的计划中,许多用户可能曾对账单感到惊讶。主要的成本跟踪选项包括:1. 提供商仪表板(如 OpenAI 和 Anthropic),更新频率每几小时一次,但无法提供每个会话的详细信息,且免费,存在查看时已超支的问题;2. LiteLLM Proxy,能够跨多个提供商跟踪成本,需自行托管,免费且开源,但设置复杂,无法实时查看;3. Helicone,提供优秀的日志记录和分析,需通过其代理路由流量,提供免费层,但依赖仪表板,缺乏实时反馈。不同方案各有优缺点,用户需根据需求选择合适的工具。
📄 English Summary
I Compared 5 Ways to Track LLM API Costs (Only One Works in Real Time)
For users on pay-per-token plans, unexpected bills can be a common issue. The main options for tracking costs include: 1. Provider Dashboards (like OpenAI and Anthropic) that update every few hours but lack per-session breakdowns; they are free but can lead to overspending by the time users check. 2. LiteLLM Proxy, which tracks costs across multiple providers, requires self-hosting, is free and open-source, but involves setup overhead and does not provide real-time visibility. 3. Helicone offers great logging and analytics but requires routing traffic through its proxy, has a free tier, and is dashboard-based, lacking real-time feedback. Each option has its pros and cons, and users need to choose based on their specific needs.
Powered by Cloudflare Workers + Payload CMS + Claude 3.5
数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等