我建立了一个每月400美元的LLM成本监控系统(我学到了什么)

📄 中文摘要

六个月前,我收到了3000美元的LLM账单,完全不知道费用来源。为了应对这个问题,我建立了一个监控系统,能够跟踪每一次调用的成本。在运行一个使用GPT-4的小型SaaS时,我发现每个用户的LLM成本高达6美元,而我的产品每月仅收取20美元,导致我在每个高频用户身上都在亏损。通过监控系统,我开始跟踪每个用户的成本、每个功能的成本、每个模型的成本以及请求量,从而有效控制了支出。

📄 English Summary

I Built a $400/mo LLM Cost Monitoring System (Here's What I Learned)

Six months ago, I received a $3,000 LLM bill and had no idea where the costs were coming from. To address this issue, I built a monitoring system that tracks every call's cost. While running a small SaaS with GPT-4, I discovered that the LLM costs per user were around $6, while my product was priced at $20 per month, resulting in losses for every power user. With the monitoring system, I began tracking costs per user, costs per feature, costs per model, and request volume, which allowed me to effectively manage expenses.

Powered by Cloudflare Workers + Payload CMS + Claude 3.5

数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等