我构建了一个支持12个提供商的自托管LLM代理(Claude、GPT-4o、Gemini、Ollama等)

📄 中文摘要

随着新的大语言模型(LLM)的不断推出,开发团队需要不断添加新的SDK、API密钥和错误处理逻辑,工作繁琐且重复。为了解决这一问题,开发者构建了llm-gateway,这是一个单一的Go二进制文件,作为应用程序与各个LLM提供商之间的中介。用户只需发送一种请求格式(兼容OpenAI),llm-gateway会根据模型名称将请求路由到正确的提供商,从而简化了与不同LLM的交互过程。

📄 English Summary

I built a self-hosted LLM proxy that supports 12 providers (Claude, GPT-4o, Gemini, Ollama...)

With the continuous emergence of new large language models (LLMs), development teams face the tedious task of adding new SDKs, API keys, and error handling logic repeatedly. To address this issue, the developer created llm-gateway, a single Go binary that acts as an intermediary between applications and various LLM providers. Users only need to send one request format (OpenAI-compatible), and llm-gateway routes the request to the appropriate provider based on the model name, thereby simplifying interactions with different LLMs.

Powered by Cloudflare Workers + Payload CMS + Claude 3.5

数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等