📄 中文摘要
LLM 推理在单节点设置中看似“只是另一个 API”，但当延迟激增、队列积压以及 GPU 内存使用率达到 95% 时，问题变得复杂。监控在超越单节点设置或优化吞吐量时变得至关重要，传统的 API 指标已不足以满足需求。需要关注令牌、批处理行为、队列时间和 KV 缓存压力等关键指标，这些是现代 LLM 系统的真正瓶颈。该内容是更广泛的可观察性和监控指南的一部分，涵盖了监控与可观察性的基本概念、Prometheus 架构和生产最佳实践。
生产环境中监控 LLM 推理（2026）：使用 Prometheus 和 Grafana 监控 vLLM、TGI 和 llama.cpp
📄 English Summary
Monitor LLM Inference in Production (2026): Prometheus & Grafana for vLLM, TGI, llama.cpp
LLM inference may appear as just another API until latency spikes, queues back up, and GPUs are at 95% memory usage without clear reasons. Monitoring becomes critical when moving beyond a single-node setup or optimizing for throughput, as traditional API metrics fall short. Visibility into tokens, batching behavior, queue time, and KV cache pressure becomes essential, highlighting the real bottlenecks in modern LLM systems. This content is part of a broader observability and monitoring guide that covers the fundamentals of monitoring vs. observability, Prometheus architecture, and best practices for production environments.