LLM提供商故障的断路器

Every application powered by large language models (LLMs) relies on external providers such as OpenAI, Anthropic, Google, or self-hosted models. These providers can experience outages, leading to request failures, increased rate limits, and ballooning latency. Without a circuit breaker, applications continue to send requests to a non-responsive API, wasting budget, accumulating timeouts, and providing a poor user experience. A circuit breaker detects when the downstream service is failing and stops requests for a cooldown period. This approach is not about retrying harder; it is about failing fast and deliberately to protect the rest of the system. Utilizing a Redis-backed failure state allows for rapid load shedding and automatic recovery, ensuring consistency across restarts.