通过放弃 OpenAI 实现 375ms 语音到语音延迟的经验

The author, running an AI automation agency for healthcare clients, faced significant latency issues while building voice agents using a standard tech stack (Twilio → Vapi/Retell → GPT-4o → ElevenLabs). The latency typically ranged from 800ms to 1200ms, resulting in a poor user experience where users would interrupt the bot, causing it to continue speaking for a second before realizing. Each time audio left the server to reach OpenAI or ElevenLabs, approximately 200ms was lost. To combat this, the author made the drastic decision to purchase dedicated hardware, transitioning to a Bare Metal NVIDIA Blackwell solution, which successfully reduced latency to 375ms and improved user interaction significantly.