📄 中文摘要
AWS 宣布推出两个新的 Amazon CloudWatch 指标，分别是 TimeToFirstToken 和 EstimatedTPMQuotaUsage。这些指标旨在提升 Amazon Bedrock 的推理工作负载的操作可视化。TimeToFirstToken 反映了从请求到第一个令牌生成所需的时间，而 EstimatedTPMQuotaUsage 则提供了对每分钟处理能力的估算。这些指标能够帮助用户设置警报、建立基线，并主动管理容量，以优化推理性能和资源使用效率。
通过新的 CloudWatch 指标提升 Amazon Bedrock 推理工作负载的操作可视化
📄 English Summary
Improve operational visibility for inference workloads on Amazon Bedrock with new CloudWatch metrics for TTFT and Estimated Quota Consumption
AWS has announced two new Amazon CloudWatch metrics for Amazon Bedrock: TimeToFirstToken and EstimatedTPMQuotaUsage. These metrics aim to enhance operational visibility for inference workloads. TimeToFirstToken measures the time taken from request to the generation of the first token, while EstimatedTPMQuotaUsage provides an estimate of the tokens processed per minute. These metrics enable users to set alarms, establish baselines, and proactively manage capacity, optimizing inference performance and resource utilization.