TherapyGym：评估和对齐治疗聊天机器人的临床忠实性与安全性

The increasing use of large language models (LLMs) for mental health support has highlighted the inadequacy of existing evaluation methods, such as fluency metrics, preference tests, and generic dialogue benchmarks, in capturing the clinically critical dimensions of psychotherapy. The proposed THERAPYGYM framework aims to evaluate and enhance therapy chatbots along two clinical pillars: fidelity and safety. Fidelity is assessed using the Cognitive Therapy Rating Scale (CTRS), implemented as an automated pipeline that scores adherence to cognitive behavioral therapy (CBT) techniques across multi-turn sessions. Safety is evaluated through a multi-label annotation scheme that addresses therapy-specific risks, such as the failure to address harm or abuse. Additionally, measures are taken to mitigate bias and unreliability in LLM-based judges.