在线约束马尔可夫决策过程的近最优样本复杂度

Safety is a fundamental challenge in reinforcement learning (RL), particularly in real-world applications such as autonomous driving, robotics, and healthcare. To address this challenge, Constrained Markov Decision Processes (CMDPs) are employed to enforce safety constraints while optimizing performance. However, existing methods often lead to significant safety violations or require high sample complexity to generate near-optimal policies. This research addresses two scenarios: relaxed feasibility, where small violations are permissible, and strict feasibility, where no violations are allowed. A model-based primal-dual algorithm is proposed that balances regret and bounded constraint violations, drawing on techniques from online RL and constrained optimization.