Reinforcement Q-Learning for PDF Tracking Control of Stochastic Systems with Unknown Dynamics

Yang, Weiqing; Zhou, Yuyang; Zhang, Yong; Ren, Yan

Abstract

Tracking control of the output probability density function presents significant challenges, particularly when dealing with unknown system models and multiplicative noise disturbances. To address these challenges, this paper introduces a novel tracking control algorithm based on reinforce-ment Q-learning. Initially, a B-spline model is employed to represent the original system, thereby transforming the control problem into a state weight tracking issue within the B-spline stochastic system model. Moreover, to tackle the challenge of unknown stochastic system dynamics and the presence of multiplicative noise, a model-free reinforcement Q-learning algorithm is employed to solve the control problem. Finally, the proposed algorithm’s effectiveness is validated through comprehensive simulation examples.

Citation

Yang, W., Zhou, Y., Zhang, Y., & Ren, Y. (2024). Reinforcement Q-Learning for PDF Tracking Control of Stochastic Systems with Unknown Dynamics. Mathematics, 12(16), Article 2499. https://doi.org/10.3390/math12162499

• Journal Article Type: Article
• Acceptance Date: Aug 9, 2024
• Online Publication Date: Aug 13, 2024
• Publication Date: 2024
• Deposit Date: Sep 9, 2024
• Publicly Available Date: Sep 9, 2024
• Journal: Mathematics
• Electronic ISSN: 2227-7390
• Publisher: MDPI
• Peer Reviewed: Peer Reviewed
• Volume: 12
• Issue: 16
• Article Number: 2499
• DOI: https://doi.org/10.3390/math12162499
• Keywords: model-free, 93E35, reinforcement learing, tracking control, Q-learning, B-spline model, probability density function