화학공학소재연구정보센터
IEEE Transactions on Automatic Control, Vol.65, No.8, 3600-3607, 2020
Posterior Sampling-Based Reinforcement Learning for Control of Unknown Linear Systems
We propose a posterior sampling-based learning algorithm for the linear quadratic (LQ) control problem with unknown system parameters. The algorithm is called posterior sampling-based reinforcement learning for LQ regulator (PSRL-LQ) where two stopping criteria determine the lengths of the dynamic episodes in posterior sampling. The first stopping criterion controls the growth rate of episode length. The second stopping criterion is triggered when the determinant of the sample covariance matrix is less than half of the previous value. We show under some conditions on the prior distribution that the expected (Bayesian) regret of PSRL-LQ accumulated up to time T is bounded by (O) over tilde (root T). Here, (O) over tilde (.) hides constants and logarithmic factors. Numerical simulations are provided to illustrate the performance of PSRL-LQ.