Continuous-Time Infinite-Horizon Linear-Quadratic Regulator

For a linear time-variant continuous-time system $$\dot{x}(t)=\underset{f(x(t),u(t),t)}{\underbrace{A(t)x(t)+B(t)u(t)}}$$ and a quadratic total cost $$J(x(t_0),u(\tau ),t_0)=\underset{\Phi (x_N)}{\underbrace{x(t_f{)}^{\top }{Q}_{f}x(t_f)}}+{\int }_{t_0}^{t_f}\underset{l(x(\tau ),u(\tau ),\tau )}{\underbrace{x(\tau {)}^{\top }Q(t)x(\tau )+u(\tau {)}^{\top }R(t)u(\tau )}}d\tau ,$$ where $Q_f⪰0$, $Q(t)⪰0$, and $R(t)\succ 0$, of its trajectory $(x(\tau ),u(\tau ))$, $\tau \in ⟨t_0,t_f⟩$ the optimal controller can be derived based on the assumption that the value function takes the form $$V(x(t),t)=x(t{)}^{\top }P(t)x(t),P(t)⪰0.$$ When substituted into the Hamilton-Jacobi-Bellman Equation along with the system’s dynamics we attain $$\begin{array}{c}-x(t{)}^{\top }\dot{P}(t)x(t)=\\ \underset{u(t)}{\min }\left\{x(t{)}^{\top }Q(t)x(t)+u(t{)}^{\top }R(t)u(t)+2x(t{)}^{\top }P(t)\left(A(t)x(t)+B(t)u(t)\right)\right\}.\end{array}\text{\hspace{1em}(1)}$$ To find the minimum, we may take the gradient of its argument (which is by design quadratic and convex) with respect to $u(t)$, set it to zero and find the solution (optimal control input) $$u^{\ast }(t)=-R(t{)}^{-1}B(t{)}^{\top }P(t)x(t).$$

The input can then be substituted back into (1). As the equation must hold for all $x(t)$, through basic manipulations we then attain the differential Riccati equation (DRE) $$-\dot{P}(t)=Q(t)-P(t)B(t{)}^{\top }R(t{)}^{-1}B(t)P(t)+P(t)A(t)+A(t{)}^{\top }P(t)$$ for a finite horizon $t_f\in \mathbb{R}$. Supplemented with the boundary conditon $P(t_f)=Q_f$, the DRE forms an initial value problem (IVP) which can be solved using numerical integration. However, solving the problem is not straght-forward. The positive semi-definiteness of $P(t)$ is a property that is not inherently preserved by all integration methods. This necessitates the use of either symplectic integration methods (often tailored for DREs) or factorizing $P(t)$.