Transformed Path Integral Based Approaches for Stochastic Dynamical Systems: Prediction, Filtering, and Optimal Control

Abstract

The study of stochastic systems, their characterization, prediction, and control are of great importance to many fields in science and engineering. This often involves obtaining accurate estimates of quantities of interest such as the system state distribution and/or the expected cost in nonlinear dynamical systems subjected to random forces. The computational prediction and control of such systems are often challenging (and involve large computational costs) due to the presence of nonlinearities, model and measurement uncertainties. Novel path integral–based frameworks for efficient solutions to problems in prediction, nonlinear filtering, and optimal control of stochastic dynamical systems are presented to address several key challenges. The presented frameworks are as follows: (1) the transformed path integral (TPI) approach for solution of the Fokker-Planck equation in stochastic dynamical systems with a full rank diffusion coefficient matrix, (2) the generalized transformed path integral (GTPI) approach—a non-trivial extension of the TPI to stochastic dynamical systems with rank deficient diffusion coefficient matrices, (3) the generalized transformed path integral filter (GTPIF) for solution of nonlinear filtering problems, and (4) the generalized transformed path integral control (GTPIC) for solution of a large class of stochastic optimal control problems are presented. The proposed frameworks are based on the underlying short-time propagators and dynamic transformations of the state variables that ensure the appropriate distributions in the transformed space (state distributions in TPI and GTPI; and corresponding conditional distributions in GTPIF and GTPIC) always have zero mean and identity covariance. In systems where the dynamics are linear with respect to the state variables and initial distribution is Gaussian, the appropriate distributions in the transformed space remain invariant with a standard normal distribution as expected. The frameworks thus allow for the underlying distributions necessary for evaluating the quantities of interest to be accurately represented and evolved in a transformed computational domain. Compared to conventional fixed grid approaches and Monte-Carlo simulations, the challenges in dynamical systems with large drift, diffusion, and concentration of PDF can be addressed more efficiently using the proposed frameworks. In addition, straightforward error bounds for the underlying distributions in the transformed space can be established via Chebyshev's inequality.