Robustness to Approximations and Incorrect Models in Stochastic Control

Add to your list(s) Download to your calendar using vCal

• Serdar Yuksel, Queen’s University
• Wednesday 01 May 2024, 14:00-15:00
• Denis Gabor Seminar Room, 611.

If you have a question about this talk, please contact Giordano Scarciotti.

Abstract: In stochastic control, typically, an ideal model is assumed or an estimate model is learned, and the control design is based on this model, raising the robustness problem of performance loss due to the mismatch between the assumed model and the actual system. Even when a correct model is available, computation constraints may dictate the use of approximation methods. In this talk, we will view approximations and robustness under a unified theme. Robustness may be with regard to the system model, driving noise distribution, the initial prior, or approximations as required by computational methods. We will study this problem in both discrete-time and continuous-time and under several criteria and information structures.

We will first study the discrete-time setup and consider robustness to approximations for stochastic control with standard Borel spaces and present conditions under which finite models obtained through quantization of the state and action sets can be used to construct approximately optimal policies. We will then investigate robustness to more general modeling errors and study the mismatch loss of optimal control policies designed for incorrect models applied to the true system, as the incorrect model approaches the true model. We show that continuity and robustness cannot be established under weak and setwise convergences of transition kernels in general, but that the expected induced cost is robust under total variation convergence. By imposing further assumptions (such as continuous convergence of transition kernels or total variation continuity on a measurement channel), we show that the optimal cost can be made continuous and robust under weak convergence of the transition kernels as well. Robustness to incorrect noise models and priors will also be studied for discounted and average cost criteria.

For continuous-time models, we first present existence and discrete-time approximation results for finite horizon and infinite-horizon discounted/ergodic optimal control problems for a general class of non-degenerate controlled diffusion processes. Under a general set of assumptions and a convergence criteria on the models, we establish that the error due to mismatch that occurs by application of a control policy, designed for an incorrectly estimated model, to a true model decreases to zero as the incorrect model approaches the true model. We also discuss robustness to the noise process where we present a robustness theorem for controlled stochastic differential equations driven by approximations of Brownian motion, where the approximations are those that converge to the Brownian under the rough paths topology. Finally, discrete-time approximations under several criteria and information structures will be established via a unified probabilistic approach. (Joint work with Ali D. Kara, Somnath Pradhan and Zachary Selk).

This talk is part of the Control and Power Seminars series.

This talk is included in these lists:

• Andrea Picciau's list
• Control and Power Seminars
• Denis Gabor Seminar Room, 611

Note that ex-directory lists are not shown.