Accelerating Optimization and Reinforcement Learning with Quasi-Stochastic Approximation

The ODE method has been a workhorse for algorithm design and analysis since the introduction of the stochastic approximation. It is now understood that convergence theory amounts to establishing robustness of Euler approximations for ODEs, while theory of rates of convergence requires finer analysis. This talk concerns a parallel theory for quasi-stochastic approximation, based on algorithms in which the “noise” is based on deterministic signals.  Part of the motivation is pedagogical:  theory for convergence and convergence rates is greatly simplified.  The other major motivation is practical:   the speed of convergence is remarkably fast when compared to its stochastic counterparts.  The talk will survey recent theory and applications to gradient free optimization and reinforcement learning.

Based on Chapter 4 of the new monograph,  Control Systems and Reinforcement Learning, Cambridge University Press (to appear).

Sean Meyn

University of Florida on October 22, 2021 at 10:15 AM in Zoom
Join Zoom Webinar

Sean Meyn received the B.A. degree in mathematics from UCLA in 1982 and the Ph.D. degree in electrical engineering from McGill University in 1987 (with Prof. Caines). He is now Professor and Robert C. Pittman Eminent Scholar Chair in the Department of Electrical and Computer Engineering at the University of Florida, and the director of the Laboratory for Cognition & Control. His academic research interests include theory and applications of decision and control, stochastic processes, and optimization. He has held visiting positions at universities all over the world, including at IISc, Bangalore during 1997-1998 where he was a Fulbright Research Scholar. He is a fellow of the IEEE, Inria International Chair, and IEEE CSS Distinguished lecturer.
His award-winning 1993 monograph with Richard Tweedie, Markov Chains and Stochastic Stability, has been cited thousands of times in journals from a range of fields. The latest version is published in the Cambridge Mathematical Library.
For the past ten years his applied research has focused on engineering, markets, and policy in energy systems.

This lecture series features exciting and dynamic visiting and virtual speakers from across the range of ECE disciplines. Take some time every Friday morning to be inspired by these great scientists and engineers before heading into the weekend!