Data Driven Learning and Control seminar series is organized by the Information and Decision Science Lab at Cornell University and aims to explore the latest advancements and interdisciplinary approaches to data-driven learning and control systems.
Watch on YouTube Live
Learning Pipelines Bridging RL and Adaptive Control
The adjacent fields of reinforcement learning (RL) and adaptive control share the same objectives, yet they are separated by a wide cultural gap. In this presentation, I attempt to bridge this gap for the linear quadratic regulator (LQR) problem, which serves as a cornerstone and the benchmark for both fields. I begin by discussing different learning pipelines, including direct and indirect (model-based) approaches, as well as episodic and online (adaptive) approaches. Despite the extensive literature spanning several decades, numerous problems remain unsolved. For instance, RL methods are seldom concerned with closed-loop stability certificates or efficient implementations, while the adaptive control community has dedicated minimal effort to optimality. We address the data-driven LQR problem in an adaptive setting, which entails online recursive algorithms and closed-loop data, and we seek both algorithmic as well as closed-loop certificates. Our approach encompasses different variations of policy gradient methods, employs a novel covariance parameterization, and leverages different kind of regularizations promoting robustness or exploration. Finally, all our theoretical results are validated through simulations and experiments in diverse domains, demonstrating the computational and sample efficiency of our method.
Bio: Florian Dorfler is an associate professor at the Automatic Control Laboratory at ETH Zurich and the associate head of the Department of Information Technology and Electrical Engineering. He received his Ph.D. degree in mechanical engineering from the University of California at Santa Barbara in 2013, and a Diplom degree in engineering cybernetics from the University of Stuttgart in 2008. From 2013 to 2014 he was an assistant professor at the University of California Los Angeles. He is a recipient of the distinguished young research awards by IFAC (Manfred Thoma Medal 2020) and EUCA (European Control Award 2020). His students were winners or finalists for Best Student Paper awards at the European Control Conference (2013, 2019), American Control Conference (2016), Conference on Decision and Control (2020), PES General Meeting (2020), and PES PowerTech Conference (2017). He received the 2010 ACC Student Best Paper Award, the 2011 O. Hugo Schuck Best Paper Award, the 2012-2014 Automatica Best Paper Award, the 2016 IEEE Circuits and Systems Best Paper Award, and the 2015 UCSB ME Best Ph.D. award.