Thomas Berrueta (California Institute of Technology)- Robot learning on the edge: Online learning in hardware

Abstract: Deploying learning algorithms on safety-critical hardware, such as autonomous race cars, remains a significant challenge. While sim-to-real transfer is a popular paradigm, persistent model mismatch and unmodeled dynamics can lead to suboptimal or unsafe behavior on high-performance hardware. This talk argues for an alternative approach: learning directly on the physical system. However, online learning in hardware introduces critical challenges—the need for sample efficiency, operational safety requirements during deployment, and the difficulties of deploying new models in real-time. First, I address the problem of sample efficiency by examining how temporal correlations can hinder online reinforcement learning. Second, I address the challenge of integrating learning modules within sensitive control stacks by designing contractive interfaces between modules. Lastly, I present a framework for safety-critical envelope expansion that allows continual deployment of learned models in real-time. I demonstrate these approaches on diverse hardware platforms, including an autonomous IndyCar capable of reaching speeds exceeding 170mph.