Control Stability

I have developed / codeveloped few RL-based methods that can guarantee control stability in the Lyapunov sense. The policy parameterization is such that stability can be guaranteed even with randomly initialized parameters and random exploration. These methods are demonstrated on a real 7-DOF robot for performing various contact-rich manipulation tasks.

 

i-MOGIC-based policy

In this work, a special policy parameterization with stability property ( i-MOGIC ) is adopted and a novel Evolution Strategy (ES) method, that is inspired from Cross Entropy Method (CEM) , is introduced to perform gradient-free policy search in a model-free RL framework. 

Khader, SA , Yin, H., Falco, P., & Kragic, D. (2020). Stability-guaranteed reinforcement learning for contact-rich manipulation. IEEE Robotics and Automation Letters (RA-L).  [ IEEE ] [ arXiv ]

 

Normalizing-Flow policy

In this work, a stability-aware policy parameterization is obtained by combining a fixed spring-damper regulator and a nonlinear (invertible) transformation parameterized as a deep neural network. By learning the parameters of the transformation network through model-free RL, the optimal transformation that solves the manipulation task is learned. 

Khader, SA *, Yin, H. *, Falco, P., & Kragic, D. (2021). Learning Stable Normalizing-Flow Control for Robotic Manipulation. IEEE International Conference on Robotics and Automation (ICRA)  [ IEEE ] [ arXiv ]

* Equal contribution

 

 

Energy Shaping policy

In this work, a stability-aware policy parameterization is obtained from physics-based prior of Lagrangian mechanics . A deep neural network policy is parameterized in the form of energy shaping control of Lagrangian systems. The energy shaping control form has a natural Lyapunov function with which stability is easily established.

Khader, SA , Yin, H., Falco, P., & Kragic, D. (2021). Learning Deep Energy Shaping Policies for Stability-Guaranteed Manipulation. IEEE Robotics and Automation Letters (RA-L).  [ IEEE ] [ arXiv ]