Skill Reconstruction as Induction of LQ Controllers with Subgoals

Abstract

Cont rolling a comp lex dyn am ic system, such as a plane or a crane, usually requir es a skilled operator . Such a control ski ll is typically hard to reconstruct through intr ospection. Therefore an attractive approach to the reconstruction of control ski ll involves ma chine learning from operators ' control traces, also kn ow n as behavioural cloning . In the most co mmon approach to behavioural cloning, a contro ll er is induced in the form of a rule set or a decision or regression tree that maps system states to actions. Unfortunately , induced co ntr o ll ers usually suffer from la ck of robustness and lack typical elemen ts of human control strategies, such as subgoals and substages of the control plan. In this paper we pr esent a new approach to behavioural cloning which involves the induction of a mode l of the controlled system and enab les the identification of subgoals that the operator is pursuing at various stages of the execution trace. The und erlying formal basis for the present approach to behavioural cloning is the theory of LQ controllers. Experimenta l results show that this ap proach greatly improves the robustness of the induced controllers and also offers a new way of understanding the operato r 's subcognitive skill.