Transparent Intent for Explainable Shared Control in Assistive Robotics

1 Introduction

Equipping robots with the capability to infer human intent from observed behaviour holds immense potential for assistive robotics [Demiris, 2007]. For instance, in shared control tasks where a human and robot collaborate towards a common goal, the synergy will be more effective if the robot's actions are in agreement with their partner's intentions. Nevertheless, the quality of assistance administered through shared control also depends on whether the rationale behind robot behaviour is clearly explicated [Abbink et al., 2018]. Whenever robot actions do not align with a person's mental models of expected system behaviour, there is risk of causing obstruction and frustration rather than support. A means of reconciling mismatched agent models is therefore required.

Within explainable AI and specifically planning, the problem of "model reconciliation" is regarded as a process of explanation [Fox et al., 2017; Chakraborti et al., 2017]. Explanations here refer to model updates that help resolve any differences between a human's expectations of robot plans and their actual representations. One way of generating these explanations is to expose internal representations to human observers, for which augmented reality (AR) is an increasingly prevalent choice of medium [Walker et al., 2018].

Whilst explainable AI is a widely explored area of interest, it has only recently garnered similar attention in shared control [Zolotas et al., 2018; Zolotas and Demiris, 2019]. Nevertheless, bringing explainability to shared control poses new challenges, namely the dependence on a continuous communication channel, the requirement for implicit signals of intent, and the overall lack of consensus on guidelines for shared control [Abbink et al., 2018]. We introduced the Explainable Shared Control (XSC) paradigm to address these challenges and resolve model misalignment by using AR visualisations as explanations [Zolotas and Demiris, 2019].

The key perspective behind XSC is transparency, where the objective is to construct a communication bridge of intent between human and robot (see Fig. 1). This paper elaborates on XSC by outlining our contributions to "transparent intent" and providing outlooks on how the continuous interplay of transmitting intentions could allow online model updates.

2 Transparent Intent

There are two perspectives of another's intent in shared control: the human's and the robot's.

2.1 Human Perspective of the Interaction

Numerous modes of sensory feedback can be employed to relay information back to users. An increasingly popular interface for conveying robot intent during human-robot interaction is AR [Walker et al., 2018]. In particular, we advocate for AR headsets, as their heightened sense of presence better combats model mismatch.

For example, Fig. 2 illustrates how a user's immersive view of a dual-arm collaborative robot might be captured in AR, where affordable robot behaviours are projected as indicators of intent. To create this user perspective, we broadcasted realtime data originating from the robot's perceptive and motion planning capabilities into a concurrent AR application. This data included detected object poses in the shared workspace, as well as the robot arm joint poses corresponding to different trajectories for manipulating objects. When users can simply select collaborative actions via such an AR interface and do not need to physically interact or share control with the robot, then we find this visual overlay approach to be sufficient.

However, visualising a robot's inner workings in shared control requires more careful consideration due to the active user engagement. Our findings from a user study involving an AR headset integrated onto an assistive robotic wheelchair exemplified how poorly designed visualisations can harm task performance and even cause greater misunderstanding [Zolotas et al., 2018]. In turn, we introduced a set of interface guidelines under XSC on how to best demystify shared control [Zolotas and Demiris, 2019]. By deploying contextual and predictive AR visual aids to augment the environment, users in our follow-up study exhibited quicker recovery times from adverse events that are typically encountered during model misalignment in wheelchair navigation.

2.2 Robot Perspective of the Interaction

From the robot's perspective, estimating human intent essentially revolves around deriving a model to match observed behaviour [Demiris, 2007]. There is a vast range of literature supporting this endeavour, with deep learning algorithms becoming commonplace due to their capacity to model highdimensional, non-linear human behaviour conditioned on the environment state perceived by the robot. Although powerful, these "black-box" algorithms are notoriously difficult to interpret and hence explain to end-users [Fox et al., 2017].

As with the AR interface design, we produced a set of guidelines on how to recognise human intent in a suitable manner for user interpretation [Zolotas and Demiris, 2019]. In this work, we determined that the differential-drive dynamics of our robotic wheelchair could be used in conjunction with the user's joystick commands to yield a trajectory of navigational intent that could be easily depicted in AR. The predicted trajectories benefitted transparency by acting as indicators of human intent for the robot, as well as by granting users with a visual glimpse at the robot's internal models.

3 Ongoing Research

There are multiple research avenues to pursue within XSC before our proposed notion of transparent intent can bolster the widespread use of assistive robots. To expand the scope of this paradigm, we have made progress in an unsupervised clustering approach for learning human intentions in tasks without prior knowledge of the world dynamics or user goals.

Moreover, the evaluation of XSC demands that model mismatch is traced over a continuous interaction instead of solely on a session basis. As a result, we plan to further investigate how the frequency of mutual model updates impacts alignment whilst the transparent interaction is unfolding.

Acknowledgements

This research was supported by an EPSRC Doctoral Training Award to Mark Zolotas and a Royal Academy of Engineering Chair in Emerging Technologies to Yiannis Demiris.