---
title: "Identifying Differences in Social Responsiveness among Preschoolers Interacting with or Watching Social Robots"
authors:
  - "Marie D. Manner"
doi: "10.24963/ijcai.2018/832"
url: "https://ijcai.org/proceedings/2018/832"
pdf: "https://ijcai.org/proceedings/2018/0832.pdf"
venue: "Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence"
year: 2018
pages: "5777-5778"
track: "Doctoral Consortium"
keywords:
  - "Multidisciplinary Topics and Applications: Human-Computer Interaction"
---

# Identifying Differences in Social Responsiveness among Preschoolers Interacting with or Watching Social Robots

Marie D. Manner

*Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence*, pages 5777-5778. DOI: [10.24963/ijcai.2018/832](https://doi.org/10.24963/ijcai.2018/832)

## Abstract

We describe experiments performed with a large number of preschool children (ages 1.5 to 4 years) in a two-task eye tracking experiment and a human-robot interaction experiment. The resulting data of mostly neuro-typical children forms a baseline with which to compare children with autism, allowing us to further characterize the autism phenotype. Eye tracking task results indicate a strong preference for a humanoid robot and a social being (a four year old girl) over other robot types. Results from the human-robot interaction task, a semi-structured play interaction between child and robot, showed we can cluster participants based on social distances and other social responsiveness metrics.

## Full text

## 1 Introduction

Autism spectrum disorder (ASD) is a highly variable, developmental disorder characterized by a variety of symptoms, such as lack of eye contact, lack of physical contact, restricted interests (e.g. intense interest in train schedules), and repetitive movements (e.g. hand flapping) [Zwaigenbaum et al., 2005]. These behaviors are thus considered early markers for autism. Research has also shown that children with autism have a strong interest in technology such as autonomous robots. Given that the autism phenotype, or observable characteristics, varies from child to child, this work seeks to combine the interests in technology and robots, social behaviors of robots, and automated social distance detection to add more nuanced characterization to the autism phenotype.

Socially assistive robotics research in autism is over a decade old, yet does not meet standards of psychology and child development researchers [Diehl et al., 2012; Scassellati et al., 2012; Pennisi et al., 2016]. The criticisms include lack of robot integration to established treatments, lack of participant follow-up, small sample sizes, little scrutiny on therapeutic protocol, and limited characterization of participants. Robotics research with children with autism stems from the fact that afflicted children tend to enjoy autonomous (or seemingly autonomous) robots, and researchers have used a wide variety of robots [Scassellati et al., 2012]. While the reason for this high interest is unknown, this provides the potential to leverage robotics for autism diagnosis or treatment.

This work describes two experiments. The first uses two eye tracking tasks to determine which of several robot types, or a human, children find more interesting. The second experiment explores a human-robot interaction between a young child and a humanoid robot, the one preferred in the first task. This helps describe and quantify how children interact with and socially distance themselves from a social robot.

This work contributes gaze preference data for eye tracking, and clinically significant, consistent clustering of participants when grouped by high-risk autism behaviors and social distances when interacting with a social robot. We also contribute off-line automated software to determine social distances between people (and robots).

## 2 Eye Tracking Experiments

We created two eye tracking experiments, a dynamic video comparison and a static image search, to determine gaze preferences over a set of robots and a social being. Of the 50 children recruited, 2 could not finish the eye tracking study, resulting in a final analysis of 48 children (27 males, 21 females, 18 - 45 months, µ = 29.2 m, σ = 8.8 m).

The first task, dynamic video comparison, shows participants two videos on-screen at once. The five choices, which we call 'animates,' are the NAO (humanoid robot), AIBO (robot dog), Pleo (robot dinosaur), Sphero (solid sphere), and a social being (four year old human girl). We pair five-second videos of moving animates on-screen at the same time; these pairs are also reversed on-screen to guard against any bias in screen location. We create two such video sets, for 40 total video pairs of the 10 possible animate combinations. The task purpose is to see which animate is of more interest in paired comparisons. Results show that in videos pairing NAO and Social child, participants prefer (gaze longer at) NAO; in videos pairing NAO and any other robot, participants prefer NAO; and in videos pairing Social with any non-NAO robot, participants prefer Social. Results are shown in Fig. 1.

The second task, static image search, used the same robots but displayed static pictures of all five animates at once. Participants had three seconds on each of five images, with the order of the animates on-screen randomized in each image. Results showed that children gazed first at the social being, then the humanoid robot, with a preference for both over any of the other robot types.

Figure 1: Preference for robot or social animates, as a percent of ratio spent on animate during videos.

![Image](https://ijcai.org/proceedings/2018/fulltext/832/image_000000_6d2850c2af28c185e7a4b41b4b4f0c6c63d1a7cc7d07b1663dc56e6dcc17e0c5.png)

## 3 Human-Robot Interaction Experiments

The robot interaction study with toddlers is made to further characterize the autism phenotype. If we can better determine subsets of autism phenotypes, we may be able to identify children at high risk for ASD when they are very young. We recruited 65 participants, at low to medium risk (mostly low risk) for ASD, of which 59 (31 males, 28 females, 25 - 45 months, µ = 32.9 months, σ = 4.6 months) completed the interaction study. The interactions range from 9 to 15 minutes, depending on the child's willingness or ability to continue interacting with the robot.

We collected multiple data sets from each participant, including standardized and novel assessments and video footage of the interaction experiment from several perspectives. The robot interaction is a series of seven games: two looking games, two imitation games, and three dances. We generate a novel metric using social distances from the overhead video footage of the robot interaction. Social distances have been shown to differ between typically developed people and people with autism, so we are interested in the distance between the child and robot, the child and experimenter, and the child and their caregiver. We developed software to find the social distances over time through the video.

To measure social distances, we needed a new social distance metric that normalized the physical distance. We must be able to compare a child sitting on their parent and two feet away from the robot, for example, to a child two feet away from the robot and ten feet away from their parent. This Distance Ratio metric, introduced in [Manner et al., 2018], is (Distance to NAO) / (Distance to NAO + Distance to Caregiver), and it converts physical distance between child and NAO and child and caregiver into a continuous value from 1 (closest to caregiver), to 0 (closest to robot). This distance ratio shows clear interaction patterns. Fig. 2 shows a subset of participants; some children stay close to the robot, some stay close to the caregiver, and some move from caregiver to robot throughout the interaction.

We then clustered participants based on this proxemics metric and two dimensions of autistic features, one that quantifies ritualized/routinized behaviors and one that quantifies socialization. We then validated the clinical utility of this grouping strategy with two other dimensions of autism, one that quantifies restricted patterns of play and one that quantifies communication abilities. The results show that our new social distance measure, captured in this unique context, can be used with existing methods to cluster participants into clinically meaningful subgroups.

Figure 2: Average social distance metric during the 7-game interaction (games are 'presses' for social response) for some participants.

![Image](https://ijcai.org/proceedings/2018/fulltext/832/image_000001_8da93534d0e5cec3212930216af62956f77a071d0d3a5aa06c854d9ac4c185f5.png)

## 4 FutureWork

This paper summarizes a two-prong effort to characterize children's reactions to a social robot: one via eye tracking experiments and one with a human robot interaction experiment. Eye tracking results indicate children prefer the NAO robot and a fellow social being, a human child, over some of the other types of robots that have been used in autism research. Future work includes exploring the interaction of age and sex on gaze preferences. Robot interaction study results indicate children can be reliably clustered by assessment scores combined with our new social distance score. Future work on this experiment includes improving our motion tracker, testing the improved motion tracker with unrelated social distance studies, and further validation of cluster numbers and clinical differences between clusters.

## Acknowledgments

Partial support provided by the National Science Foundation [NSF IIP-1439728] and the MnDRIVE RSAM initiative at the University of Minnesota.

## References

1. [Diehl et al., 2012] Joshua J Diehl, Lauren M Schmitt, Michael Villano, and Charles R Crowell. The clinical use of robots for individuals with autism spectrum disorder: a critical review. Res. Autism Spectr. Disord., 6(1):249-262, 2012.
2. [Manner et al., 2018] Marie D Manner, Maria Gini, and Jed Elison. Graphically representing child-robot interaction proxemics. In Workshop on Health Intelligence at AAAI, 2018.
3. [Pennisi et al., 2016] Paola Pennisi, Alessandro Tonacci, Gennaro Tartarisco, Lucia Billeci, Liliana Ruta, Sebas- tiano Gangemi, and Giovanni Pioggia. Autism and social robotics: A systematic review. Autism Res., 9(2):165-183, 2016.
4. [Scassellati et al., 2012] Brian Scassellati, Maja J Matarić, and Henny Admoni. Robots for use in autism research. Annu. Rev. Biomed. Eng., 14(1):275-294, 2012.
5. [Zwaigenbaum et al., 2005] Lonnie Zwaigenbaum, Susan Bryson, Tracey Rogers, Wendy Roberts, Jessica Brian, and Peter Szatmari. Behavioral manifestations of autism in the first year of life. Int. J. Dev. Neurosci., 23(2-3 SPEC. ISS.):143-152, 2005.