Sound-Based Storytelling for Vocabulary Learning in Child–Robot Interaction

Description:

Interactive storytelling is one of the oldest teaching methods known to humankind. It not only engages listeners in passive listening but also makes them an active part of the story. Unlike passive listening, participatory storytelling encourages children to actively engage with narrative content, repeat key elements, and connect words with contextual cues [1].

Individualized language support is often difficult to provide due to limited instructional time and large class sizes. Social robots offer a promising medium for structured yet engaging activities that complement classroom teaching [2 – 3] thereby creating opportunities for teachers to provide individual support to students. Their physical presence, predictable interaction patterns, and ability to produce consistent multimodal stimuli make them particularly suitable for language learning tasks[4].

This thesis focuses on the development and evaluation of a robot-supported interactive storytelling system for elementary school children using the Navel robot, which is currently deployed in two schools in Heilbronn. The system will narrate short stories and dynamically integrate contextually appropriate sound effects. For example, when describing environmental events (“the wind blew”), the robot will generate a corresponding blowing sound and invite children to imitate it. The design of the system is informed by preliminary feedback from teachers at the schools participating in the RobotErzählen project, who emphasized the importance of interactive and engaging formats for vocabulary learning. In particular, they highlighted the potential of multimodality to maintain attention during storytelling activities.

The empirical evaluation will investigate the effects of sound-augmented, participatory storytelling on children’s comprehension and vocabulary learning. In a controlled laboratory setting or directly at school, participants will listen to short interactive stories presented either with robot-generated contextual sounds or without additional auditory enhancement. Comprehension and vocabulary acquisition will be assessed through structured tasks following the storytelling session.

The overall goal of the thesis is to assess whether sound-augmented robot storytelling can serve as a meaningful tool for contextual vocabulary learning and to determine the importance of robot-generated auditory cues in child-directed interaction.

.

Objectives:

  • Develop a storytelling game on the Navel robot integrating speech and contextual sound effects

  • Design interaction patterns that encourage children to mimic sounds and participate actively

  • Conduct a user study in an controlled lab setting

  • Analyze interaction flow, misunderstandings, and breakdowns

  • Provide design recommendations for sound-based educational robot systems

Requirements:
  • Interest in children's interaction with robots or learning technologies
  • Proficiency with Python
  • Motivation to work with the Navel robot
  • Motivation to conduct a user study
  • Interest in experimental design and evaluation
Thesis Type:

M.Sc.

Starting Date:

Flexible. Contact the supervisor of this thesis if you are interested.

Supervisor:

Katharina Biernacka

References:

[1] Alsumait, Asmaa, and Zahraa S. Al‐Musawi. "Creative and innovative e‐learning using interactive storytelling." International Journal of Pervasive Computing and Communications 9.3 (2013): 209-226.

[2] Woo, Hansol, et al. "The use of social robots in classrooms: A review of field-based studies." Educational Research Review 33 (2021): 100388.

[3] Leite, Iolanda, et al. "Emotional storytelling in the classroom: Individual versus group interaction between children and robots." Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction. 2015.

[4] Huang, Guanyu, and Roger K. Moore. "Using social robots for language learning: are we there yet?." Journal of China Computer-Assisted Language Learning 3.1 (2023): 208-230.