Multimodal interaction strategies for safe robot-assisted object transfer with restricted visibility

MIRobO was a funded project on behalf of the BMBF. We successfully developed a multimodal, accessible and safe robot assistance system for handing over objects with an inherent risk of injury without line of sight.

It was important for us to achieve a high level of trust and acceptance in the interaction with the robot within the visually disabled target group. Therefore, we focused on close interdisciplinary cooperation and the consistent application of a user-centred design and development process from the very beginning. The focus of our sub-project was the development of the language interaction. The process included several iterations of analysis, design & development and evaluation phases:

• Context & requirements analysis with a focus group of blind and visually impaired people.
• Design workshop to create use-cases
• Explorative study in the form of a role play to analyse handover processes
• Construction of a soundboard (prototyping)
• Wizard-of-Oz experiment with test persons from the target group to evaluate the dialogue design
• Online workshops with human factors experts to further develop the dialogue
• Large-scale user study with fully integrated demonstrator (60 test persons, one third each sighted, visually disabled and blind people)

The results show that the voice control successfully enabled all test participants to take over objects with an error rate of only 1.8 %. In particular, the interaction processes were rated well and the concept was successfully tested.