Decide or Delegate: How Script Knowledge Based Conversational Assistants Should Act in Inconclusive Situations

Abstract

We present a study addressing the question how script knowledge based conversational assistants should act in situations of inconclusive information. Such situations occur for example in case of alternative or optional events that lead to multiple correct paths through the script. We have conducted a user study with four typical everyday activities (Making Coffee, Baking Cake, Finding the route to main station, Finding the route to camping ground) that may be represented in scripts. In this study, we have compared and evaluated four different presentation styles to handle situations of conflicting script information. A total of 182 persons participated in our study. The evaluation results show that, in case of a conflicting script state, users find the assistant most useful and are most satisfied if the assistant guesses the next correct event and provides a direct instruction instead of disclosing his incompetence. Alternatives in which the assistant delegates the decision to the user score worse.