Crossmodal combination among verbal, facial, and flexion expression for anthropomorphic acceptability

This paper proposes an effective communication with an agent's appearance and reality of the acceptance level of the user's order using verbal and nonverbal expressions. The crossmodal combination is expected to enable delicate expressions of the agent's internal state especially when the agent's decision of the acceptance is different from the agent's mind, so the user becomes aware of the difficult situations. We have proposed to adopt the facial expression and the flexion of the word ending as the parameters of the agent's internal state. The expressions are attached to the scripts of each acceptance level. The results of the expressions showed the following: 1) the effectiveness of both the facial and flexion expressions, and 2) the crossmodal combinations that generate the agent's concealed but real feeling.