Evaluating the Usability of an LLM-Aided Hybrid Avatar Agent System

Abstract

The demand for conversational systems using digital humans has increased across various fields, including education and psychological counseling. Conversational digital humans can be categorized into avatar and agent. This paper introduces the Hybrid Avatar Agent System (HAAS), designed to implement a conversational digital human capable of addressing diverse questions. To facilitate seamless transitions between avatars and agents, we developed a Korean Dialogue Breakdown Detection (DBD) model. While a digital human typically functions as an agent, the DBD model identifies conversational pauses, prompting human operator intervention for user assistance. We trained DBD model using the RoBERTa_base model, achieving an accuracy of 60.77%. In a user study, a comparison was drawn between HAAS and the Agent Only System (AOS). Users noted that HAAS provided more appropriate answers than AOS.