Analyzing the expression of annoyance during phone calls to complaint services

Abstract

The identification of emotional hints from speech shows a large number of applications. Machine learning researchers have analyzed sets of acoustic parameters as potential cues for the identification of discrete emotional categories or, alternatively, of the dimensions of emotions. Experiments have been carried out over records including simulated or induced emotions, even if recently more research has been carried out on spontaneous emotions. However, it is well known that emotion expression depends not only on cultural factors but also on the individual and also on the specific situation. In this work we deal with the tracking of annoyance shifts during real phone-calls to complaint services. The audio files analyzed show different ways to express annoyance, as, for example, disappointment, impotence or anger. However variations of parameters derived from intensity combined with some spectral information and suprasegmental features have shown to be very robust for each speaker and annoyance rate. The work also discussed the annotation problem and proposed an extended rating scale in order to include annotators disagreements. Our frame classification results validated the annotation procedure. Experimental results also showed that shifts in customer annoyance rates could be potentially tracked during phone calls.