{"title":"Annotation and Evaluation of Utterance Intention Tag for Interview Dialogue Corpus","authors":"M. Sasayama, Kazuyuki Matsumoto","doi":"10.1145/3508230.3508236","DOIUrl":null,"url":null,"abstract":"In this paper, we proposed the utterance intention tags for an interview dialogue corpus. We constructed an interview dialogue corpus with the tags we designed. Three or five annotators annotated the tags to the interview dialogue corpus (the total number of 49999 utterances) consisting of 30 dialogues. We conducted an evaluation experiment using Fleiss's kappa value to evaluate the reliability of the proposed tags. When three annotators annotated 18 different tags to the corpus, we obtained the kappa value of 0.55.","PeriodicalId":252146,"journal":{"name":"Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval","volume":"300 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3508230.3508236","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1
Abstract
In this paper, we proposed the utterance intention tags for an interview dialogue corpus. We constructed an interview dialogue corpus with the tags we designed. Three or five annotators annotated the tags to the interview dialogue corpus (the total number of 49999 utterances) consisting of 30 dialogues. We conducted an evaluation experiment using Fleiss's kappa value to evaluate the reliability of the proposed tags. When three annotators annotated 18 different tags to the corpus, we obtained the kappa value of 0.55.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
