Zenonas Theodosiou, Olga Georgiou, N. Tsapatsoulis
{"title":"Evaluating Annotators Consistency with the Aid of an Innovative Database Schema","authors":"Zenonas Theodosiou, Olga Georgiou, N. Tsapatsoulis","doi":"10.1109/SMAP.2011.22","DOIUrl":null,"url":null,"abstract":"Automatic semantic tagging of multimedia is still inefficient due to the difficulties in modelling abstract or complex terms using low level features. The degree of consensus and homogeneity in judgements among annotators is very important in semantic image and video retrieval. In this paper we present a novel method in evaluating the annotators consistency, which uses an innovative database schema and combines two different annotation approaches. A set of 100 images were annotated by 16 annotators using vocabulary keywords and free keywords. The results indicate that combination of annotation methods may lead to increased annotation consistency compared to a single method but this is not a general fact. As expected the use of free keywords and images require tagging that is not directly related to their content, lead to increase the annotators inconsistency.","PeriodicalId":346975,"journal":{"name":"2011 Sixth International Workshop on Semantic Media Adaptation and Personalization","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2011-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"5","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2011 Sixth International Workshop on Semantic Media Adaptation and Personalization","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/SMAP.2011.22","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 5
Abstract
Automatic semantic tagging of multimedia is still inefficient due to the difficulties in modelling abstract or complex terms using low level features. The degree of consensus and homogeneity in judgements among annotators is very important in semantic image and video retrieval. In this paper we present a novel method in evaluating the annotators consistency, which uses an innovative database schema and combines two different annotation approaches. A set of 100 images were annotated by 16 annotators using vocabulary keywords and free keywords. The results indicate that combination of annotation methods may lead to increased annotation consistency compared to a single method but this is not a general fact. As expected the use of free keywords and images require tagging that is not directly related to their content, lead to increase the annotators inconsistency.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
