{"title":"Online metrics for web search relevance","authors":"Jan O. Pedersen","doi":"10.1145/2513150.2513165","DOIUrl":null,"url":null,"abstract":"Information Retrieval has a long tradition of being metrics driven. Ranking algorithms are assessed with respect to some utility measure that reflects the likelihood of satisfying an information need. Traditionally these metrics are based on offline judgments. This is very flexible since judgments can be made for any desired output. However, judgments are no better than judgment guidelines and are at some distance from the actual user experience. Modern Web Search engines enjoy an additional resource; existing web search traffic and its attendant wealth of user engagement data. Primarily this signal consists of logged queries and user actions, including clicks and reformulations. I will discuss how this data can be used to derive Web Search quality metrics that have very different properties than traditional offline metrics.","PeriodicalId":436800,"journal":{"name":"LivingLab '13","volume":"40 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"LivingLab '13","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/2513150.2513165","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1
Abstract
Information Retrieval has a long tradition of being metrics driven. Ranking algorithms are assessed with respect to some utility measure that reflects the likelihood of satisfying an information need. Traditionally these metrics are based on offline judgments. This is very flexible since judgments can be made for any desired output. However, judgments are no better than judgment guidelines and are at some distance from the actual user experience. Modern Web Search engines enjoy an additional resource; existing web search traffic and its attendant wealth of user engagement data. Primarily this signal consists of logged queries and user actions, including clicks and reformulations. I will discuss how this data can be used to derive Web Search quality metrics that have very different properties than traditional offline metrics.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
