Traditional recommender systems present a relatively static list of recommendations to a user where the feedback is typically limited to an accept/reject or a rating model. However, these simple modes of feedback may only provide limited insights as to why a user likes or dislikes an item and what aspects of the item the user has considered. Interactive recommender systems present an opportunity to engage the user in the process by allowing them to interact with the recommendations, provide feedback and impact the results in real-time. Evaluation of the impact of the user interaction typically requires an extensive user study which is time consuming and gives researchers limited opportunities to tune their solutions without having to conduct multiple rounds of user feedback. Additionally, user experience and design aspects can have a significant impact on the user feedback which may result in not necessarily assessing the quality of some of the underlying algorithmic decisions in the overall solution. As a result, we present an evaluation framework which aims to simulate the users interacting with the recommender. We formulate metrics to evaluate the quality of the interactive recommenders which are outputted by the framework once simulation is completed. While simulation alone is not sufficient to evaluate a complete solution, the results can be useful to help researchers tune their solution before moving to the user study stage.
{"title":"An Evaluation Framework for Interactive Recommender Systems","authors":"O. Alkan, E. Daly, A. Botea","doi":"10.1145/3314183.3323680","DOIUrl":"https://doi.org/10.1145/3314183.3323680","url":null,"abstract":"Traditional recommender systems present a relatively static list of recommendations to a user where the feedback is typically limited to an accept/reject or a rating model. However, these simple modes of feedback may only provide limited insights as to why a user likes or dislikes an item and what aspects of the item the user has considered. Interactive recommender systems present an opportunity to engage the user in the process by allowing them to interact with the recommendations, provide feedback and impact the results in real-time. Evaluation of the impact of the user interaction typically requires an extensive user study which is time consuming and gives researchers limited opportunities to tune their solutions without having to conduct multiple rounds of user feedback. Additionally, user experience and design aspects can have a significant impact on the user feedback which may result in not necessarily assessing the quality of some of the underlying algorithmic decisions in the overall solution. As a result, we present an evaluation framework which aims to simulate the users interacting with the recommender. We formulate metrics to evaluate the quality of the interactive recommenders which are outputted by the framework once simulation is completed. While simulation alone is not sufficient to evaluate a complete solution, the results can be useful to help researchers tune their solution before moving to the user study stage.","PeriodicalId":240482,"journal":{"name":"Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124267039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Musto, A. Rapp, F. Cena, F. Hopfgartner, J. Kay, A. Lawlor, P. Lops, G. Semeraro, N. Tintarev
It is our great pleasure to welcome you to the UMAP 2019 Workshop on Explainable and Holisitic User Modeling (ExHUM). Our workshop took inspiration from the analysis of the recent Web dynamics: according to a recent claim by IBM, 90% of the data available today have been created in the last two years. Such an exponential growth of personal information has given new life to research in the area of user modelling, since information about users' preferences, sentiment and opinions, as well as signals describing their physical and psychological state, can now be obtained by mining data gathered from many heterogeneous sources. How can we use such data to drive personalization and adaptation mechanisms? How can we effectively merge such data to obtain a holistic representation of all (or some of) the facets describing people? Moreover, as the importance of such technologies in our everyday lives grows, it is also fundamental that the internal mechanisms that guide personalization algorithms are as clear as possible. It is not by chance that the recent General Data Protection Regulation (GDPR) emphasized the users' right to explanation when people face machine learning-based systems. Unfortunately, the current research tends to go in the opposite direction, since most of the approaches try to maximize the effectiveness of the personalization strategy (e.g., recommendation accuracy) at the expense of the explainability and the transparency of the model.
{"title":"UMAP 2019 Workshop on Explainable and Holistic User Modeling (ExHUM) Chairs' Welcome & Organization","authors":"C. Musto, A. Rapp, F. Cena, F. Hopfgartner, J. Kay, A. Lawlor, P. Lops, G. Semeraro, N. Tintarev","doi":"10.1145/3314183.3323712","DOIUrl":"https://doi.org/10.1145/3314183.3323712","url":null,"abstract":"It is our great pleasure to welcome you to the UMAP 2019 Workshop on Explainable and Holisitic User Modeling (ExHUM). Our workshop took inspiration from the analysis of the recent Web dynamics: according to a recent claim by IBM, 90% of the data available today have been created in the last two years. Such an exponential growth of personal information has given new life to research in the area of user modelling, since information about users' preferences, sentiment and opinions, as well as signals describing their physical and psychological state, can now be obtained by mining data gathered from many heterogeneous sources. How can we use such data to drive personalization and adaptation mechanisms? How can we effectively merge such data to obtain a holistic representation of all (or some of) the facets describing people? Moreover, as the importance of such technologies in our everyday lives grows, it is also fundamental that the internal mechanisms that guide personalization algorithms are as clear as possible. It is not by chance that the recent General Data Protection Regulation (GDPR) emphasized the users' right to explanation when people face machine learning-based systems. Unfortunately, the current research tends to go in the opposite direction, since most of the approaches try to maximize the effectiveness of the personalization strategy (e.g., recommendation accuracy) at the expense of the explainability and the transparency of the model.","PeriodicalId":240482,"journal":{"name":"Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization","volume":"156 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124063415","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization","authors":"","doi":"10.1145/3314183","DOIUrl":"https://doi.org/10.1145/3314183","url":null,"abstract":"","PeriodicalId":240482,"journal":{"name":"Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization","volume":"69 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128678131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：