A sensemaking system for grouping and suggesting stories from multiple affective viewpoints in museums

ABSTRACTThis article presents an affective-based sensemaking system for grouping and suggesting stories created by the users about the cultural artefacts in a museum. By relying on the TCL commonsense reasoning framework, the system exploits the spatial structure of the Plutchik’s “wheel of emotions” to organize the stories according to their extracted emotions. The process of emotion extraction, reasoning, and suggestion is triggered by an app, called GAMGame, and integrated with the sensemaking engine. Following the framework of Citizen Curation, the system allows classifying and suggesting stories encompassing cultural items able to evoke not only the very same emotions of already experienced or preferred museum objects but also novel items sharing different emotional stances and, therefore, able to break the filter bubble effect and open the users’ view toward more inclusive and empathy-based interpretations of cultural content. The system has been designed tested, in the context of the H2020EU SPICE project (Social cohesion, Participation, and Inclusion through Cultural Engagement), in cooperation with the community of the d/Deaf and on the collection of the Gallery of Modern Art (GAM) in Turin. We describe the user-centered design process of the web app and of its components and we report the results concerning the effectiveness of the diversity-seeking, affective-driven, recommendations of stories.KEYWORDS: Story-based recommendationsdiversity-seeking emotional recommendationscommonsense reasoningaffective computingrecommender systems AcknowledgmentsThe research leading this publication has been partially funded by the European Union’s Horizon 2020 research and innovation programme http://dx.doi.org/10.13039/501100007601 under grant agreement SPICE 870811. The publication reflects the author’s views. The Research Executive Agency (REA) is not liable for any use that may be made of the information contained therein. We thank the GAM Museum and the Istituto dei Sordi di Torino for their help in setting up the evaluation.Disclosure statementNo potential conflict of interest was reported by the author(s).Notes1 TCL is an acronym for Typicality-based Compositional Logic: the reasoning framework driving the behavior of the sensemaking system. The framework is described in Section 4.12 https://spice-h2020.eu/3 https://www.gamtorino.it/en4 http://conventions.coe.int/Treaty/EN/Treaties/Html/199.htm5 https://icom.museum/en/resources/standards-guidelines/museum-definition/6 DEGARI is an acronym that stands for Dynamic Emotion Generator and ReclassIfier.7 https://www.who.int/news-room/fact-sheets/detail/disability-and-health8 https://www.who.int/health-topics/disability9 https://access.si.edu/10 https://universaldesign.ie/What-is-Universal-Design/The-7-Principles/11 36 stories were created using Google Forms, but they are not included in the analysis due to the differences with the prototype.12 https://www.gamtorino.it/it/archivio-catalogo/estate-lamaca/13 https://www.gamtorino.it/it/archivio-catalogo/via-a-parigi/14 https://www.gamtorino.it/it/archivio-catalogo/le-tre-finestre-la-pianura-della-torre/15 https://reactjs.org/16 https://spice-h2020.eu/document/deliverable/D1.2.pdf17 The reasons leading to the choice of this model as grounding element of the DEGARI 2.0 system is twofold: on the one hand, this it is well-grounded in psychology and general enough to guarantee a wide coverage of emotions, thus giving the possibility of going beyond the emotional classification and recommendations in terms of the standard basic emotions suggested by models like the Ekman’s one (widely used in computer vision and sentiment analysis tasks). This affective extension is aligned with the literature on the psychology of art suggesting that the encoding of complex emotions, such as Pride and Shame, could give further interesting results in AI emotion-based classification and recommendation systems (Silvia, Citation2009). Second, the Plutchik wheel of emotions is perfectly compliant with the generative model underlying the TCL logic.18 The ontology is available here: https://raw.githubusercontent.com/spice-h2020/SON/main/PlutchikEmotion/ontology.owl and queryable via SPARQL endpoint at: http://130.192.212.225/fuseki/dataset.html?tab=query ds=/ArsEmotica-core19 Such lexicon provides a list of English words, each with real-values representing intensity scores for the eight basic emotions of Plutchik’s theory. The intensity scores were obtained via crowd-sourcing, using best-worst scaling annotation scheme.20 https://www.nltk.org/21 https://www.cis.uni-muenchen.de/schmid/tools/TreeTagger/22 https://www.w3.org/TR/rdf-sparql-query/23 The analysis of the recommendations based on stories represents the major difference with a previous work (Lieto et al., Citation2022) that was, on the other hand, focused only on singe-items diversity-seeking recommendations24 This is one of the most commonly used methodology for the evaluation of recommender systems based on controlled small groups analysis, see (Shani & Gunawardana, Citation2011).25 Thus representing an even more challenging evaluation setup compared to the first evaluation since the users were, arguably, less incline to provide higher ratings for collections that do not elicit their original preferred emotional setting.Additional informationFundingThe work was supported by the Horizon 2020 Framework Programme [870811].Notes on contributorsAntonio LietoAntonio Lieto is an Assistant Professor in Computer Science at University of Turin (Italy) and at the ICAR-CNR (Italy). His main research topics include commonsense reasoning, language and knowledge technologies, cognitive architectures for intelligent interactive agents (embodied and not).Manuel StrianiManuel Striani received his PhD at the University of Torino (Italy) - Science and High Technology (spec. Computer Science), Computer Science Department in February 2019. He is currently a temporary research fellow INF/01 at the Department of Sciences and Technological Innovation (DiSIT) of the University of Eastern Piedmont. His main research interests focus on Artificial Intelligence in healthcare, in particular on process mining, knowledge abstraction, representation, reasoning and formalization through ontologies, language/semantic technologies, multicriteria data structures for compression and optimization algorithms and Machine/Deep learning methodologies on clinical trials.Cristina GenaCristina Gena is an Associate Professor in Computer Science at University of Turin, where she teaches web programming, HCI and HRI. She heads the smart HCI lab of the ICxT Innovation center of the University of Turin. Her main research interests regard Human Computer Interaction, Human Robot Interaction, Intelligent User Interfaces and User Modeling.Enrico DolzaEnrico Dolza is a Professional Educator specialized in pedagogy for people with special needs. He is also the Director of the Instituto dei Sordi (Institue for the Deaf) of Turin and he is (or has been) Adjunct Professor at the University of Turin, University of Milan and University of Bologna teaching the courses of Special Pedagogy and Italian Sign Language (LIS).Anna Maria MarrasAnna Maria Marras is a Librarianship and Archivistics research fellow at the Department of Historical Studies of the University of Turin. Her main research fields concern digital transformation, digitalization, communication and digital accessibility of Heritage and GLAM. She is general secretary of AVICOM – ICOM and she is a councilor of the Europeana Network Association.Gian Luca PozzatoGian Luca Pozzato (1978) obtained his Ph.D. in Computer Science in February 2007 at the University of Turin, Italy. Since November 2015 he is an Associate Professor at the Department of Computer Science of the same University, where he is member of the ”Knowledge representation, Automated Reasoning, Logic and ontologies” group. His research interests include proof theory for nonclassical logics, logic programming, description logics, and nonmonotonic reasoning.Rossana DamianoRossana Damiano is an Associate Professor at the Computer Science Department of the University of Torino, where she teaches Web Programming and Semantic Technologies.Her research interests mainly concern artificial intelligence for cultural heritage, with a focus on affect and storytelling. She has taken part in several applicative projects, ranging from social semantic environments for learning and cultural dissemination, to semantic annotation of drama and artificial characters.