ACM Journal on Computing and Cultural Heritage最新文献

This article presents a global framework dedicated to the structuring of iconographic heritage collections. To alleviate the poor interlinking both between collections and contents, a first step of automatic linking exploiting content-based image retrieval approaches is evaluated and adapted to the visual variability of such heritage contents. To ensure understanding and analysis of the contents in a structured fashion, a 3D immersive web platform is also introduced alongside visual-based analysis tools. Finally, by exploiting both automatic linking and manual interventions in the visualization platform, an iterative, semi-automatic structuring pipeline is proposed to solve difficult cases missed by automatic structuring, and then improve structuring optimally. Here, we demonstrate the potential of the proposal on the geographic iconographic heritage of Paris, with a dataset of 10k images belonging to several institutions, thus poorly connected nor organized globally.

Digitising The Deep Past (DDP) is an interdisciplinary project based at Griffith University, Australia, that innovates in three areas: Indigenous cultural heritage, Indigenous education, and Machine Learning (ML) and Artificial Intelligence (AI). The project investigates the use of a purpose-built citizen science application that engages Indigenous youth in educational exercises rooted in local cultural heritage, specifically rock art, making learning more engaging and exposing them to digital technologies. Furthermore, ML models trained with the data gathered through these educational activities can then assist with classifying new rock art images and assisting rangers and archaeologists with site archiving and conservation efforts. This paper discusses the project's significance in enhancing Indigenous science and technology education and outlines its results in utilising ML for rock art classification. Adopting deep learning in rock art classification offers a compelling avenue for the automated analysis and interpretation of heritage objects and places. However, training deep neural networks from scratch often requires enormous datasets and computational resources, posing challenges for domain-specific applications with smaller datasets. With a dataset comprising approximately 3,100 labelled rock art images, we evaluated various tools within the transfer learning toolbox using three prominent pre-trained architectures: VGG19, ResNet50, and EfficientNet V2 S. Through the collaborative efforts of Indigenous students and ML, we demonstrate that even with limited training resources, using transfer learning to re-purpose an existing model can achieve motif classification Top-1 accuracy of 79.76% and Top-5 of 94.56%. The project ran from 2021 to 2023, including three week-long sessions with students of Laura State School to trial the citizen science app and the evaluation, development and refinement of the ML models.

The DDP project not only serves as a beacon for community-centric research but also forges a new frontier in integrating Indigenous cultural heritage with modern technology. The impact reaches beyond academia, directly enriching the educational experience for Indigenous students in Laura and equipping local rangers and archaeologists with advanced tools for rock art conservation.

The EU H2020 project SPICE (Social cohesion, Participation, and Inclusion through Cultural Engagement) focuses on developing, designing, and implementing new methods and digital tools for citizen curation. This paper delineates several software tools developed within the project, presenting innovative approaches to represent and visualize citizens and communities resulting from their engagement with cultural heritage. Aligned with the central tenets of SPICE –particularly the notions of belonging and the Interpretation Reflection loop– the primary objective is to bolster citizens’ participation and inclusion in fostering social cohesion. This paper describes how the SPICE tools can be utilized to guide the processes of interpretation and reflection on cultural heritage artefacts. The Community Model serves as a pivotal component, enabling the modeling of citizens and communities through the utilization of similarity functions for clustering citizens based on perspectives. The clustering algorithm is intricately crafted to generate coherent communities, iterating until all clusters are interpretable using demographic attributes, centroid-based representations, and similarity attributes. Authors posit that this model holds significant value in comprehending and structuring complex data within cultural heritage contexts.

To exemplify our approach, the paper examines different attributes of individual citizens and citizen groups in the GAM (Galleria Civica d’Arte Moderna e Contemporanea) case study. Here, perspectives are delineated based on visitors’ demographic attributes and their emotional responses when engaging with artworks. These perspectives are then visualized using the VISIR tool, facilitating the exploration and revelation of connections between citizens and communities, thereby bridging the realms of citizen space and cultural heritage space.

The classification of works of art in terms of artistic style is a complex task. Some painting styles are closely related to the form of their brushstrokes. Salient examples are Pointillism and Impressionism, having both distinguishable brushstrokes characteristics which are small, rounded of clear color, repetitive dots for Pointillism style and visible, elongated and slanting, repetitive touches for Impressionism style. As Impressionism is the ancestral style of Pointillism, the two styles have many elements in common and distinguishing them is difficult. In this paper, specific texture features are investigated for the classification of the two styles, focusing mainly on small differences of their brushstrokes. The texture features adopted are: Granulometric features, Grey level co-occurrence matrix features, and Run length features. It is shown experimentally that Run Length method outperforms the other features and can efficiently (up to 95%) discriminate the two textured styles, since it incorporates information about, size, direction and intensity of brushstrokes.

The search for alternative teaching-learning processes that attract more interest and involvement of young people, has inspired the development of a game with a chatbot architecture based on interactive storytelling and multiple learning paths. Thus, we introduce in this article the GameBot ZoAm, developed for the Discord instant messaging and social platform. ZoAm offers a unique learning experience centered around storytelling, focusing on fundamental computing concepts and logical challenges that enhance computational thinking skills. Furthermore, the game also promotes an appreciation for Amazonian culture and folklore, with decision-making with human values. An action research study was conducted involving students from the last years of the end of elementary school. The research utilized a heuristic analysis based on the Gameplay Heuristics (PLAY) by Desurvire and Wiberg (ANO), and the evaluation model proposed by Korhonen and Koivisto (ANO) for mobile devices. The analysis employed a reduced and merged set of heuristics from these models, suited for the gamebot’s context, focusing on I) Usability, II) Gameplay and Immersion, and III) Mobility. Regarding the reliability coefficient used to evaluate the survey applied to students after playing the gamebot, Cronbach’s Alpha and Guttman Lambda-6 (G6(smc)) coefficients were applied. These metrics were chosen to ensure the internal consistency and reliability of survey items, reflecting on how effectively the questions measured the focuses proposed by the heuristic analysis. The findings indicate that the game has the potential to facilitate the assimilation of the integrated concepts and sustain student interest throughout gameplay.

The availability of digital music data in various modalities provides opportunities both for music enjoyment and music research. Regarding the latter, the computer-assisted analysis of tonal structures is a central topic. For Western classical music, studies typically rely on machine-readable scores, which are tedious to create for large-scale works and comprehensive corpora. As an alternative, music audio recordings, which are readily available, can be analyzed with computational methods. With this paper, we want to bridge the gap between score- and audio-based measurements of tonal structures by leveraging the power of deep neural networks. Such networks are commonly trained in an end-to-end fashion, which introduces biases towards the training repertoire or towards specific annotators. To overcome these problems, we propose a multi-step strategy. First, we compute pitch-class representations of the audio recordings using networks trained on score–audio pairs. Second, we measure the presence of specific tonal structures using a pattern-matching technique that solely relies on music theory knowledge and does not require annotated training data. Third, we highlight these measurements with interactive visualizations, thus leaving the interpretation to the musicological experts. Our experiments on Richard Wagner's large-scale cycle Der Ring des Nibelungen indicate that deep pitch-class representations lead to a high similarity between score- and audio-based measurements of tonal structures, thus demonstrating how to leverage multimodal data for application scenarios in the computational humanities, where an explicit and interpretable methodology is essential.

While a substantial amount of work has recently been devoted to improving the accuracy of computational Authorship Identification (AId) systems for textual data, little to no attention has been paid to endowing AId systems with the ability to explain the reasons behind their predictions. This substantially hinders the practical application of AId methods, since the predictions returned by such systems are hardly useful unless they are supported by suitable explanations. In this paper, we explore the applicability of existing general-purpose eXplainable Artificial Intelligence (XAI) techniques to AId, with a focus on explanations addressed to scholars working in cultural heritage. In particular, we assess the relative merits of three different types of XAI techniques (feature ranking, probing, factual and counterfactual selection) on three different AId tasks (authorship attribution, authorship verification, same-authorship verification) by running experiments on real AId textual data. Our analysis shows that, while these techniques make important first steps towards explainable Authorship Identification, more work remains to be done in order to provide tools that can be profitably integrated in the workflows of scholars.

The understanding of how users interact with the virtual cultural heritage could provide digital curators valuable insights into user behaviors, and also improve the overall user experience through the ability to observe and record interactions of virtual visitors. This paper introduces the new User Behavior (UB) tracking algorithm that we developed investigating a salience of the Virtual Reality (VR) panoramic regions. The algorithm extracts the importance of Region of Interest (ROI) determining patterns of the visitors’ virtual movement and interest in combination with statistics of captured browser activity. The input of our algorithm is the virtual online interactive platform (Virtual Museum of the Civic Art Gallery of Ascoli Piceno in Italy) with eighty-one 16386x8192 pixels panoramic images and several interactive features including maps, thumbnails, and menus. The software engine of the tracking model “Vice Versa” VR (3VR) operates on inverse functions of all descriptive functions (descriptors), which are assigned particularly to each interactive feature such as viewing multimedia content and observing the panoramic environment. The tracking experiment was performed online and the web virtual museum key study collected behavior information from 171 visitors around the world. Collected data, multimedia and textual content, and the coordinates of the ROIs are then subjected to standard statistics operations in order to define common patterns of UBs. Thus, we have discovered that the ROIs are mostly mapped onto the artworks and it is possible to obtain patterns about the main interests of users. The developed tool offers a guideline for the panoramic tours design and the potential benefits for museums are to understand the public, verify the effectiveness of choices, and re-shape a cultural offer based on visitors’ needs. Exploiting this kind of user experience, our algorithm ensures relevant feedback during virtual visits, and thus paves the way for further development of the recommender system.

An important challenge of Digital Cultural Heritage is to contribute to the recovery of artworks with their original shape and appearance. Many altarpieces, which are very relevant Christian art elements, have been damaged and/or, partly or fully, lost. Therefore, the only way to recover them is to carry out their digital reconstruction. Although the procedure that we present here is valid for any altarpiece with similar characteristics, and even for other akin elements, our test bench is the altarpieces damaged, destroyed, or disappeared during the Spanish Civil War (1936-1939) in Catalonia where most suffered these effects. The first step of our work has been the classification of these artworks into different categories on the basis of their degree of destruction and of the available visual information related to each one.

This paper proposes, for the first time to our knowledge, a workflow for the virtual reconstruction, through photogrammetry, digital modeling, and digital color restoration; of whole altarpieces partially preserved with very little visual information. Our case study is the Rosary’s altarpiece of Sant Pere Màrtir de Manresa church. Currently, this altarpiece is partially preserved in fragments in the Museu Comarcal de Manresa (Spain). But, it can not be reassembled physically owing to the lack of space (actually the church does not exist anymore) and the cost of such an operation. Thus, there is no other solution than the digital one to contemplate and study the altarpiece as a whole. The reconstruction that we provide allows art historians and the general public to virtually see the altarpiece complete and assembled as it was until 1936. The results obtained also allow us to see in detail the reliefs and ornaments of the altarpiece with their digitally restored color.

The teaching and learning of architectural heritage can play a vital role in engaging students in envisioning the past and nurturing their cultural identity. However, this endeavour often faces challenges stemming from limited access to heritage sites and varying levels of student interest in the subject matter. This project introduces an augmented reality (AR) book design that seamlessly integrates pop-up three-dimensional models with a dedicated mobile app, effectively illustrating the architectural designs of educational heritages in Hong Kong Central and Western District. An evaluative study was conducted to assess the usability of the AR book for architectural heritage education, which employed a mixed-methods design comprising questionnaire survey with heritage education undergraduate students (N = 80) and semi-structured interviews with a subset of the participants. The results revealed a favourable response to the use of AR technology for virtual representations of cultural heritage, alongside participants’ positive attitudes towards the virtual learning experience facilitated by the AR book. The findings of this study underscore the feasibility and potential benefits of integrating AR technology into architectural heritage education. This integration can offer digitally-mediated learning experiences that actively engage young learners in the exploration and preservation of cultural heritage.