The cultural heritage buildings (CHB), which are part of mankind’s history and identity, are in constant danger of damage, or in extreme cases, complete destruction. Thus, it’s of utmost importance to preserve them by identifying the existent, or presumptive, defects using novel methods so that renovation processes can be done in a timely manner and with higher accuracy. The main goal of this research is to use new Deep Learning (DL) methods in the process of preserving CHBs (situated in Iran); a goal that has been neglected especially in developing countries such as Iran, as these countries still preserve their CHBs using manual, and even archaic, methods that need direct human supervision. Having proven their effectiveness and performance when it comes to processing images, the Convolutional Neural Networks (CNNs) are a staple in computer vision (CV) literacy and this paper is not exempt. When lacking enough CHB images, training a CNN from scratch would be very difficult and prone to overfitting; that’s why we opted to use a technique called transfer learning (TL) in which we used pre-trained ResNet, MobileNet, and Inception networks, for classification. Even more, the Grad-CAM was utilized to localize the defects to some extent. The final results were very favorable, compared to similar papers. We reached 94% in Precision, Recall, and F1-Score with our fine-tuned MobileNetV2 model, which showed a 4-5% improvement over other similar works. The final proposed model can pave the way for moving from manual to unmanned CHB conservation, hence an increase in accuracy and a decrease in human-induced errors.
{"title":"Deep Learning for Identifying Iran’s Cultural Heritage Buildings in Need of Conservation Using Image Classification and Grad-CAM","authors":"Mahdi Bahrami, Amir Albadvi","doi":"10.1145/3631130","DOIUrl":"https://doi.org/10.1145/3631130","url":null,"abstract":"The cultural heritage buildings (CHB), which are part of mankind’s history and identity, are in constant danger of damage, or in extreme cases, complete destruction. Thus, it’s of utmost importance to preserve them by identifying the existent, or presumptive, defects using novel methods so that renovation processes can be done in a timely manner and with higher accuracy. The main goal of this research is to use new Deep Learning (DL) methods in the process of preserving CHBs (situated in Iran); a goal that has been neglected especially in developing countries such as Iran, as these countries still preserve their CHBs using manual, and even archaic, methods that need direct human supervision. Having proven their effectiveness and performance when it comes to processing images, the Convolutional Neural Networks (CNNs) are a staple in computer vision (CV) literacy and this paper is not exempt. When lacking enough CHB images, training a CNN from scratch would be very difficult and prone to overfitting; that’s why we opted to use a technique called transfer learning (TL) in which we used pre-trained ResNet, MobileNet, and Inception networks, for classification. Even more, the Grad-CAM was utilized to localize the defects to some extent. The final results were very favorable, compared to similar papers. We reached 94% in Precision, Recall, and F1-Score with our fine-tuned MobileNetV2 model, which showed a 4-5% improvement over other similar works. The final proposed model can pave the way for moving from manual to unmanned CHB conservation, hence an increase in accuracy and a decrease in human-induced errors.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"143 ","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136102845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Conventional studies on the satisfaction of museum visitors focus on collecting information through surveys to provide a one-way service to visitors, and thus it is impossible to obtain feedback on the real-time satisfaction of visitors who are experiencing the museum exhibition program. In addition, museum practitioners lack research on automated ways to evaluate a produced content program's life cycle and its appropriateness. To overcome these problems, we propose a novel multi-convolutional neural network (CNN), called VimoNet, which is able to recognize visitors emotions automatically in real-time based on their facial expressions and body gestures. Furthermore, we design a user preference model of content and a framework to obtain feedback on content improvement for providing personalized digital cultural heritage content to visitors. Specifically, we define seven emotions of visitors and build a dataset of visitor facial expressions and gestures with respect to the emotions. Using the dataset, we proceed with feature fusion of face and gesture images trained on the DenseNet-201 and VGG-16 models for generating a combined emotion recognition model. From the results of the experiment, VimoNet achieved a classification accuracy of 84.10%, providing 7.60% and 14.31% improvement, respectively, over a single face and body gesture-based method of emotion classification performance. It is thus possible to automatically capture the emotions of museum visitors via VimoNet, and we confirm its feasibility through a case study with respect to digital content of cultural heritage.
{"title":"Real-time Multi-CNN based Emotion Recognition System for Evaluating Museum Visitors’ Satisfaction","authors":"Do Hyung Kwon, Jeong Min Yu","doi":"10.1145/3631123","DOIUrl":"https://doi.org/10.1145/3631123","url":null,"abstract":"Conventional studies on the satisfaction of museum visitors focus on collecting information through surveys to provide a one-way service to visitors, and thus it is impossible to obtain feedback on the real-time satisfaction of visitors who are experiencing the museum exhibition program. In addition, museum practitioners lack research on automated ways to evaluate a produced content program's life cycle and its appropriateness. To overcome these problems, we propose a novel multi-convolutional neural network (CNN), called VimoNet, which is able to recognize visitors emotions automatically in real-time based on their facial expressions and body gestures. Furthermore, we design a user preference model of content and a framework to obtain feedback on content improvement for providing personalized digital cultural heritage content to visitors. Specifically, we define seven emotions of visitors and build a dataset of visitor facial expressions and gestures with respect to the emotions. Using the dataset, we proceed with feature fusion of face and gesture images trained on the DenseNet-201 and VGG-16 models for generating a combined emotion recognition model. From the results of the experiment, VimoNet achieved a classification accuracy of 84.10%, providing 7.60% and 14.31% improvement, respectively, over a single face and body gesture-based method of emotion classification performance. It is thus possible to automatically capture the emotions of museum visitors via VimoNet, and we confirm its feasibility through a case study with respect to digital content of cultural heritage.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"140 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136104310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pérez-Pascual A., Giménez-López J.L., Palacio D., Marín-Roig J.
O-City is a non-profit project funded by European Union with the aim of promoting Orange Economy throughout education and collaboration among municipalities, educational entities and businesses. This project has two main assets, the O-City e-learning platform and the O-City World platform. This paper presents the technical aspects of the O-City World platform, which is a digital application that allows interaction among cities, educators and professionals. This platform has a role-based access control with seven different users able to perform different functionalities. Thanks to the collaboration among these stakeholders the platform is growing exponentially.
{"title":"O-City: Implementation of an Innovative Multimedia Platform for Promoting Orange Economy","authors":"Pérez-Pascual A., Giménez-López J.L., Palacio D., Marín-Roig J.","doi":"10.1145/3631121","DOIUrl":"https://doi.org/10.1145/3631121","url":null,"abstract":"O-City is a non-profit project funded by European Union with the aim of promoting Orange Economy throughout education and collaboration among municipalities, educational entities and businesses. This project has two main assets, the O-City e-learning platform and the O-City World platform. This paper presents the technical aspects of the O-City World platform, which is a digital application that allows interaction among cities, educators and professionals. This platform has a role-based access control with seven different users able to perform different functionalities. Thanks to the collaboration among these stakeholders the platform is growing exponentially.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136019605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
As a combination of information computing technology and the cultural field, cultural computing is gaining more and more attention. The knowledge graph is also gradually applied as a particular data structure in the cultural area. Based on the domain knowledge graph data of the Beijing Municipal Social Science Project ”Mining and Utilization of Cultural Resources in the Ancient Capital of Beijing,” this paper proposes a graph representation learning model CR-TransR that integrates cultural attributes. Through the analysis of the data in the cultural field of the ancient capital of Beijing, a cultural feature dictionary is constructed, and a domain-specific feature matrix is constructed in the form of word vector splicing. The feature matrix is used to constrain the embedding graph model TransR, and then the feature matrix and the TransR model are jointly trained to complete the embedded expression of the knowledge graph. Finally, a comparative experiment is carried out on the Beijing ancient capital cultural knowledge graph dataset and the effects of the classic graph embedding algorithms TransE, TransH, and TransR. At the same time, we try to reproduce the embedding method with the core idea of neighbor node information aggregation as the core idea, and CRTransR are compared. The experimental tasks include link prediction and triplet classification, and the experimental results show that the CRTransR model performs better.
{"title":"CR-TransR: A Knowledge Graph Embedding Model for Cultural Domain","authors":"Wenjun Hou, Bing Bai, Chenyang Cai","doi":"10.1145/3625299","DOIUrl":"https://doi.org/10.1145/3625299","url":null,"abstract":"As a combination of information computing technology and the cultural field, cultural computing is gaining more and more attention. The knowledge graph is also gradually applied as a particular data structure in the cultural area. Based on the domain knowledge graph data of the Beijing Municipal Social Science Project ”Mining and Utilization of Cultural Resources in the Ancient Capital of Beijing,” this paper proposes a graph representation learning model CR-TransR that integrates cultural attributes. Through the analysis of the data in the cultural field of the ancient capital of Beijing, a cultural feature dictionary is constructed, and a domain-specific feature matrix is constructed in the form of word vector splicing. The feature matrix is used to constrain the embedding graph model TransR, and then the feature matrix and the TransR model are jointly trained to complete the embedded expression of the knowledge graph. Finally, a comparative experiment is carried out on the Beijing ancient capital cultural knowledge graph dataset and the effects of the classic graph embedding algorithms TransE, TransH, and TransR. At the same time, we try to reproduce the embedding method with the core idea of neighbor node information aggregation as the core idea, and CRTransR are compared. The experimental tasks include link prediction and triplet classification, and the experimental results show that the CRTransR model performs better.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"96 4","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135218912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrea Ballatore, Valeri Katerinchuk, Alexandra Poulovassilis, Peter T. Wood
The COVID-19 pandemic led to the temporary closure of all museums in the UK, closing buildings and suspending all on-site activities. Museum agencies aim at mitigating and managing these impacts on the sector, in a context of chronic data scarcity. “Museums in the Pandemic” is an interdisciplinary project that utilises content scraped from museums’ websites and social media posts in order to understand how the UK museum sector, currently comprising over 3,300 museums, has responded and is currently responding to the pandemic. A major part of the project has been the design of computational techniques to provide the project’s museum studies experts with appropriate data and tools for undertaking this research, leveraging web analytics, natural language processing, and machine learning. In this methodological contribution, firstly, we developed techniques to retrieve and identify museum official websites and social media accounts (Facebook and Twitter). This supported the automated capture of large-scale online data about the entire UK museum sector. Secondly, we harnessed convolutional neural networks to extract activity indicators from unstructured text in order to detect museum behaviours, including openings, closures, fundraising, and staffing. This dynamic dataset is enabling the museum studies experts in the team to study patterns in the online presence of museums before, during, and after the pandemic, according to museum size, governance, accreditation, and location
{"title":"Tracking museums’ online responses to the Covid-19 pandemic: a study in museum analytics","authors":"Andrea Ballatore, Valeri Katerinchuk, Alexandra Poulovassilis, Peter T. Wood","doi":"10.1145/3627165","DOIUrl":"https://doi.org/10.1145/3627165","url":null,"abstract":"The COVID-19 pandemic led to the temporary closure of all museums in the UK, closing buildings and suspending all on-site activities. Museum agencies aim at mitigating and managing these impacts on the sector, in a context of chronic data scarcity. “Museums in the Pandemic” is an interdisciplinary project that utilises content scraped from museums’ websites and social media posts in order to understand how the UK museum sector, currently comprising over 3,300 museums, has responded and is currently responding to the pandemic. A major part of the project has been the design of computational techniques to provide the project’s museum studies experts with appropriate data and tools for undertaking this research, leveraging web analytics, natural language processing, and machine learning. In this methodological contribution, firstly, we developed techniques to retrieve and identify museum official websites and social media accounts (Facebook and Twitter). This supported the automated capture of large-scale online data about the entire UK museum sector. Secondly, we harnessed convolutional neural networks to extract activity indicators from unstructured text in order to detect museum behaviours, including openings, closures, fundraising, and staffing. This dynamic dataset is enabling the museum studies experts in the team to study patterns in the online presence of museums before, during, and after the pandemic, according to museum size, governance, accreditation, and location","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135729355","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this article, by the ability to translate Aramaic to another spoken languages, we investigated Machine Translation (MT) in a cultural heritage domain for two primary purposes: evaluating the quality of ancient translations and preserving Aramaic (an endangered language). First, we detailed the construction of a publicly available Biblical parallel Aramaic-Hebrew corpus based on two ancient (early 2 nd - late 4 th century) Hebrew–Aramaic translations: Targum Onkelus and Targum Jonathan. Then using the Statistical Machine Translation (SMT) approach, which in our use-case significantly outperforms the Neural Machine Translation (NMT), we validated the excepted high quality of the translations. The trained model failed to translate Aramaic texts of other dialects. However, when we trained the same SMT model on another Aramaic-Hebrew corpus of a different dialect (Zohar - 13 th century) a very high translation score was achieved. We examined an additional important cultural heritage source of Aramaic texts, the Babylonian Talmud (early 3 rd - late 5 th century). Since we do not have a parallel Aramaic-Hebrew corpus of the Talmud, we used the model trained on the Bible corpus for translation. We performed an analysis of the results and suggest some potential promising future research.
{"title":"Machine Translation for Historical Research: A case study of Aramaic-Ancient Hebrew Translations","authors":"Chaya Liebeskind, Shmuel Liebeskind, Dan Bouhnik","doi":"10.1145/3627168","DOIUrl":"https://doi.org/10.1145/3627168","url":null,"abstract":"In this article, by the ability to translate Aramaic to another spoken languages, we investigated Machine Translation (MT) in a cultural heritage domain for two primary purposes: evaluating the quality of ancient translations and preserving Aramaic (an endangered language). First, we detailed the construction of a publicly available Biblical parallel Aramaic-Hebrew corpus based on two ancient (early 2 nd - late 4 th century) Hebrew–Aramaic translations: Targum Onkelus and Targum Jonathan. Then using the Statistical Machine Translation (SMT) approach, which in our use-case significantly outperforms the Neural Machine Translation (NMT), we validated the excepted high quality of the translations. The trained model failed to translate Aramaic texts of other dialects. However, when we trained the same SMT model on another Aramaic-Hebrew corpus of a different dialect (Zohar - 13 th century) a very high translation score was achieved. We examined an additional important cultural heritage source of Aramaic texts, the Babylonian Talmud (early 3 rd - late 5 th century). Since we do not have a parallel Aramaic-Hebrew corpus of the Talmud, we used the model trained on the Bible corpus for translation. We performed an analysis of the results and suggest some potential promising future research.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136079062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper is part of a doctoral research that proposes to capitalize on the environmental knowledge drawn from traditional Algerian domestic architecture, supported by a knowledge-based platform. This research aims to: 1) build a capital of knowledge related to traditional environmental devices (EDs) allowing to suggest them as "references" to propose conceptual solutions during the upstream phases of the architectural design; 2) support the Algerian government's policy of preservation and digitization of architectural heritage; 3) support the government's policy of reducing energy consumption in the building sector. From this perspective, this paper proposes an interactive knowledge capitalization process involving Information and Communication Technologies (ICTs) for the modeling, exploitation and visualization of EDs-related knowledge. This paper will present the proposed process for knowledge capitalization leading to the development of the knowledge-driven platform Eco-Diars intended to enable designers to, efficiently, perform their queries related to traditional environmental devices.
{"title":"Proposal of a knowledge capitalization process to construct <i>Eco-Diars</i> : A Knowledge-driven platform applied to traditional Algerian domestic architecture","authors":"Racha Amrani, Sabrina Kacher, Selma Khouri, Houda Oufaida, Safia Ouahaba, Mouna Cherrad","doi":"10.1145/3627166","DOIUrl":"https://doi.org/10.1145/3627166","url":null,"abstract":"This paper is part of a doctoral research that proposes to capitalize on the environmental knowledge drawn from traditional Algerian domestic architecture, supported by a knowledge-based platform. This research aims to: 1) build a capital of knowledge related to traditional environmental devices (EDs) allowing to suggest them as \"references\" to propose conceptual solutions during the upstream phases of the architectural design; 2) support the Algerian government's policy of preservation and digitization of architectural heritage; 3) support the government's policy of reducing energy consumption in the building sector. From this perspective, this paper proposes an interactive knowledge capitalization process involving Information and Communication Technologies (ICTs) for the modeling, exploitation and visualization of EDs-related knowledge. This paper will present the proposed process for knowledge capitalization leading to the development of the knowledge-driven platform Eco-Diars intended to enable designers to, efficiently, perform their queries related to traditional environmental devices.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135856072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The sustainability of digital research outputs, particularly in the humanities where these frequently comprise archives of digital cultural heritage material, has always offered a challenge to the researchers and institutions who have responsibility for them. The amount of upfront care, effort and funding that goes into developing a research project during the active (and funded) research phase is rarely replicated within the post-project maintenance and curation of the delivered digital assets or archives. What often defines the sustainability of a research project and its archive is a combination of research method and expected life span for the digital collection. Innovation in research data design is often at the expense of its longevity. But this does not need to be so. The trade-off between longevity and functionality is a false dichotomy. Yet what is clear is that care and consideration in planning the research data storage or archive for a project can make a big difference. A data management plan that meets grant funder requirements is asked for many research projects, but is more than simply a funding document. Good research data management ensures outputs are available online for years to come, and available for future research and innovation. This paper offers a practical insight to the methods being employed at the University of Oxford to support Digital Humanities scholars (and beyond) safeguard their digital legacy for future generations.
{"title":"Consolidating Research Data Management Infrastructure: Towards Sustainable Digital Scholarship","authors":"Megan Gooch, Damon Strange","doi":"10.1145/3627169","DOIUrl":"https://doi.org/10.1145/3627169","url":null,"abstract":"The sustainability of digital research outputs, particularly in the humanities where these frequently comprise archives of digital cultural heritage material, has always offered a challenge to the researchers and institutions who have responsibility for them. The amount of upfront care, effort and funding that goes into developing a research project during the active (and funded) research phase is rarely replicated within the post-project maintenance and curation of the delivered digital assets or archives. What often defines the sustainability of a research project and its archive is a combination of research method and expected life span for the digital collection. Innovation in research data design is often at the expense of its longevity. But this does not need to be so. The trade-off between longevity and functionality is a false dichotomy. Yet what is clear is that care and consideration in planning the research data storage or archive for a project can make a big difference. A data management plan that meets grant funder requirements is asked for many research projects, but is more than simply a funding document. Good research data management ensures outputs are available online for years to come, and available for future research and innovation. This paper offers a practical insight to the methods being employed at the University of Oxford to support Digital Humanities scholars (and beyond) safeguard their digital legacy for future generations.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136098468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Audiovisual (AV) archives, as an essential reservoir of our cultural assets, are suffering from the issue of accessibility. The complex nature of the medium itself made processing and interaction an open challenge still in the field of computer vision, multimodal learning, and human-computer interaction, as well as in culture and heritage. In recent years, with the raising of video retrieval tasks, methods in retrieving video content with natural language (text-to-video retrieval) gained quite some attention and have reached a performance level where real-world application is on the horizon. Appealing as it may sound, such methods focus on retrieving videos using plain visual-focused descriptions of what has happened in the video and finding videos such as instructions. It is too early to say such methods would be the new paradigms for accessing and encoding complex video content into high-dimensional data, but they are indeed innovative attempts and foundations to build future exploratory interfaces for AV archives (e.g. allow users to write stories and retrieve related snippets in the archive, or encoding video content at high-level for visualisation). This work filled the application gap by examining such text-to-video retrieval methods from an implementation point of view and proposed and verified a classifier-enhanced workflow to allow better results when dealing with in-situ queries that might have been different from the training dataset. Such a workflow is then applied to the real-world archive from Télévision Suisse Romande (RTS) to create a demo. At last, a human-centred evaluation is conducted to understand whether the text-to-video retrieval methods improve the overall experience of accessing AV archives.
{"title":"Write What You Want: Applying Text-to-video Retrieval to Audiovisual Archives","authors":"Yuchen Yang","doi":"10.1145/3627167","DOIUrl":"https://doi.org/10.1145/3627167","url":null,"abstract":"Audiovisual (AV) archives, as an essential reservoir of our cultural assets, are suffering from the issue of accessibility. The complex nature of the medium itself made processing and interaction an open challenge still in the field of computer vision, multimodal learning, and human-computer interaction, as well as in culture and heritage. In recent years, with the raising of video retrieval tasks, methods in retrieving video content with natural language (text-to-video retrieval) gained quite some attention and have reached a performance level where real-world application is on the horizon. Appealing as it may sound, such methods focus on retrieving videos using plain visual-focused descriptions of what has happened in the video and finding videos such as instructions. It is too early to say such methods would be the new paradigms for accessing and encoding complex video content into high-dimensional data, but they are indeed innovative attempts and foundations to build future exploratory interfaces for AV archives (e.g. allow users to write stories and retrieve related snippets in the archive, or encoding video content at high-level for visualisation). This work filled the application gap by examining such text-to-video retrieval methods from an implementation point of view and proposed and verified a classifier-enhanced workflow to allow better results when dealing with in-situ queries that might have been different from the training dataset. Such a workflow is then applied to the real-world archive from Télévision Suisse Romande (RTS) to create a demo. At last, a human-centred evaluation is conducted to understand whether the text-to-video retrieval methods improve the overall experience of accessing AV archives.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136295540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Eljas Oksanen, Frida Ehrnsten, Heikki Rantala, Eero Hyvönen
Museums, heritage agencies and other institutions responsible for managing archaeological cultural heritage across Europe are engaged in developing digital platforms to better open their collections to the public as a common resource for the purposes of discovering, learning about, and sharing our common past. This paper explores the potential of new semantic computing technologies in democratising not only public access to digital cultural heritage records, but also to computational and Linked Open Data -assisted data analysis and knowledge discovery. As a case study, we consider archaeological and numismatic Open Data services in Finland, and discuss the research results obtained during the ongoing development work for the CoinSampo framework for opening Finnish and international numismatic data. Existing digital cultural heritage services are often built with the needs of professional collections management in mind. The presentation of the records is typically structured after the familiar format established for the printed catalogues of yesteryear, with few analytical tools that would take advantage of the potential of digital data to probe and visualize internal relationships and patterns within the full body of the opened material. CoinSampo, however, will provide scientific tools to new audiences among the non-professional public who have not enjoyed such a level of access to numismatic data. The broad range of target audiences we envisage includes collections managers, who will benefit from enhanced access to their own data for updating records and for error detection and correction, as well as academic researchers interested in using the material in scientific analysis. Importantly, it also includes non-professional groups such as coin collectors, educators, local historians, and the archaeological hobby metal-detectorists who produce most of the new coin finds entering Finnish and European collections. By adopting a citizen science and participatory heritage approach in the development of Open Data services, we aim to promote a technological model for cultural heritage dissemination that addresses the needs of a wide spectrum of different user audiences inside and outside the professional sphere.
{"title":"Semantic Solutions for Democratizing Archaeological and Numismatic Data Analysis","authors":"Eljas Oksanen, Frida Ehrnsten, Heikki Rantala, Eero Hyvönen","doi":"10.1145/3625302","DOIUrl":"https://doi.org/10.1145/3625302","url":null,"abstract":"Museums, heritage agencies and other institutions responsible for managing archaeological cultural heritage across Europe are engaged in developing digital platforms to better open their collections to the public as a common resource for the purposes of discovering, learning about, and sharing our common past. This paper explores the potential of new semantic computing technologies in democratising not only public access to digital cultural heritage records, but also to computational and Linked Open Data -assisted data analysis and knowledge discovery. As a case study, we consider archaeological and numismatic Open Data services in Finland, and discuss the research results obtained during the ongoing development work for the CoinSampo framework for opening Finnish and international numismatic data. Existing digital cultural heritage services are often built with the needs of professional collections management in mind. The presentation of the records is typically structured after the familiar format established for the printed catalogues of yesteryear, with few analytical tools that would take advantage of the potential of digital data to probe and visualize internal relationships and patterns within the full body of the opened material. CoinSampo, however, will provide scientific tools to new audiences among the non-professional public who have not enjoyed such a level of access to numismatic data. The broad range of target audiences we envisage includes collections managers, who will benefit from enhanced access to their own data for updating records and for error detection and correction, as well as academic researchers interested in using the material in scientific analysis. Importantly, it also includes non-professional groups such as coin collectors, educators, local historians, and the archaeological hobby metal-detectorists who produce most of the new coin finds entering Finnish and European collections. By adopting a citizen science and participatory heritage approach in the development of Open Data services, we aim to promote a technological model for cultural heritage dissemination that addresses the needs of a wide spectrum of different user audiences inside and outside the professional sphere.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135347690","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：