Ana Regina Mizrahy Cuperschmid, Aline Prado Costa, Igor Neves, Flávio Raffaelli Fonseca, Ethel Mizrahy Cuperschmid
Built between 1911 and 1914 by architect Francisco Izidoro Monteiro, the former neoclassical building of the Belo Horizonte School of Medicine (BHSM) was located in a central area of the city. Its design, inspired by the European precepts at the time, was embedded in the same architectural language that characterized the cityscape of the capital of Minas Gerais, Brazil. The original building was gradually dismantled and completely demolished in 1960 to make way for the present headquarters. Few records of its existence remain. In order to contribute to the construction of the digital heritage of the old BHSM building, this paper presents its virtual reconstruction, demonstrates and analyzes the application of Heritage Building Information Modeling (HBIM) for this purpose. Based on data collection and the organization of numerous references (historical photographs, former students' recollections, magazine articles, books, minutes and photogrammetry of the remains), a hypothetical model of the building was developed to represent its condition as built and prior to demolition. Despite the difficulties in modeling, it was possible to perform a virtual historical reconstruction of the building, providing a better understanding of the building that does not currently exist. The preservation of the digital heritage of the demolished building aims to raise public awareness of its memory, which is consistent with the collection of the Memory Center of the UFMG School of Medicine (CEMEMOR).
贝洛奥里藏特医学院(Belo Horizonte School of Medicine,BHSM)的前身是一座新古典主义建筑,由建筑师弗朗西斯科-伊齐多罗-蒙泰罗(Francisco Izidoro Monteiro)于 1911 年至 1914 年间建造,位于城市中心地带。其设计灵感来自当时的欧洲理念,与巴西米纳斯吉拉斯州首府城市景观的建筑语言如出一辙。最初的建筑逐渐被拆除,并于 1960 年被完全拆毁,为现在的总部大楼让路。有关其存在的记录已所剩无几。为了促进 BHSM 老建筑数字遗产的建设,本文介绍了该建筑的虚拟重建,演示并分析了遗产建筑信息建模(HBIM)在这方面的应用。在收集数据和整理大量参考资料(历史照片、昔日学生的回忆、杂志文章、书籍、会议记录和遗迹摄影测量)的基础上,建立了该建筑的假想模型,以表现其建成时和拆除前的状况。尽管建模困难重重,但仍有可能对该建筑进行虚拟历史重建,从而更好地了解这座目前尚不存在的建筑。保存被拆除建筑的数字遗产旨在提高公众对其记忆的认识,这与 UFMG 医学院记忆中心(CEMEMOR)的收藏是一致的。
{"title":"Virtual Reconstruction of the Deconstructed Heritage: The Former Building of the Belo Horizonte School of Medicine","authors":"Ana Regina Mizrahy Cuperschmid, Aline Prado Costa, Igor Neves, Flávio Raffaelli Fonseca, Ethel Mizrahy Cuperschmid","doi":"10.1145/3639931","DOIUrl":"https://doi.org/10.1145/3639931","url":null,"abstract":"<p>Built between 1911 and 1914 by architect Francisco Izidoro Monteiro, the former neoclassical building of the Belo Horizonte School of Medicine (BHSM) was located in a central area of the city. Its design, inspired by the European precepts at the time, was embedded in the same architectural language that characterized the cityscape of the capital of Minas Gerais, Brazil. The original building was gradually dismantled and completely demolished in 1960 to make way for the present headquarters. Few records of its existence remain. In order to contribute to the construction of the digital heritage of the old BHSM building, this paper presents its virtual reconstruction, demonstrates and analyzes the application of Heritage Building Information Modeling (HBIM) for this purpose. Based on data collection and the organization of numerous references (historical photographs, former students' recollections, magazine articles, books, minutes and photogrammetry of the remains), a hypothetical model of the building was developed to represent its condition as built and prior to demolition. Despite the difficulties in modeling, it was possible to perform a virtual historical reconstruction of the building, providing a better understanding of the building that does not currently exist. The preservation of the digital heritage of the demolished building aims to raise public awareness of its memory, which is consistent with the collection of the Memory Center of the UFMG School of Medicine (CEMEMOR).</p>","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"149 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139408620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Călin Neamțu, Radu Comes, Dorin-Mircea Popovici, Elena Băutu, Mateescu-Suciu Liliana, Adrien Syrotnik, Matei-Ioan Popovici
Cultural institutions are increasingly prioritizing the utilization of diverse mixed reality technologies to enrich the visitor experience during their physical exhibitions. These institutions are actively integrating virtual assets to enhance interactivity and facilitate immersive storytelling experiences. The use of extended reality technologies in cultural heritage dissemination has become a common solution to improve the perception on cultural heritage of their visitors. The objective of the proposed research is to conduct a comparative analysis of three distinct extended reality applications used in cultural heritage experiences. The proposed workflow integrates advanced 3D scanning technology and additive manufacturing techniques to produce tangible replicas which are further enhanced using extended reality technologies, such as Augmented Reality, Desktop Virtual Reality, and Immersive Virtual Reality. To evaluate the developed extended reality applications, we have conducted various System Usability Scale tools to assess applications’ usability while also trying to identify the most appropriate application that can be integrated within the museum exhibition. From a utility point of view we found no notable differences in the user perceptions between proposed solutions, while from a usability and satisfaction point of view, the most preferred solution was the Augmented Reality one. The workflow is presented in sufficient detail to ensure the replication of the results.
{"title":"Evaluating User Experience in the Context of Cultural Heritage Dissemination Using Extended Reality: A Case Study of the Dacian Bronze Matrix with Hollow Design","authors":"Călin Neamțu, Radu Comes, Dorin-Mircea Popovici, Elena Băutu, Mateescu-Suciu Liliana, Adrien Syrotnik, Matei-Ioan Popovici","doi":"10.1145/3639933","DOIUrl":"https://doi.org/10.1145/3639933","url":null,"abstract":"<p>Cultural institutions are increasingly prioritizing the utilization of diverse mixed reality technologies to enrich the visitor experience during their physical exhibitions. These institutions are actively integrating virtual assets to enhance interactivity and facilitate immersive storytelling experiences. The use of extended reality technologies in cultural heritage dissemination has become a common solution to improve the perception on cultural heritage of their visitors. The objective of the proposed research is to conduct a comparative analysis of three distinct extended reality applications used in cultural heritage experiences. The proposed workflow integrates advanced 3D scanning technology and additive manufacturing techniques to produce tangible replicas which are further enhanced using extended reality technologies, such as Augmented Reality, Desktop Virtual Reality, and Immersive Virtual Reality. To evaluate the developed extended reality applications, we have conducted various System Usability Scale tools to assess applications’ usability while also trying to identify the most appropriate application that can be integrated within the museum exhibition. From a utility point of view we found no notable differences in the user perceptions between proposed solutions, while from a usability and satisfaction point of view, the most preferred solution was the Augmented Reality one. The workflow is presented in sufficient detail to ensure the replication of the results.</p>","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"8 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139409295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Digital reconstruction of ancient buildings is very challenging due to its architectural complexity and structural delicacy. Therefore, how to apply new Earth Observation (EO) technology to digitally reconstruct the complete and real model of ancient buildings without damage has become an urgent issue. This paper proposed a multi-scan point cloud hierarchical registration method for 3D reconstruction of ancient Chinese Great Wild Goose Pagoda based on Terrestrial Laser Scanner (TLS) data. First, a multi-scan point cloud hierarchical registration method is elaborate designed for 3D reconstruction. Then, the outline feature of ancient Chinese Great Wild Goose Pagoda is automatically extracted by point cloud slicing. Finally, fine modeling is carried out according to the outline feature extracted from the reconstructed point cloud, after that block combination and texture mapping are conducted for 3D digitalization and documentation. The experiments indicate that the Root Mean Square Error (RMSE) of multi-scan hierarchical registration of the translation and rotation are less than 3 cm and 0.862° respectively, and the complete and real model of the Great Wild Goose Pagoda is obtained by fine modeling and texture mapping, which provides an important reference for 3D digital protection of ancient buildings.
{"title":"Digital 3D Reconstruction of Ancient Chinese Great Wild Goose Pagoda by TLS Point Clouds Hierarchical Registration","authors":"Xiaohu Lin, Bei Xue, Xiqi Wang","doi":"10.1145/3639932","DOIUrl":"https://doi.org/10.1145/3639932","url":null,"abstract":"<p>Digital reconstruction of ancient buildings is very challenging due to its architectural complexity and structural delicacy. Therefore, how to apply new Earth Observation (EO) technology to digitally reconstruct the complete and real model of ancient buildings without damage has become an urgent issue. This paper proposed a multi-scan point cloud hierarchical registration method for 3D reconstruction of ancient Chinese Great Wild Goose Pagoda based on Terrestrial Laser Scanner (TLS) data. First, a multi-scan point cloud hierarchical registration method is elaborate designed for 3D reconstruction. Then, the outline feature of ancient Chinese Great Wild Goose Pagoda is automatically extracted by point cloud slicing. Finally, fine modeling is carried out according to the outline feature extracted from the reconstructed point cloud, after that block combination and texture mapping are conducted for 3D digitalization and documentation. The experiments indicate that the Root Mean Square Error (RMSE) of multi-scan hierarchical registration of the translation and rotation are less than 3 <i>cm</i> and 0.862° respectively, and the complete and real model of the Great Wild Goose Pagoda is obtained by fine modeling and texture mapping, which provides an important reference for 3D digital protection of ancient buildings.</p>","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"2 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139408652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Innovative technologies are improving the accessibility, preservation and searchability of born-digital and digitised records. In particular, Artificial Intelligence (AI) is opening new opportunities for archivists and researchers. However, the experience of scholars (particularly humanities scholars) and other users remain understudied. This article asks how and why researchers and general users are, or are not, using computational methods. This research is informed by an open-call survey, completed by 22 individuals, and semi-structured interviews with 33 professionals, including archivists, librarians, digital humanists, literary scholars, historians, and computer scientists. Drawing on these results, this article offers an analysis of user experiences of computational research methods applied to digitised and born-digital archives. With a focus on humanities and social science researchers, this article also discusses users who resist this kind of research, perhaps because they lack the skills necessary to engage with these materials at scale, or because they prefer to use more traditional methods, such as close reading and historical analysis. Here, we explore the uses of computational and more ‘traditional’ research methodologies applied to digital records. We also make a series of recommendations to elevate users’ computational skills but also to improve the digital infrastructure to make archives more accessible and usable.
{"title":"Are Users of Digital Archives Ready for the AI Era? Obstacles to the Application of Computational Research Methods and New Opportunities","authors":"Lise Jaillant, Katherine Aske","doi":"10.1145/3631125","DOIUrl":"https://doi.org/10.1145/3631125","url":null,"abstract":" Innovative technologies are improving the accessibility, preservation and searchability of born-digital and digitised records. In particular, Artificial Intelligence (AI) is opening new opportunities for archivists and researchers. However, the experience of scholars (particularly humanities scholars) and other users remain understudied. This article asks how and why researchers and general users are, or are not, using computational methods. This research is informed by an open-call survey, completed by 22 individuals, and semi-structured interviews with 33 professionals, including archivists, librarians, digital humanists, literary scholars, historians, and computer scientists. Drawing on these results, this article offers an analysis of user experiences of computational research methods applied to digitised and born-digital archives. With a focus on humanities and social science researchers, this article also discusses users who resist this kind of research, perhaps because they lack the skills necessary to engage with these materials at scale, or because they prefer to use more traditional methods, such as close reading and historical analysis. Here, we explore the uses of computational and more ‘traditional’ research methodologies applied to digital records. We also make a series of recommendations to elevate users’ computational skills but also to improve the digital infrastructure to make archives more accessible and usable.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"18 4","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139380746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With increased concerns about data protection and privacy over the past several years, and concomitant introduction of regulations restricting access to personally information (PI), archivists in many jurisdictions now must undertake ‘sensitivity reviews’ of archival documents to determine if they can make those documents accessible to researchers. Such reviews are onerous, given increasing volume of records, and complex, due to how difficult it can be for archivists to identify whether records contain personal information (PI) under the provisions of various laws. Despite research into the application of tools and techniques to automate sensitivity reviews, effective solutions remain elusive. Not yet explored as a solution to the challenge of enabling access to archival holdings subject to privacy restrictions is the application of privacy enhancing technologies (PETs) - a class of emerging technologies that rest on the assumption that a body of documents is confidential or private and must remain so. While seemingly being counter-intuitive to apply PETs to making archives more accessible, we argue that PETs could provide an opportunity to protect PI in archival holdings whilst still enabling research on those holdings. In this paper, to lay a foundation for archival experimentation with use of PETs, we contribute an overview of these technologies based on a scoping review and discuss possible use cases and future research directions.
{"title":"Protecting Privacy in Digital Records: The Potential of Privacy Enhancing Technologies","authors":"Victoria L. Lemieux, John Werner","doi":"10.1145/3633477","DOIUrl":"https://doi.org/10.1145/3633477","url":null,"abstract":"<p>With increased concerns about data protection and privacy over the past several years, and concomitant introduction of regulations restricting access to personally information (PI), archivists in many jurisdictions now must undertake ‘sensitivity reviews’ of archival documents to determine if they can make those documents accessible to researchers. Such reviews are onerous, given increasing volume of records, and complex, due to how difficult it can be for archivists to identify whether records contain personal information (PI) under the provisions of various laws. Despite research into the application of tools and techniques to automate sensitivity reviews, effective solutions remain elusive. Not yet explored as a solution to the challenge of enabling access to archival holdings subject to privacy restrictions is the application of privacy enhancing technologies (PETs) - a class of emerging technologies that rest on the assumption that a body of documents is confidential or private and must remain so. While seemingly being counter-intuitive to apply PETs to making archives more accessible, we argue that PETs could provide an opportunity to protect PI in archival holdings whilst still enabling research on those holdings. In this paper, to lay a foundation for archival experimentation with use of PETs, we contribute an overview of these technologies based on a scoping review and discuss possible use cases and future research directions.</p>","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"38 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138540491","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this article, we present and discuss a user-study prototype, developed for Bakkehuset historic house museum in Copenhagen. We examine how the prototype - a digital sound installation - can expand visitors’ experiences of the house and offer encounters with immaterial cultural heritage. Historic house museums often hold back on utilizing digital communication tools inside the houses, since a central purpose of this type of museum is to preserve an original environment. Digital communication tools however hold great potential for facilitating rich encounters with cultural heritage and in particular with the immaterial aspects of museum collections and their histories. In this article we present our design steps and choices, aiming at subtly and seamlessly adding a digital dimension to a historic house. Based on qualitative interviews, we evaluate how the sound installation at Bakkehuset is sensed, interpreted, and used by visitors as part of their museum experience. In turn, we shed light on the historic house museum as a distinct design context for designing hybrid visitor experiences and point to the potentials of digital communication tools in this context.
{"title":"Subtle Sound Design: Designing for Experience Blend in a Historic House Museum","authors":"Mia F. Yates, Anders S. Løvlie","doi":"10.1145/3633476","DOIUrl":"https://doi.org/10.1145/3633476","url":null,"abstract":"<p>In this article, we present and discuss a user-study prototype, developed for Bakkehuset historic house museum in Copenhagen. We examine how the prototype - a digital sound installation - can expand visitors’ experiences of the house and offer encounters with immaterial cultural heritage. Historic house museums often hold back on utilizing digital communication tools inside the houses, since a central purpose of this type of museum is to preserve an original environment. Digital communication tools however hold great potential for facilitating rich encounters with cultural heritage and in particular with the immaterial aspects of museum collections and their histories. In this article we present our design steps and choices, aiming at subtly and seamlessly adding a digital dimension to a historic house. Based on qualitative interviews, we evaluate how the sound installation at Bakkehuset is sensed, interpreted, and used by visitors as part of their museum experience. In turn, we shed light on the historic house museum as a distinct design context for designing hybrid visitor experiences and point to the potentials of digital communication tools in this context.</p>","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"37 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138540470","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sasithorn Rattanarungrot, Kosin Kalarat, Martin White, Kannattha Chaisriya
Manora or Nora has been added to the heritage list of UNESCO as Thailand's intangible cultural heritage since 2021. This performance has been presented as a regional form of entertainment and traditionally passed down through generations within certain families. Nowadays, young people are able to learn and practice Manora instructed by their masters or their successors. However, there is currently no digital content of Manora that could support awareness and enrichment of cultural safeguarding, so this article proposes the design and development of a mobile 3D game representing Manora, which is Southern Thai dance-drama performing art. The aim of this game is to increase citizens’ engagement with their intangible cultural heritage and preservation by delivering novel digital heritage content specifically designed for a role-playing mobile game called Nara. The Nara game is used to enhance the perception of Manora dance and provide new experience of playing a culture-based game through costume and its material items. The game design process takes Manora's significant identities including the costume, dance postures, and music along with players’ requirements to create a game player character, non-player characters, storytelling, challenges, and level design. The development of the game is done on Unity game engine which provides a mobile 3D game platform to integrate all the designed content and generate the interactive Nara game. The experimentation results conclude that utilizing the unique identity of Manora in gameplay successfully enhances the audience's experience and recognition. This result indicates the effective use of creativity and interactive multimedia technology to create attractive content and enjoyment that can encourage intangible cultural heritage preservation in an innovative manner.
{"title":"Preserving Southern Thai Traditional Manora Dance Through Mobile Role-Playing Game Technology","authors":"Sasithorn Rattanarungrot, Kosin Kalarat, Martin White, Kannattha Chaisriya","doi":"10.1145/3631122","DOIUrl":"https://doi.org/10.1145/3631122","url":null,"abstract":"Manora or Nora has been added to the heritage list of UNESCO as Thailand's intangible cultural heritage since 2021. This performance has been presented as a regional form of entertainment and traditionally passed down through generations within certain families. Nowadays, young people are able to learn and practice Manora instructed by their masters or their successors. However, there is currently no digital content of Manora that could support awareness and enrichment of cultural safeguarding, so this article proposes the design and development of a mobile 3D game representing Manora, which is Southern Thai dance-drama performing art. The aim of this game is to increase citizens’ engagement with their intangible cultural heritage and preservation by delivering novel digital heritage content specifically designed for a role-playing mobile game called Nara. The Nara game is used to enhance the perception of Manora dance and provide new experience of playing a culture-based game through costume and its material items. The game design process takes Manora's significant identities including the costume, dance postures, and music along with players’ requirements to create a game player character, non-player characters, storytelling, challenges, and level design. The development of the game is done on Unity game engine which provides a mobile 3D game platform to integrate all the designed content and generate the interactive Nara game. The experimentation results conclude that utilizing the unique identity of Manora in gameplay successfully enhances the audience's experience and recognition. This result indicates the effective use of creativity and interactive multimedia technology to create attractive content and enjoyment that can encourage intangible cultural heritage preservation in an innovative manner.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"242 2 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138540500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Many Western-style paintings by Japanese artists in the early 1900s, though maintaining a unique quality, were greatly inspired by the works of Western artists. In this paper, we employ machine learning to identify relationships and classify the works of Japanese and Western artists. The relationships are of significant interest to numerous art historians, as they can reveal how Western art was introduced to Japan. Historically, art historians have manually annotated these correspondences, which is a time-consuming and labor-intensive process. In this paper, we introduce a new method for finding correspondences between related artworks by comparing their overall outline information. This technique is based on Siamese neural networks (SNNs) and a self-supervised learning approach. Additionally, we have compiled a dataset of illustrations from Japanese artists such as Seiki Kuroda and Western artists such as Raphaël Collin, complete with correspondence annotations. On the other hand, to exhibit the unique quality of works by Japanese artists, we demonstrate that machine learning can classify between artworks created by Japanese artists and those created by Western artists.
{"title":"Identifying Relationships and Classifying Western-Style Paintings: Machine Learning Approaches for Artworks by Western Artists and Meiji-Era Japanese Artists","authors":"Phongtharin Vinayavekhin, Banphatree Khomkham, Vorapong Suppakitpaisarn, Phillippe Codognet, Torahiko Terada, Atsushi Miura","doi":"10.1145/3631136","DOIUrl":"https://doi.org/10.1145/3631136","url":null,"abstract":"Many Western-style paintings by Japanese artists in the early 1900s, though maintaining a unique quality, were greatly inspired by the works of Western artists. In this paper, we employ machine learning to identify relationships and classify the works of Japanese and Western artists. The relationships are of significant interest to numerous art historians, as they can reveal how Western art was introduced to Japan. Historically, art historians have manually annotated these correspondences, which is a time-consuming and labor-intensive process. In this paper, we introduce a new method for finding correspondences between related artworks by comparing their overall outline information. This technique is based on Siamese neural networks (SNNs) and a self-supervised learning approach. Additionally, we have compiled a dataset of illustrations from Japanese artists such as Seiki Kuroda and Western artists such as Raphaël Collin, complete with correspondence annotations. On the other hand, to exhibit the unique quality of works by Japanese artists, we demonstrate that machine learning can classify between artworks created by Japanese artists and those created by Western artists.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136346987","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The need to preserve and care for cultural heritage makes information technology a tool for finding, documenting, describing, and visualizing important historical and archaeological sites. This article offers an example of a demo platform providing users with a unique immersive experience in a virtual clothing museum environment. The proposed concept combines such types of media resources as audio, video, text, and 3D models of the virtual environment. The augmented reality elements perform the function of additional informing the users about certain facts about the exhibits. The methodology of platform development and the application's architecture are described. The creation of the platform is intended to achieve the objectives set by VR/AR technologies, namely, to provide a wide range of opportunities for interaction of the user with the objects of cultural heritage. The peculiarity of the application is the possibility to choose between exploring the objects of cultural heritage on one's own or following a programmed path with the help of an audio guide. The proposed solution can be used for stationary expositions of art objects, historical or archaeological relics, and objects of national economy and creativity. Testing of the demo version showed a rather high degree of user satisfaction (89% of respondents reported a positive experience). Users noted the atmospheric and immersive experience of virtual reality. The developed platform can perform educational and popularization functions.
{"title":"Individually Integrated Virtual/Augmented Reality Environment for Interactive Perception of Cultural Heritage","authors":"Jia Hu","doi":"10.1145/3631145","DOIUrl":"https://doi.org/10.1145/3631145","url":null,"abstract":"The need to preserve and care for cultural heritage makes information technology a tool for finding, documenting, describing, and visualizing important historical and archaeological sites. This article offers an example of a demo platform providing users with a unique immersive experience in a virtual clothing museum environment. The proposed concept combines such types of media resources as audio, video, text, and 3D models of the virtual environment. The augmented reality elements perform the function of additional informing the users about certain facts about the exhibits. The methodology of platform development and the application's architecture are described. The creation of the platform is intended to achieve the objectives set by VR/AR technologies, namely, to provide a wide range of opportunities for interaction of the user with the objects of cultural heritage. The peculiarity of the application is the possibility to choose between exploring the objects of cultural heritage on one's own or following a programmed path with the help of an audio guide. The proposed solution can be used for stationary expositions of art objects, historical or archaeological relics, and objects of national economy and creativity. Testing of the demo version showed a rather high degree of user satisfaction (89% of respondents reported a positive experience). Users noted the atmospheric and immersive experience of virtual reality. The developed platform can perform educational and popularization functions.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"106 11","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135136411","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study examines the longstanding need and challenge of providing contextual analysis of historical images stored in digital visual archives and the accessibility of retrieving contextual information from these historical archives. Contextual analysis is essential for disciplines such as history and art history, as it allows for the contextualization of artwork and historical sources with historical narratives, which in turn enhances understanding of the artistic or political expression in the contents of cultural products. To address this challenge, a novel approach is proposed utilizing computer vision to trace the circulation and dissemination of historical photographs in their original contexts. This method involves first using YOLO v7 to crop historical images from pictorial magazines, then training machine learning models on the cropped printed images plus another large dataset of original historical photographs, and comparing the similarity of images between the datasets of printed images and original photographs. To ensure accuracy of image similarities between the two subsets with distinct image qualities, an ensemble of three machine learning models—Vision Transformer, EfficientNetv2, and Swin Transformer—was developed. Through this system, contexts in the circulation of historical photographs were discovered and new insights regarding the editing strategies of propaganda magazines in East Asia during WWII were uncovered. These outcomes offer supporting evidence for previous research in the history and art historical disciplines, and demonstrate the potential of computer vision for uncovering new information from digital visual archives. Our model achieves a 77.8% top-15 retrieval accuracy on our evaluation dataset.
{"title":"Uncovering and Tracing Historical Image Contexts: Image Retrieval in Visual Archives via Computer Vision","authors":"Lin Du, Brandon Le, Edouardo Honig","doi":"10.1145/3631129","DOIUrl":"https://doi.org/10.1145/3631129","url":null,"abstract":"This study examines the longstanding need and challenge of providing contextual analysis of historical images stored in digital visual archives and the accessibility of retrieving contextual information from these historical archives. Contextual analysis is essential for disciplines such as history and art history, as it allows for the contextualization of artwork and historical sources with historical narratives, which in turn enhances understanding of the artistic or political expression in the contents of cultural products. To address this challenge, a novel approach is proposed utilizing computer vision to trace the circulation and dissemination of historical photographs in their original contexts. This method involves first using YOLO v7 to crop historical images from pictorial magazines, then training machine learning models on the cropped printed images plus another large dataset of original historical photographs, and comparing the similarity of images between the datasets of printed images and original photographs. To ensure accuracy of image similarities between the two subsets with distinct image qualities, an ensemble of three machine learning models—Vision Transformer, EfficientNetv2, and Swin Transformer—was developed. Through this system, contexts in the circulation of historical photographs were discovered and new insights regarding the editing strategies of propaganda magazines in East Asia during WWII were uncovered. These outcomes offer supporting evidence for previous research in the history and art historical disciplines, and demonstrate the potential of computer vision for uncovering new information from digital visual archives. Our model achieves a 77.8% top-15 retrieval accuracy on our evaluation dataset.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":" 7","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135286134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}