Drawing on historical and contemporary examples, this paper discusses relationships between humans and machines in creative co-production with generative AI. It distinguishes between generative systems as augmenting tools and autonomous collaborators, and adopts the term communion to describe the close degree of exchange that is possible when working with these systems. It draws on historical examples of surrealist and AI artworks, and more recent examples made with Large Language Models and GAN-based generation. It concludes with a discussion of the primacy of text as an entry point to the possibilities of creative AI.
{"title":"Communing with Creative AI","authors":"R. Twomey","doi":"10.1145/3597633","DOIUrl":"https://doi.org/10.1145/3597633","url":null,"abstract":"Drawing on historical and contemporary examples, this paper discusses relationships between humans and machines in creative co-production with generative AI. It distinguishes between generative systems as augmenting tools and autonomous collaborators, and adopts the term communion to describe the close degree of exchange that is possible when working with these systems. It draws on historical examples of surrealist and AI artworks, and more recent examples made with Large Language Models and GAN-based generation. It concludes with a discussion of the primacy of text as an entry point to the possibilities of creative AI.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"6 1","pages":"1 - 7"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41770341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chinese painting incorporated with poems is famous for its distinct characteristics and unspeakable moods. Ink lines are used to depict scenes in paintings while concise words are used to describe scenes and events in poems, making for a beautiful yet enigmatic art style. Such an art style requires viewers to patiently examine the details and attempt to understand the meaning, which creates a barrier for a wide audience to relish and disseminate it. The condition can be effectively improved with immersive and interactive animation techniques. In this paper, we present the design and production of an interactive 3D animation entitled "Illustrating Ten Poems" based on the famous Chinese painting with the same name. Compared with viewing the original painting, the animation provides immersive scene reconstruction in Chinese painting style, interactive characters and visual-audio experience. Rather than complete three-dimensional reconstruction, we attempted some innovative methods which proved to be effective and efficient in Chinese painting style artwork production.
{"title":"Immersive experience of Chinese poems and paintings","authors":"Chong Cao, Lingfei Yang, Xukun Shen","doi":"10.1145/3597621","DOIUrl":"https://doi.org/10.1145/3597621","url":null,"abstract":"Chinese painting incorporated with poems is famous for its distinct characteristics and unspeakable moods. Ink lines are used to depict scenes in paintings while concise words are used to describe scenes and events in poems, making for a beautiful yet enigmatic art style. Such an art style requires viewers to patiently examine the details and attempt to understand the meaning, which creates a barrier for a wide audience to relish and disseminate it. The condition can be effectively improved with immersive and interactive animation techniques. In this paper, we present the design and production of an interactive 3D animation entitled \"Illustrating Ten Poems\" based on the famous Chinese painting with the same name. Compared with viewing the original painting, the animation provides immersive scene reconstruction in Chinese painting style, interactive characters and visual-audio experience. Rather than complete three-dimensional reconstruction, we attempted some innovative methods which proved to be effective and efficient in Chinese painting style artwork production.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"6 1","pages":"1 - 6"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42447888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yurui Xie, G. Barbareschi, Ayesha Nabila, K. Kunze, M. Inakage
Wheelchair dance is an important form of disability art that is still subject to significant levels of ableism and art exclusion. Wheelchair dancers face challenges finding teachers and choreographers who can accommodate their needs, documenting and sharing choreographies that suit their body shapes and their assistive technologies. In turn, this hinders their ability to share creative expressions. Accessible resources and communication tools could help address these challenges. The goal of this research is the development of a visualization system grounded on Laban Movement Analysis (LMA) that notates movement quality while opening new horizons on perceptions of disabled bodies and the artistic legitimacy of wheelchair dance. The system uses video to identify the body landmarks of the dancer and wheelchair and extracts key features to create visualizations of expressive qualities from LMA basic effort. The current evaluation includes a pilot study with the general public and an online questionnaire targeting professionals to gain feedback supporting practical implementation and real-world deployment. Results from the general public evaluation showed that the visualization was effective in conveying basic effort movement qualities even to a novice audience. Expert consulted via questionnaire stated that the tool could be employed for reflective evaluation, as well as performance augmentation. The LMA visualization tool can support the artistic legitimization of wheelchair dancing through education, communication, performance, and documentation.
{"title":"Movement Quality Visualization for Wheelchair Dance","authors":"Yurui Xie, G. Barbareschi, Ayesha Nabila, K. Kunze, M. Inakage","doi":"10.1145/3597628","DOIUrl":"https://doi.org/10.1145/3597628","url":null,"abstract":"Wheelchair dance is an important form of disability art that is still subject to significant levels of ableism and art exclusion. Wheelchair dancers face challenges finding teachers and choreographers who can accommodate their needs, documenting and sharing choreographies that suit their body shapes and their assistive technologies. In turn, this hinders their ability to share creative expressions. Accessible resources and communication tools could help address these challenges. The goal of this research is the development of a visualization system grounded on Laban Movement Analysis (LMA) that notates movement quality while opening new horizons on perceptions of disabled bodies and the artistic legitimacy of wheelchair dance. The system uses video to identify the body landmarks of the dancer and wheelchair and extracts key features to create visualizations of expressive qualities from LMA basic effort. The current evaluation includes a pilot study with the general public and an online questionnaire targeting professionals to gain feedback supporting practical implementation and real-world deployment. Results from the general public evaluation showed that the visualization was effective in conveying basic effort movement qualities even to a novice audience. Expert consulted via questionnaire stated that the tool could be employed for reflective evaluation, as well as performance augmentation. The LMA visualization tool can support the artistic legitimization of wheelchair dancing through education, communication, performance, and documentation.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 12"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47735473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We developed a new musical instrument by equipping juggling balls with accelerometers, gyroscopes, and WiFi sensors. The system measures acceleration and rotation, allowing us to distinguish between ball movement and static states. We can determine the direction of each throw based on the acceleration data. A movement-driven instrument animates the virtual audio source position based on the throw direction. A different synthesizer is assigned to each ball, resulting in a synesthetic experience. The system integrates the juggling patterns simulator with OSC and MIDI API, allowing real-time rhythm generation and composition.
{"title":"Sonification of a juggling performance using spatial audio","authors":"Vojtěch Leischner, Pavel Husa","doi":"10.1145/3597619","DOIUrl":"https://doi.org/10.1145/3597619","url":null,"abstract":"We developed a new musical instrument by equipping juggling balls with accelerometers, gyroscopes, and WiFi sensors. The system measures acceleration and rotation, allowing us to distinguish between ball movement and static states. We can determine the direction of each throw based on the acceleration data. A movement-driven instrument animates the virtual audio source position based on the throw direction. A different synthesizer is assigned to each ball, resulting in a synesthetic experience. The system integrates the juggling patterns simulator with OSC and MIDI API, allowing real-time rhythm generation and composition.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 6"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43223805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Inspired by storytelling ideas in surrealist creative productions and experimental films, we created a brain-computer interface (BCI) designed to offer users an exercise of the imagination. While many previous BCIs have used simple visual interfaces, we wanted to understand if an open-ended, story-based BCI experience could be helpful for allowing users to effectively interact with the story using their brain activity. We employed experimental storytelling techniques to prompt users to explore new thought patterns and ideas for changing brain activity. In our BCI, users were able to change the color of the main character in the story using their thoughts. We found that many users were able to change brain activity and enjoyed the BCI experience, while others were primarily frustrated. Based on our results, we offer ideas for future contributions to the development of engaging, story-based BCI applications.
{"title":"Using Experimental Filmmaking to Create an Engaging Brain-Computer Interface","authors":"Claudia Krogmeier, E. Bravo, Christos Mousas","doi":"10.1145/3597624","DOIUrl":"https://doi.org/10.1145/3597624","url":null,"abstract":"Inspired by storytelling ideas in surrealist creative productions and experimental films, we created a brain-computer interface (BCI) designed to offer users an exercise of the imagination. While many previous BCIs have used simple visual interfaces, we wanted to understand if an open-ended, story-based BCI experience could be helpful for allowing users to effectively interact with the story using their brain activity. We employed experimental storytelling techniques to prompt users to explore new thought patterns and ideas for changing brain activity. In our BCI, users were able to change the color of the main character in the story using their thoughts. We found that many users were able to change brain activity and enjoyed the BCI experience, while others were primarily frustrated. Based on our results, we offer ideas for future contributions to the development of engaging, story-based BCI applications.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 10"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45288979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We discuss Beauty, a machine-microbial artwork, currently in development by Phylum, an experimental research collective specializing in cultural production informed by the intersections of science, technology and the arts (and of which the authors are founding members). The work is based upon an artificial intelligence agent that uses deep reinforcement learning to interact with and alter cultures of pattern-forming social bacteria in order to make them more aesthetically pleasing.
{"title":"Beauty","authors":"J. DiBlasi, Carlos Castellanos, Bello Bello","doi":"10.1145/3597626","DOIUrl":"https://doi.org/10.1145/3597626","url":null,"abstract":"We discuss Beauty, a machine-microbial artwork, currently in development by Phylum, an experimental research collective specializing in cultural production informed by the intersections of science, technology and the arts (and of which the authors are founding members). The work is based upon an artificial intelligence agent that uses deep reinforcement learning to interact with and alter cultures of pattern-forming social bacteria in order to make them more aesthetically pleasing.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 10"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48678661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we introduce the interactive installation Aquaterrestrial Recolonization as an artistic response to rising climate change concerns and trigger the audience's thoughts and discussions. In the project, we are recording and archiving the current state of the dead coral reefs and feeding artificial intelligence with that data to be able to visualize the virtual 3D dimensional recolonized coral reefs, relieved from human devastation and behaviour from the past and conduct changes in the future. We aim to preserve the current state of the Bahamian coral reefs and use artificial intelligence to return to the planet parts mankind has destroyed and affected the future with necessary human behavioural changes. Our AI Environmentalist is designed to be a chronologist, provocateur, and uncompromised planet Earth's protector.
{"title":"Aquaterrestrial Recolonization","authors":"Predrag K. Nikolić, Giacomo Bertin, Kishan Munroe","doi":"10.1145/3597630","DOIUrl":"https://doi.org/10.1145/3597630","url":null,"abstract":"In this paper, we introduce the interactive installation Aquaterrestrial Recolonization as an artistic response to rising climate change concerns and trigger the audience's thoughts and discussions. In the project, we are recording and archiving the current state of the dead coral reefs and feeding artificial intelligence with that data to be able to visualize the virtual 3D dimensional recolonized coral reefs, relieved from human devastation and behaviour from the past and conduct changes in the future. We aim to preserve the current state of the Bahamian coral reefs and use artificial intelligence to return to the planet parts mankind has destroyed and affected the future with necessary human behavioural changes. Our AI Environmentalist is designed to be a chronologist, provocateur, and uncompromised planet Earth's protector.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 9"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48082523","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Artificial intelligence (AI) is redefining roads and skyways with advanced mobility technologies such as autonomous driving and Urban Air Mobility (UAM), suggesting a new paradigm for human mobility. This project sheds new light on drone mobility technology in terms of environmental aesthetics, focusing on global ecological issues. The Kestrel Drone is proposed based on a speculative scenario where "birds" become front-line users of drone technology and share their air routes with drones. At the exhibition hall, five Kestrel Drones were equipped with bird-mimicking wings and an AI tracker. The drones simulated a guiding bird for resident birds in a city. This project aims to evolve AI into a technology that has various parallel values that can coexist with non-human values, thus expanding the possibilities of technology.
{"title":"The Kestrel Drone","authors":"J. Oh","doi":"10.1145/3597629","DOIUrl":"https://doi.org/10.1145/3597629","url":null,"abstract":"Artificial intelligence (AI) is redefining roads and skyways with advanced mobility technologies such as autonomous driving and Urban Air Mobility (UAM), suggesting a new paradigm for human mobility. This project sheds new light on drone mobility technology in terms of environmental aesthetics, focusing on global ecological issues. The Kestrel Drone is proposed based on a speculative scenario where \"birds\" become front-line users of drone technology and share their air routes with drones. At the exhibition hall, five Kestrel Drones were equipped with bird-mimicking wings and an AI tracker. The drones simulated a guiding bird for resident birds in a city. This project aims to evolve AI into a technology that has various parallel values that can coexist with non-human values, thus expanding the possibilities of technology.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"6 1","pages":"1 - 10"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42069870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bodylab is a virtual reality system for creating full-body virtual reality avatars. It provides anatomically realistic mannequins as canvases, which can be painted and textured, sculpted and deformed, adorned with objects and animated with particle effects. The artist can work from a first or third person perspective, and variety of interaction modalities are made available. To assess the system, we invite costume designers and theatrical artists to provide expert reviews. We also describe the use of Bodylab in a live dance performance. We believe Bodylab to be the first virtual reality tool for in virtuo painting and sculpting of full-body avatars.
{"title":"Bodylab","authors":"Daniel Zeidler, M. Mcginity","doi":"10.1145/3597631","DOIUrl":"https://doi.org/10.1145/3597631","url":null,"abstract":"Bodylab is a virtual reality system for creating full-body virtual reality avatars. It provides anatomically realistic mannequins as canvases, which can be painted and textured, sculpted and deformed, adorned with objects and animated with particle effects. The artist can work from a first or third person perspective, and variety of interaction modalities are made available. To assess the system, we invite costume designers and theatrical artists to provide expert reviews. We also describe the use of Bodylab in a live dance performance. We believe Bodylab to be the first virtual reality tool for in virtuo painting and sculpting of full-body avatars.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"6 1","pages":"1 - 12"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44500839","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Our concept of a digital spirit refers to the agency of technology in the creative process and the potential to use it as an autonomous agent that both influences and participates in the artistic process. By attributing a digital spirit, we are able to have a more collaborative and interactive relationship with technology, where the role of the artist is to guide and direct the design rather than fully control it. In this work, we extend the autonomy of this spirit to allow the machine and the material to interact freely and generate forms independently from our will. In this new approach to digital fabrication, we transform a deterministic mechanical procedure into a creative exploration. Rather than designing traditional machine operations, we propose machine-material operations tailored for a specific material context. As a test case, we show a digital agent that manipulates clay while allowing the material's internal dynamics to play a dominant role in the finalization of the design. Through this machine-material interaction, we are able to generate an abundance of forms and complexity that would be impossible to create by hand or through traditional programming. It exposes an untapped expressive potential of digital tools that is made possible by operating digital machines outside the paradigm of tight control.
{"title":"Fine Motor Skills","authors":"S. Elran, Y. Harel, Amit R. Zoran","doi":"10.1145/3597627","DOIUrl":"https://doi.org/10.1145/3597627","url":null,"abstract":"Our concept of a digital spirit refers to the agency of technology in the creative process and the potential to use it as an autonomous agent that both influences and participates in the artistic process. By attributing a digital spirit, we are able to have a more collaborative and interactive relationship with technology, where the role of the artist is to guide and direct the design rather than fully control it. In this work, we extend the autonomy of this spirit to allow the machine and the material to interact freely and generate forms independently from our will. In this new approach to digital fabrication, we transform a deterministic mechanical procedure into a creative exploration. Rather than designing traditional machine operations, we propose machine-material operations tailored for a specific material context. As a test case, we show a digital agent that manipulates clay while allowing the material's internal dynamics to play a dominant role in the finalization of the design. Through this machine-material interaction, we are able to generate an abundance of forms and complexity that would be impossible to create by hand or through traditional programming. It exposes an untapped expressive potential of digital tools that is made possible by operating digital machines outside the paradigm of tight control.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"6 1","pages":"1 - 9"},"PeriodicalIF":0.0,"publicationDate":"2023-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47401192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：