E. André, Wolfgang Finkler, W. Graf, T. Rist, A. Schauder, W. Wahlster
Due to the growing complexity of information that has to be communicated by current AI systems, there comes an increasing need for building advanced intelligent user interfaces that take advantage of a coordinated combination of different modalities, e.g., natural language, graphics, and animation, to produce situated and user-adaptive presentations. A deeper understanding of the basic principles underlying multimodal communication requires theoretical work on computational models as well as practical work on concrete systems. In this article, we describe the system WIP, an implemented prototype of a knowledge-based presentation system that generates illustrated texts that are customized for the intended audience and situation. We present the architecture of WIP and introduce as its major components the presentation planner, the layout manager, and the generators for text and graphics. To achieve a coherent output with an optimal media mix, the single components have to be interleaved. The interplay of the presentation planner, the text and the graphics generator will be demonstrated by means of a system run. In particular, we show how a text-picture combination containing a crossmodal referring expression is generated by the system.
{"title":"WIP: The Automatic Synthesis of Multimodal Presentations","authors":"E. André, Wolfgang Finkler, W. Graf, T. Rist, A. Schauder, W. Wahlster","doi":"10.22028/D291-24861","DOIUrl":"https://doi.org/10.22028/D291-24861","url":null,"abstract":"Due to the growing complexity of information that has to be communicated by current AI systems, there comes an increasing need for building advanced intelligent user interfaces that take advantage of a coordinated combination of different modalities, e.g., natural language, graphics, and animation, to produce situated and user-adaptive presentations. A deeper understanding of the basic principles underlying multimodal communication requires theoretical work on computational models as well as practical work on concrete systems. In this article, we describe the system WIP, an implemented prototype of a knowledge-based presentation system that generates illustrated texts that are customized for the intended audience and situation. We present the architecture of WIP and introduce as its major components the presentation planner, the layout manager, and the generators for text and graphics. To achieve a coherent output with an optimal media mix, the single components have to be interleaved. The interplay of the presentation planner, the text and the graphics generator will be demonstrated by means of a system run. In particular, we show how a text-picture combination containing a crossmodal referring expression is generated by the system.","PeriodicalId":281243,"journal":{"name":"AAAI Workshop on Intelligent Multimedia Interfaces","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1993-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114697838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Not only the generation of text, but also the generation of multimodal documents can be considered as a sequence of communicative acts which aim to achieve certain goals. For the realization of a system able to automatically generate illustrated documents, a plan-based approach seems adequate. To represent knowledge about how to present information, we have designed presentation strategies which relate to both text and picture production. These strategies are considered as operators of a planning system. However, a conventional hierarchical planner for determining the contents and the rhetorical structure of a document has proven inappropriate to handle the various dependencies between content determination, mode selection and content realization. To overcome these problems, a new planning scheme has been developed that supports data transfer between the content planner and the mode-specific generation components and allows for revising an initial document structure.
{"title":"The Design of Illustrated Documents as a Planning Task","authors":"E. André, T. Rist","doi":"10.22028/D291-24860","DOIUrl":"https://doi.org/10.22028/D291-24860","url":null,"abstract":"Not only the generation of text, but also the generation of multimodal documents can be considered as a sequence of communicative acts which aim to achieve certain goals. For the realization of a system able to automatically generate illustrated documents, a plan-based approach seems adequate. To represent knowledge about how to present information, we have designed presentation strategies which relate to both text and picture production. These strategies are considered as operators of a planning system. However, a conventional hierarchical planner for determining the contents and the rhetorical structure of a document has proven inappropriate to handle the various dependencies between content determination, mode selection and content realization. To overcome these problems, a new planning scheme has been developed that supports data transfer between the content planner and the mode-specific generation components and allows for revising an initial document structure.","PeriodicalId":281243,"journal":{"name":"AAAI Workshop on Intelligent Multimedia Interfaces","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1993-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123457502","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abstract : We address one of the problems at the heart of automated multimedia presentation production and interpretation. The media problem can be stated as follows: how does the producer of a presentation determine which information to allocate to which medium, and how does a perceiver recognize the function of each part as displayed in the presentation and integrate them into a coherent whole? What knowledge is used, and what processes? We describe the four major types of knowledge that play a role in the allocation problem as well as interdependencies that hold among them. We discuss two formalisms that can be usedto represent this knowledge and, using examples, describe the kinds of processing required for the media allocation problem. Multimedia presentations, Human-computer interaction, Presentation planning.
{"title":"On the Knowledge Underlying Multimedia Presentations","authors":"Y. Arens, E. Hovy, Mira Vossers","doi":"10.21236/ADA278690","DOIUrl":"https://doi.org/10.21236/ADA278690","url":null,"abstract":"Abstract : We address one of the problems at the heart of automated multimedia presentation production and interpretation. The media problem can be stated as follows: how does the producer of a presentation determine which information to allocate to which medium, and how does a perceiver recognize the function of each part as displayed in the presentation and integrate them into a coherent whole? What knowledge is used, and what processes? We describe the four major types of knowledge that play a role in the allocation problem as well as interdependencies that hold among them. We discuss two formalisms that can be usedto represent this knowledge and, using examples, describe the kinds of processing required for the media allocation problem. Multimedia presentations, Human-computer interaction, Presentation planning.","PeriodicalId":281243,"journal":{"name":"AAAI Workshop on Intelligent Multimedia Interfaces","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1991-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114915852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：