Knowledge extraction from diagram and text for media integration

We introduce a new framework for knowledge extraction from written texts and diagrams and utilization of the obtained knowledge. We use a combination of multiple media for effective communication. To realize this mechanism as a hyper-medium, we propose a noble idea of media integration. First, the correlation between the two media is analyzed to automatically interpret the semantic structure of both media. The integrated hypermedia obtained as a result of the correspondence and the semantic interpretation is quite useful to derive knowledge about topics being described. To demonstrate the usefulness of the integrated media, we have constructed a prototype system for flexible explanation generation. Various kinds of explanation can be generated in our system.