A novel approach for open domain event schema discovery from twitter

Abstract

Open domain event extraction is a recently-introduced type of event extraction that extracts, aggregate and categorize important events. This is done without any domain specific guidance such as special training data or extraction rules. Because OEE is domain-independent, it helps the final users managing the unstructured data in an easy way. OEE help users creating complex queries or finding a new domain when they have an unknown structure of the explored events. We can help the user by generating a simplified relational schema that describes the extracted events in any given domain. For systems that extract events within a narrow domain, the schema is easily specified in advance. While, the events and types extracted with OEE do not fit with full schema information: we can't know in advance what schema is appropriate for each discovered type. In this paper, we introduce a novel approach of schema discovery based on probabilistic generative models especially LinkLDA for open-domain event extraction. This approach aims to develop an algorithm to automatically derive high quality smart schemas from the extracted events. To evaluate the quality of our results, we will carry out our experiments on a set of events extracted from twitter, the most up-to-date stream of current events.