IntentFinder: A system for discovering significant information implicit in large, heterogeneous document collections and computationally mapping social networks and command nodes

Abstract

IntentFinder is a computational method of extracting mutually relevant information from a large collection of narrative data. We describe an approach that takes advantage of a new view of documents as coming from evolving stories. IntentFinder consists of six main components: 1) A document management system 2) A story extraction system 3) A significance determination system 4) A reputation management 5) A lexical-semantic analysis 6) A user interface In addition a method has been found for quantitatively determining the topology and hierarchy of a social subnetwork embedded inside a very noisy self-reorganizing network (e.g., the Internet). All these components will work together to allow analysts to discover and understand events and stories implicit in collections of documents, including newswire, reports, emails and tweets, which would be prohibitively difficult to uncover manually, and ultimately estimating the organizational structure of a social network.