A Real-Time Text Analysis System

Abstract

We live in an age of information overload. Manual information processing is increasingly overwhelmed with the enormous amount of information created by the explosive growth of news portals and online social networks. Such a situation calls for an automatic system that can support the process of handling, analyzing, and filtering information, especially information from online sources. In this work, we proposed a text analysis system that automatically collects, extracts, and analyses information from public-source-text documents such as news portals and social media networks. The proposed system can handle both long and short-text documents. It also has real-time features and is not restricted by any input data domain. The system can be used in different domains, such as scientific research, marketing, and security-related domains. Moreover, the system is engineered in modules and is flexible. Each module is an independent micro-service that can be used as a separate standalone application. The system is also extensible since new modules can be added easily. Index Terms—text analysis system, data mining, natural language processing