Migration of Batch Processing Systems in Financial Sectors to Near Real-Time Processing

Abstract

- Technology has evolved and has become part of people’s lives today. Information systems (IS) are now embraced in all spheres of management. Essentially, this is because of its efficiency and reliability in different fields. Knowledge of IS has enabled the control of advanced sectors (1). IS helps distinguish raw and factual data, which are helpful to any firm. A company must always follow a specific protocol while handling these transactions, whether placing orders for a customer or processing many invoices. The two most popular methods are batch and real-time processing. Batch processing is the procedure to process a large volume of data all at once whereas real time is the procedure to process data instantaneously record by record usually in a matter of seconds or milliseconds. They both solve different needs in financial sector and the industry chose one vs other depending on the criticality and complexity of the need, users of the outcome (internal vs external) and overall customer satisfaction index. In addition to above two methods, near real time processing is the process of being able to almost instantaneously analyze data that is streaming from one device to another. The financial sector is looking for solution to migrate batch processing system to near real time systems using streaming solutions like Apache Kafka and AWS Kinesis and re-use real time systems wherever possible for better customer experience