Streaming Graph Ingestion with Resource-Aware Buffering and Graph Compression

Abstract

Ingesting high-speed streaming data from social media into a graph database must overcome three problems – 1) the data can be really bursty, 2) the data must be transformed into a graph and 3) the graph database may not be able to ingest high-burst, high-velocity data. We have developed an adaptive buffering mechanism and a graph compression technique that effectively mitigate the problem.