GPartition-store: A multi-group collaborative parallel data storage mechanism for permissioned blockchain sharding

Abstract

The problem of insufficient storage space caused by the full-replication mechanism, which is commonly employed in existing blockchains, poses an obstacle to system scalability. Moreover, existing storage sharding mechanisms are confronted with the risk of data tampering by reason of the existence of Byzantine nodes. To address the above problems, the storage partition mechanisms, integrating Erasure Coding with Byzantine Fault Tolerance consensus protocol, are proposed such as BFT-Store and PartitionChain. While promising, these solutions still encounter three significant challenges. First, the substantial computational complexity associated with encoding during data storage and decoding during data recovery will impede the efficiency (e.g., latency and throughput) of the permissioned blockchain. Second, the signature schemes employed for verifying the completeness and correctness of encoded data on each node lead to massive communication over the network, thereby further limiting the system efficiency. Third, the process of system re-initialization, which necessitates the participation of all nodes, degrades the system stability. This paper proposes a Multi-group Collaborative Parallel Data Storage Mechanism for Permissioned Blockchain Sharding called GPartition-Store to alleviate the above problems, where the nodes are divided into multiple Storage Groups (SGs). First, the original block is partitioned into $g$ sub-blocks (assuming $g$ is the number of SGs), with each sub-block being further partitioned and encoded into smaller encoded-blocks or recovered by decoding in parallel across all SGs. Hence, the computational complexity of coding (i.e., encoding and decoding) can be decreased by about $g^2$ and $g^3$ times respectively. Second, the bloom filter is utilized to generate the verification proofs of the sub-blocks and encoded-block sets, which simultaneously avoids the heavy amount of transmitted messages, while liberating the requirement for dependence on any trusted third party. Third, the re-initialization process is launched exclusively within a specific SG when a node joins/quits the system or a single crashed node needs repair, thereby enhancing the system stability. Compared with the full-replication mechanism, BFT-Store and PartitionChain, the experimental results illustrate that GPartition-Store can improve the scalability, efficiency and stability of the dynamic blockchain network while maintaining the availability of the blocks.