Study of fast data access based on hierarchical storage for EAST Tokamak

Abstract

The Experimental Advanced Superconducting Tokamak is a large fusion research device which aims at long-term and high parameters plasma operation. A distributed and continuous data acquisition system based on PXI/PXI Express technology has been developed for EAST. At present the system has more than 50 data acquisition units and more than 2000 raw channels, the maximum data throughput is about 5GBytes/s. How to access the large amount of data as fast as possible becomes a big problem. So we planned to design a special hierarchical data storage system for EAST. The system is composed of 4 storage tiers. The tier1 is based on PCIe SSD storage which provide the data access cache. The tier2 is the local SAS raid on MDSplus server cluster for real-time data collection from the data acquisition units. The tier3 is Lustre storage which stored all the current campaign data and can provide fast parallel data access for data processing. The tier4 is NAS storage for historic data achieving. A schedule program is designed to control the data flow between tier1 and tier2/tier3/tier4. The system has been designed and will be adopted in the next campaign and the system details will be given in the paper.