Fast Entropy Coding for ALICE Run 3

Abstract

In LHC Run 3, the upgraded ALICE detector will record Pb-Pb collisions at a rate of 50 kHz using continuous readout. The resulting stream of raw data at 3.5 TB/s has to be processed with a set of lossy and lossless compression and data reduction techniques to a storage data rate of 90 GB/s while preserving relevant data for physics analysis. This contribution presents a custom lossless data compression scheme based on entropy coding as the final component in the data reduction chain which has to compress the data rate from 300 GB/s to 90 GB/s. A flexible, multi-process architecture for the data compression scheme is proposed that seamlessly interfaces with the data reduction algorithms of earlier stages and allows to use parallel processing in order to keep the required firm real-time guarantees of the system. The data processed inside the compression process have a structure that allows the use of an rANS entropy coder with more resource efficient static distribution tables. Extensions to the rANS entropy coder are introduced to efficiently work with these static distribution tables and large but sparse source alphabets consisting of up to 25 Bit per symbol. Preliminary performance results show compliance with the firm real-time requirements while offering close-to-optimal data compression.