Alessio P Buccino, Olivier Winter, David Bryant, David Feng, Karel Svoboda, Joshua H Siegle
{"title":"Compression strategies for large-scale electrophysiology data.","authors":"Alessio P Buccino, Olivier Winter, David Bryant, David Feng, Karel Svoboda, Joshua H Siegle","doi":"10.1088/1741-2552/acf5a4","DOIUrl":null,"url":null,"abstract":"<p><p><i>Objective.</i>With the rapid adoption of high-density electrode arrays for recording neural activity, electrophysiology data volumes within labs and across the field are growing at unprecedented rates. For example, a one-hour recording with a 384-channel Neuropixels probe generates over 80 GB of raw data. These large data volumes carry a high cost, especially if researchers plan to store and analyze their data in the cloud. Thus, there is a pressing need for strategies that can reduce the data footprint of each experiment.<i>Approach.</i>Here, we establish a set of benchmarks for comparing the performance of various compression algorithms on experimental and simulated recordings from Neuropixels 1.0 (NP1) and 2.0 (NP2) probes.<i>Main results.</i>For lossless compression, audio codecs (FLACandWavPack) achieve compression ratios (CRs) 6% higher for NP1 and 10% higher for NP2 than the best general-purpose codecs, at the expense of decompression speed. For lossy compression, theWavPackalgorithm in 'hybrid mode' increases the CR from 3.59 to 7.08 for NP1 and from 2.27 to 7.04 for NP2 (compressed file size of ∼14% for both types of probes), without adverse effects on spike sorting accuracy or spike waveforms.<i>Significance.</i>Along with the tools we have developed to make compression easier to deploy, these results should encourage all electrophysiologists to apply compression as part of their standard analysis workflows.</p>","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":3.7000,"publicationDate":"2023-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of neural engineering","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1088/1741-2552/acf5a4","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0
Abstract
Objective.With the rapid adoption of high-density electrode arrays for recording neural activity, electrophysiology data volumes within labs and across the field are growing at unprecedented rates. For example, a one-hour recording with a 384-channel Neuropixels probe generates over 80 GB of raw data. These large data volumes carry a high cost, especially if researchers plan to store and analyze their data in the cloud. Thus, there is a pressing need for strategies that can reduce the data footprint of each experiment.Approach.Here, we establish a set of benchmarks for comparing the performance of various compression algorithms on experimental and simulated recordings from Neuropixels 1.0 (NP1) and 2.0 (NP2) probes.Main results.For lossless compression, audio codecs (FLACandWavPack) achieve compression ratios (CRs) 6% higher for NP1 and 10% higher for NP2 than the best general-purpose codecs, at the expense of decompression speed. For lossy compression, theWavPackalgorithm in 'hybrid mode' increases the CR from 3.59 to 7.08 for NP1 and from 2.27 to 7.04 for NP2 (compressed file size of ∼14% for both types of probes), without adverse effects on spike sorting accuracy or spike waveforms.Significance.Along with the tools we have developed to make compression easier to deploy, these results should encourage all electrophysiologists to apply compression as part of their standard analysis workflows.
期刊介绍:
The goal of Journal of Neural Engineering (JNE) is to act as a forum for the interdisciplinary field of neural engineering where neuroscientists, neurobiologists and engineers can publish their work in one periodical that bridges the gap between neuroscience and engineering. The journal publishes articles in the field of neural engineering at the molecular, cellular and systems levels.
The scope of the journal encompasses experimental, computational, theoretical, clinical and applied aspects of: Innovative neurotechnology; Brain-machine (computer) interface; Neural interfacing; Bioelectronic medicines; Neuromodulation; Neural prostheses; Neural control; Neuro-rehabilitation; Neurorobotics; Optical neural engineering; Neural circuits: artificial & biological; Neuromorphic engineering; Neural tissue regeneration; Neural signal processing; Theoretical and computational neuroscience; Systems neuroscience; Translational neuroscience; Neuroimaging.