Compression strategies for large-scale electrophysiology data.

IF 3.7 3区 医学 Q2 ENGINEERING, BIOMEDICAL Journal of neural engineering Pub Date : 2023-09-18 DOI:10.1088/1741-2552/acf5a4
Alessio P Buccino, Olivier Winter, David Bryant, David Feng, Karel Svoboda, Joshua H Siegle
{"title":"Compression strategies for large-scale electrophysiology data.","authors":"Alessio P Buccino,&nbsp;Olivier Winter,&nbsp;David Bryant,&nbsp;David Feng,&nbsp;Karel Svoboda,&nbsp;Joshua H Siegle","doi":"10.1088/1741-2552/acf5a4","DOIUrl":null,"url":null,"abstract":"<p><p><i>Objective.</i>With the rapid adoption of high-density electrode arrays for recording neural activity, electrophysiology data volumes within labs and across the field are growing at unprecedented rates. For example, a one-hour recording with a 384-channel Neuropixels probe generates over 80 GB of raw data. These large data volumes carry a high cost, especially if researchers plan to store and analyze their data in the cloud. Thus, there is a pressing need for strategies that can reduce the data footprint of each experiment.<i>Approach.</i>Here, we establish a set of benchmarks for comparing the performance of various compression algorithms on experimental and simulated recordings from Neuropixels 1.0 (NP1) and 2.0 (NP2) probes.<i>Main results.</i>For lossless compression, audio codecs (FLACandWavPack) achieve compression ratios (CRs) 6% higher for NP1 and 10% higher for NP2 than the best general-purpose codecs, at the expense of decompression speed. For lossy compression, theWavPackalgorithm in 'hybrid mode' increases the CR from 3.59 to 7.08 for NP1 and from 2.27 to 7.04 for NP2 (compressed file size of ∼14% for both types of probes), without adverse effects on spike sorting accuracy or spike waveforms.<i>Significance.</i>Along with the tools we have developed to make compression easier to deploy, these results should encourage all electrophysiologists to apply compression as part of their standard analysis workflows.</p>","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":"20 5","pages":""},"PeriodicalIF":3.7000,"publicationDate":"2023-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of neural engineering","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1088/1741-2552/acf5a4","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0

Abstract

Objective.With the rapid adoption of high-density electrode arrays for recording neural activity, electrophysiology data volumes within labs and across the field are growing at unprecedented rates. For example, a one-hour recording with a 384-channel Neuropixels probe generates over 80 GB of raw data. These large data volumes carry a high cost, especially if researchers plan to store and analyze their data in the cloud. Thus, there is a pressing need for strategies that can reduce the data footprint of each experiment.Approach.Here, we establish a set of benchmarks for comparing the performance of various compression algorithms on experimental and simulated recordings from Neuropixels 1.0 (NP1) and 2.0 (NP2) probes.Main results.For lossless compression, audio codecs (FLACandWavPack) achieve compression ratios (CRs) 6% higher for NP1 and 10% higher for NP2 than the best general-purpose codecs, at the expense of decompression speed. For lossy compression, theWavPackalgorithm in 'hybrid mode' increases the CR from 3.59 to 7.08 for NP1 and from 2.27 to 7.04 for NP2 (compressed file size of ∼14% for both types of probes), without adverse effects on spike sorting accuracy or spike waveforms.Significance.Along with the tools we have developed to make compression easier to deploy, these results should encourage all electrophysiologists to apply compression as part of their standard analysis workflows.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
大规模电生理数据的压缩策略。
目的:随着高密度电极阵列用于记录神经活动的快速采用,实验室内和整个领域的电生理学数据量正以前所未有的速度增长。例如,使用384通道Neuropixels探针进行一小时的记录,可生成超过80GB的原始数据。这些大数据量的成本很高,尤其是如果研究人员计划在云中存储和分析他们的数据。因此,迫切需要能够减少每个实验的数据足迹的策略。方法。在这里,我们建立了一组基准,用于比较各种压缩算法在Neuropixels 1.0(NP1)和2.0(NP2)探针的实验和模拟记录上的性能。主要结果。对于无损压缩,音频编解码器(FLACandWavPack)以牺牲解压缩速度为代价,实现了NP1比最佳通用编解码器高6%和NP2高10%的压缩比(CR)。对于有损压缩,“混合模式”下的WavPack算法将NP1的CR从3.59增加到7.08,将NP2的CR从2.27增加到7.04(两种类型的探针的压缩文件大小都为~14%),不会对尖峰排序精度或尖峰波形产生不利影响。意义。除了我们开发的使压缩更容易部署的工具外,这些结果应该鼓励所有电生理学家将压缩作为其标准分析工作流程的一部分。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Journal of neural engineering
Journal of neural engineering 工程技术-工程:生物医学
CiteScore
7.80
自引率
12.50%
发文量
319
审稿时长
4.2 months
期刊介绍: The goal of Journal of Neural Engineering (JNE) is to act as a forum for the interdisciplinary field of neural engineering where neuroscientists, neurobiologists and engineers can publish their work in one periodical that bridges the gap between neuroscience and engineering. The journal publishes articles in the field of neural engineering at the molecular, cellular and systems levels. The scope of the journal encompasses experimental, computational, theoretical, clinical and applied aspects of: Innovative neurotechnology; Brain-machine (computer) interface; Neural interfacing; Bioelectronic medicines; Neuromodulation; Neural prostheses; Neural control; Neuro-rehabilitation; Neurorobotics; Optical neural engineering; Neural circuits: artificial & biological; Neuromorphic engineering; Neural tissue regeneration; Neural signal processing; Theoretical and computational neuroscience; Systems neuroscience; Translational neuroscience; Neuroimaging.
期刊最新文献
Building consensus on clinical outcome assessments for BCI devices. A summary of the 10th BCI society meeting 2023 workshop. o-CLEAN: a novel multi-stage algorithm for the ocular artifacts' correction from EEG data in out-of-the-lab applications. PDMS/CNT electrodes with bioamplifier for practical in-the-ear and conventional biosignal recordings. DOCTer: a novel EEG-based diagnosis framework for disorders of consciousness. I see artifacts: ICA-based EEG artifact removal does not improve deep network decoding across three BCI tasks.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1