首页 > 最新文献

2009 Data Compression Conference最新文献

英文 中文
LZB: Data Compression with Bounded References 有界引用的数据压缩
Pub Date : 2009-03-16 DOI: 10.1109/DCC.2009.70
M. Banikazemi
In this paper, we propose a new compression/decompression algorithm called LZB which belongs to a class of algorithms related to Lempel-Ziv (LZ). The distinguishing characteristic of LZB is that it allows decompression from arbitrary points of compressed data. This is accomplished by setting a limit on how far back a reference in compressed data can directly or indirectly point to. We enforce this limit by using a sliding "gate." During the compression, we keep track of the origin of each input symbol.  The origin of a symbol is the earliest symbol in the input data that the symbol (directly or indirectly) refers to. By using this information we avoid using any reference which go beyond the gate boundary.  We modified the gzip implementation of LZ77 to implement LZB. We then compared LZB with the alternative method in which data is segmented into smaller pieces and each piece is compressed separately by using the standard gzip. The results show that LZB improves the compression ratio by 10 to 50 percent for 1024 to 128 byte segment sizes.
本文提出了一种新的压缩/解压缩算法LZB,它属于与Lempel-Ziv (LZ)相关的一类算法。LZB的显著特点是它允许从压缩数据的任意点进行解压缩。这是通过限制压缩数据中的引用可以直接或间接指向多远来实现的。我们通过使用滑动“门”来强制执行此限制。在压缩过程中,我们跟踪每个输入符号的起源。符号的起源是指该符号(直接或间接)在输入数据中最早引用的符号。通过使用这些信息,我们避免使用超出栅极边界的任何参考。我们修改了LZ77的gzip实现来实现LZB。然后,我们将LZB与另一种方法进行了比较,在这种方法中,数据被分割成更小的部分,并使用标准gzip分别压缩每个部分。结果表明,对于1024到128字节的段大小,LZB将压缩比提高了10%到50%。
{"title":"LZB: Data Compression with Bounded References","authors":"M. Banikazemi","doi":"10.1109/DCC.2009.70","DOIUrl":"https://doi.org/10.1109/DCC.2009.70","url":null,"abstract":"In this paper, we propose a new compression/decompression algorithm called LZB which belongs to a class of algorithms related to Lempel-Ziv (LZ). The distinguishing characteristic of LZB is that it allows decompression from arbitrary points of compressed data. This is accomplished by setting a limit on how far back a reference in compressed data can directly or indirectly point to. We enforce this limit by using a sliding \"gate.\" During the compression, we keep track of the origin of each input symbol.  The origin of a symbol is the earliest symbol in the input data that the symbol (directly or indirectly) refers to. By using this information we avoid using any reference which go beyond the gate boundary.  We modified the gzip implementation of LZ77 to implement LZB. We then compared LZB with the alternative method in which data is segmented into smaller pieces and each piece is compressed separately by using the standard gzip. The results show that LZB improves the compression ratio by 10 to 50 percent for 1024 to 128 byte segment sizes.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124946666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
On Transform Coding with Dithered Quantizers 用抖动量化器进行变换编码
Pub Date : 2009-03-16 DOI: 10.1109/DCC.2009.76
E. Akyol, K. Rose
This paper is concerned with optimal transform coding in conjunction with dithered quantization.While the optimal deterministic quantizer's error is uncorrelated with the reconstructed value, the dithered quantizer yields quantization errors that are correlated with the reconstruction but are white and independent of the source. These properties offer potential benefits, but also have implications on the optimization of the rest of the coder. We derive the optimal transform for consequent dithered quantization. For fixed rate coding, we show that the transform derived for dithered quantization is universally optimal (for all sources), unlike the conventional quantization case where optimality of the Karhunen-Loeve transform is guaranteed for Gaussian sources. Moreover, we establish variable rate coding optimality for Gaussian sources.
本文研究了结合抖动量化的最优变换编码。最优确定性量化器的误差与重建值不相关,而抖动量化器产生的量化误差与重建值相关,但是白色的,与源无关。这些属性提供了潜在的好处,但也对其他编码人员的优化有影响。我们推导了结果抖动量化的最优变换。对于固定速率编码,我们证明了为抖动量化导出的变换是普遍最优的(对所有源),不像传统的量化情况下,Karhunen-Loeve变换的最优性是保证高斯源。此外,我们还建立了高斯源的可变速率编码最优性。
{"title":"On Transform Coding with Dithered Quantizers","authors":"E. Akyol, K. Rose","doi":"10.1109/DCC.2009.76","DOIUrl":"https://doi.org/10.1109/DCC.2009.76","url":null,"abstract":"This paper is concerned with optimal transform coding in conjunction with dithered quantization.While the optimal deterministic quantizer's error is uncorrelated with the reconstructed value, the dithered quantizer yields quantization errors that are correlated with the reconstruction but are white and independent of the source. These properties offer potential benefits, but also have implications on the optimization of the rest of the coder. We derive the optimal transform for consequent dithered quantization. For fixed rate coding, we show that the transform derived for dithered quantization is universally optimal (for all sources), unlike the conventional quantization case where optimality of the Karhunen-Loeve transform is guaranteed for Gaussian sources. Moreover, we establish variable rate coding optimality for Gaussian sources.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126611698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Clustered Reversible-KLT for Progressive Lossy-to-Lossless 3d Image Coding 累进有损到无损三维图像编码的聚类可逆klt
Pub Date : 2009-03-16 DOI: 10.1109/DCC.2009.7
Ian Blanes, J. Serra-Sagristà
The RKLT is a lossless approximation to the KLT, and has been recently employed for progressive lossy-to-lossless coding of hyperspectral images. Both yield very good coding performance results, but at a high computational price. In this paper we investigate two RKLT clustering approaches to lessen the computational complexity problem: a normal clustering approach, which still yields good performance; and a multi-level clustering approach, which has almost no quality penalty as compared to the original RKLT. Analysis of rate-distortion evolution and of lossless compression ratio is provided. The proposed approaches supply additional benefits, such as spectral scalability, and a decrease of the side information needed to invert the transform. Furthermore,since with a clustering approach, SERM factorization coefficients are bounded to a finite range, the proposed methods allow coding of large three dimensional images within JPEG2000.
RKLT是KLT的一种无损近似,最近被用于高光谱图像的累进有损到无损编码。两者都能产生非常好的编码性能结果,但计算代价很高。在本文中,我们研究了两种RKLT聚类方法来降低计算复杂性问题:一种正常聚类方法,仍然可以产生良好的性能;以及多级聚类方法,与最初的RKLT相比,它几乎没有质量损失。分析了速率失真的演变和无损压缩比。所提出的方法提供了额外的好处,如光谱可扩展性,并减少了反变换所需的侧信息。此外,由于使用聚类方法,SERM分解系数被限定在有限范围内,因此所提出的方法允许在JPEG2000中编码大型三维图像。
{"title":"Clustered Reversible-KLT for Progressive Lossy-to-Lossless 3d Image Coding","authors":"Ian Blanes, J. Serra-Sagristà","doi":"10.1109/DCC.2009.7","DOIUrl":"https://doi.org/10.1109/DCC.2009.7","url":null,"abstract":"The RKLT is a lossless approximation to the KLT, and has been recently employed for progressive lossy-to-lossless coding of hyperspectral images. Both yield very good coding performance results, but at a high computational price. In this paper we investigate two RKLT clustering approaches to lessen the computational complexity problem: a normal clustering approach, which still yields good performance; and a multi-level clustering approach, which has almost no quality penalty as compared to the original RKLT. Analysis of rate-distortion evolution and of lossless compression ratio is provided. The proposed approaches supply additional benefits, such as spectral scalability, and a decrease of the side information needed to invert the transform. Furthermore,since with a clustering approach, SERM factorization coefficients are bounded to a finite range, the proposed methods allow coding of large three dimensional images within JPEG2000.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114965232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 25
Slepian-Wolf Coding of Binary Finite Memory Source Using Burrows-Wheeler Transform 基于Burrows-Wheeler变换的二进制有限记忆源的睡狼编码
Pub Date : 2009-03-16 DOI: 10.1109/DCC.2009.54
Chao Chen, Xiangyang Ji, Qionghai Dai, Xiaodong Liu
In this paper, an asymmetric Slepian-Wolf coding(SWC) scheme for binary finite memory source (FMS) is proposed. The a priori information about the source is extracted from the side information at the decoder by Burrows-Wheeler Transform (BWT). Such information is then utilized for the LDPC code based decoding. Benefiting from the universality of BWT, our coding scheme can be applied to any FMS. Experimental results show that our scheme performs significantly better than the scheme which does not utilize the a priori information for decoding.
提出了一种二进制有限记忆源(FMS)的非对称睡眠-狼编码(SWC)方案。利用Burrows-Wheeler变换(BWT)从解码器处的侧信息中提取有关源的先验信息。然后将这些信息用于基于LDPC码的解码。得益于BWT的通用性,我们的编码方案可以应用于任何FMS。实验结果表明,该方案的解码性能明显优于不利用先验信息的解码方案。
{"title":"Slepian-Wolf Coding of Binary Finite Memory Source Using Burrows-Wheeler Transform","authors":"Chao Chen, Xiangyang Ji, Qionghai Dai, Xiaodong Liu","doi":"10.1109/DCC.2009.54","DOIUrl":"https://doi.org/10.1109/DCC.2009.54","url":null,"abstract":"In this paper, an asymmetric Slepian-Wolf coding(SWC) scheme for binary finite memory source (FMS) is proposed. The a priori information about the source is extracted from the side information at the decoder by Burrows-Wheeler Transform (BWT). Such information is then utilized for the LDPC code based decoding. Benefiting from the universality of BWT, our coding scheme can be applied to any FMS. Experimental results show that our scheme performs significantly better than the scheme which does not utilize the a priori information for decoding.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"92 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133173557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Perceptual Relevance Measure for Generic Shape Coding 通用形状编码的感知关联测度
Pub Date : 2009-03-01 DOI: 10.1109/DCC.2009.14
Zhongyuan Lai, Wenyu Liu, Yuan Zhang
We address a fundamental problem in subjective reconstruction quality by introducing a perceptual relevance measure (ADM) for generic vertex-based shape coding. Different from traditional absolute distance measure (ADM), our proposed measure systemically considers the turn angel and two adjacent segments for corresponding vertex visual significance calculation. We embed our proposed measure into top-down and bottom-up frameworks, stages of both vertex selection and adjustments, and class one and class two distortion measure definitions. The experiment results show that our proposed measure can significantly improve subjective reconstruction quality as well as objective rate-distortion performance, especially for object shape with sharp salience.
我们通过引入一种基于通用顶点的形状编码的感知相关度量(ADM)来解决主观重构质量的基本问题。与传统的绝对距离度量(ADM)不同,我们提出的度量系统地考虑了转角和相邻的两个线段来进行相应顶点的视觉显著性计算。我们将我们提出的测量嵌入到自顶向下和自底向上的框架,顶点选择和调整的阶段,以及第一类和第二类失真测量定义中。实验结果表明,该方法可以显著提高主观重建质量和客观率失真性能,特别是对于具有显著性的物体形状。
{"title":"Perceptual Relevance Measure for Generic Shape Coding","authors":"Zhongyuan Lai, Wenyu Liu, Yuan Zhang","doi":"10.1109/DCC.2009.14","DOIUrl":"https://doi.org/10.1109/DCC.2009.14","url":null,"abstract":"We address a fundamental problem in subjective reconstruction quality by introducing a perceptual relevance measure (ADM) for generic vertex-based shape coding. Different from traditional absolute distance measure (ADM), our proposed measure systemically considers the turn angel and two adjacent segments for corresponding vertex visual significance calculation. We embed our proposed measure into top-down and bottom-up frameworks, stages of both vertex selection and adjustments, and class one and class two distortion measure definitions. The experiment results show that our proposed measure can significantly improve subjective reconstruction quality as well as objective rate-distortion performance, especially for object shape with sharp salience.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"53 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129978422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Compression-Induced Rendering Distortion Analysis for Texture/Depth Rate Allocation in 3D Video Compression 三维视频压缩中纹理/深度率分配的压缩诱导渲染失真分析
Pub Date : 2009-03-01 DOI: 10.1109/DCC.2009.27
Yanwei Liu, Siwei Ma, Qingming Huang, Debin Zhao, Wen Gao, N. Zhang
In 3D video applications, the virtual view is generally rendered by the compressed texture and depth. The texture and depth compression with different bit-rate overheads can lead to different virtual view rendering qualities. In this paper, we analyze the compression-induced rendering distortion for the virtual view. Based on the 3D warping principle, we first address how the texture and depth compression affects the virtual view quality, and then derive an upper bound for the compression-induced rendering distortion. The derived distortion bound depends on the compression-induced depth error and texture intensity error. Simulation results demonstrate that the theoretical upper bound is an approximate indication of the rendering quality and can be used to guide sequence-level texture/depth rate allocation for 3D video compression.
在3D视频应用中,虚拟视图通常是通过压缩纹理和深度来渲染的。具有不同比特率开销的纹理和深度压缩会导致不同的虚拟视图渲染质量。本文分析了虚拟视图的压缩渲染失真问题。基于三维扭曲原理,我们首先讨论了纹理和深度压缩对虚拟视图质量的影响,然后推导了压缩引起的渲染失真的上限。导出的失真边界取决于压缩引起的深度误差和纹理强度误差。仿真结果表明,理论上限是渲染质量的近似指示,可用于指导3D视频压缩的序列级纹理/深度率分配。
{"title":"Compression-Induced Rendering Distortion Analysis for Texture/Depth Rate Allocation in 3D Video Compression","authors":"Yanwei Liu, Siwei Ma, Qingming Huang, Debin Zhao, Wen Gao, N. Zhang","doi":"10.1109/DCC.2009.27","DOIUrl":"https://doi.org/10.1109/DCC.2009.27","url":null,"abstract":"In 3D video applications, the virtual view is generally rendered by the compressed texture and depth. The texture and depth compression with different bit-rate overheads can lead to different virtual view rendering qualities. In this paper, we analyze the compression-induced rendering distortion for the virtual view. Based on the 3D warping principle, we first address how the texture and depth compression affects the virtual view quality, and then derive an upper bound for the compression-induced rendering distortion. The derived distortion bound depends on the compression-induced depth error and texture intensity error. Simulation results demonstrate that the theoretical upper bound is an approximate indication of the rendering quality and can be used to guide sequence-level texture/depth rate allocation for 3D video compression.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128701137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 12
Joint Network-Source Video Coding Based on Lagrangian Rate Allocation 基于拉格朗日速率分配的联合网络源视频编码
Pub Date : 2009-03-01 DOI: 10.1109/DCC.2009.5
Xuguang Lan, Nanning Zheng, Jianru Xue, Ce Li, Songlin Zhao
Joint network-source video coding (JNSC) is targeted to achieve the optimum delivery of a video source to a number of destinations over network with capacity constraints. In this paper, a practical scalable multiple description coding is proposed for JNSC, based on Lagrangian rate allocation and scalable video coding. After the spatiotemporal wavelet transformation of input video sequence and the bit plane coding and context-based adaptive binary arithmetic coding, jointing network-source coding is performed on the coding passes of the code blocks using Lagrangian rate allocation. The principle relationship of the rate-distortion slop ratio with receiving probability in network links is derived under the link capacity constraints. In this way, scalable multiple descriptions can be generated to optimize the delivery to be robust and adaptive to the dynamics of heterogeneous networks. The performance of the proposed scalable multiple description coding is explored in the Peer-to-Peer streaming network.
联合网络源视频编码(JNSC)的目标是在容量受限的网络上实现视频源到多个目的地的最佳传输。本文提出了一种实用的基于拉格朗日速率分配和可扩展视频编码的JNSC可扩展多描述编码方法。在对输入视频序列进行时空小波变换、位平面编码和基于上下文的自适应二进制算术编码后,利用拉格朗日速率分配对码块的编码通道进行连接网络源编码。在链路容量约束下,导出了网络链路中速率畸变斜率比与接收概率的基本关系。通过这种方式,可以生成可伸缩的多个描述,以优化交付,使其健壮并适应异构网络的动态。研究了所提出的可扩展多描述编码在点对点流网络中的性能。
{"title":"Joint Network-Source Video Coding Based on Lagrangian Rate Allocation","authors":"Xuguang Lan, Nanning Zheng, Jianru Xue, Ce Li, Songlin Zhao","doi":"10.1109/DCC.2009.5","DOIUrl":"https://doi.org/10.1109/DCC.2009.5","url":null,"abstract":"Joint network-source video coding (JNSC) is targeted to achieve the optimum delivery of a video source to a number of destinations over network with capacity constraints. In this paper, a practical scalable multiple description coding is proposed for JNSC, based on Lagrangian rate allocation and scalable video coding. After the spatiotemporal wavelet transformation of input video sequence and the bit plane coding and context-based adaptive binary arithmetic coding, jointing network-source coding is performed on the coding passes of the code blocks using Lagrangian rate allocation. The principle relationship of the rate-distortion slop ratio with receiving probability in network links is derived under the link capacity constraints. In this way, scalable multiple descriptions can be generated to optimize the delivery to be robust and adaptive to the dynamics of heterogeneous networks. The performance of the proposed scalable multiple description coding is explored in the Peer-to-Peer streaming network.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121545340","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Low Complexity Spatio-Temporal Key Frame Encoding for Wyner-Ziv Video Coding 用于Wyner-Ziv视频编码的低复杂度时空关键帧编码
Pub Date : 2009-03-01 DOI: 10.1109/DCC.2009.57
Ghazaleh Esmaili, P. Cosman
In most Wyner-Ziv video coding approaches, the temporal correlation of key frames is not exploited since they are simply intra encoded and decoded. In this paper, using the previously decoded key frame as the side information for the key frame to be decoded, we propose new methods of coding key frames in order to improve the rate distortion performance. These schemes which are based on switching between intra mode and Wyner-Ziv mode for a given block or a given frequency band attempt to make use of both spatial and temporal correlation of key frames while satisfying the low complexity encoding requirement of Distributed Video Coding (DVC). Simulation results show that using the proposed methods, one can achieve up to 5 dB improvement over conventional intra coding for relatively low motion sequences and up to 1.3 dB improvement for relatively high motion sequences.
在大多数Wyner-Ziv视频编码方法中,关键帧的时间相关性没有被利用,因为它们只是在内部编码和解码。本文利用已解码的关键帧作为待解码关键帧的边信息,提出了新的关键帧编码方法,以提高帧率失真性能。这些方案基于给定块或给定频带的内模和Wyner-Ziv模之间的切换,试图在满足分布式视频编码(DVC)低复杂度编码要求的同时,利用关键帧的时空相关性。仿真结果表明,采用本文提出的方法,相对于传统的低运动序列的帧内编码,可以实现高达5 dB的改进,相对于高运动序列,可以实现高达1.3 dB的改进。
{"title":"Low Complexity Spatio-Temporal Key Frame Encoding for Wyner-Ziv Video Coding","authors":"Ghazaleh Esmaili, P. Cosman","doi":"10.1109/DCC.2009.57","DOIUrl":"https://doi.org/10.1109/DCC.2009.57","url":null,"abstract":"In most Wyner-Ziv video coding approaches, the temporal correlation of key frames is not exploited since they are simply intra encoded and decoded. In this paper, using the previously decoded key frame as the side information for the key frame to be decoded, we propose new methods of coding key frames in order to improve the rate distortion performance. These schemes which are based on switching between intra mode and Wyner-Ziv mode for a given block or a given frequency band attempt to make use of both spatial and temporal correlation of key frames while satisfying the low complexity encoding requirement of Distributed Video Coding (DVC). Simulation results show that using the proposed methods, one can achieve up to 5 dB improvement over conventional intra coding for relatively low motion sequences and up to 1.3 dB improvement for relatively high motion sequences.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114470756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 12
On the Use of Suffix Arrays for Memory-Efficient Lempel-Ziv Data Compression 后缀数组在内存高效Lempel-Ziv数据压缩中的应用
Pub Date : 2009-03-01 DOI: 10.1109/DCC.2009.50
Artur J. Ferreira, Arlindo L. Oliveira, Mário A. T. Figueiredo
Much research has been devoted to optimizing algorithms of the Lempel-Ziv (LZ) 77 family, both in terms of speed and memory requirements. Binary search trees and suffix trees (ST) are data structures that have been often used for this purpose, as they allow fast searches at the expense of memory usage.In recent years, there has been interest on suffix arrays (SA), due to their simplicity and low memory requirements. One key issue is that an SA can solve the sub-string problem almost as efficiently as an ST, using less memory.  This paper proposes two new SA-based algorithms for LZ encoding, which require no modifications on the decoder side. Experimental results on standard benchmarks show that our algorithms, though not faster,  use 3 to 5 times less memory than the ST counterparts. Another important feature of our SA-based algorithms is that the amount of memory is independent of the text to search, thus the memory that has to be allocated can be defined a priori. These features of low and predictable memory requirements are of the utmost importance in several scenarios, such as embedded systems, where memory is at a premium and speed is not critical. Finally, we point out that the new algorithms are general, in the sense that they are adequate for applications other than LZ compression, such as text retrieval and forward/backward sub-string search.
许多研究致力于优化Lempel-Ziv (LZ) 77家族的算法,无论是在速度方面还是在内存需求方面。二叉搜索树和后缀树(ST)是经常用于此目的的数据结构,因为它们允许以牺牲内存使用为代价进行快速搜索。近年来,由于后缀数组的简单性和低内存需求,人们对其产生了兴趣。一个关键问题是,SA几乎可以像ST一样有效地解决子字符串问题,使用更少的内存。本文提出了两种新的基于sa的LZ编码算法,这两种算法不需要在解码器端进行任何修改。在标准基准测试上的实验结果表明,我们的算法虽然没有更快,但使用的内存比ST对应的算法少3到5倍。我们基于sa的算法的另一个重要特性是内存的数量与要搜索的文本无关,因此必须分配的内存可以先验地定义。这些低且可预测的内存需求的特性在一些场景中非常重要,例如嵌入式系统,在这些场景中,内存非常宝贵,而速度并不重要。最后,我们指出,新算法是通用的,因为它们适用于LZ压缩以外的应用,例如文本检索和向前/向后子字符串搜索。
{"title":"On the Use of Suffix Arrays for Memory-Efficient Lempel-Ziv Data Compression","authors":"Artur J. Ferreira, Arlindo L. Oliveira, Mário A. T. Figueiredo","doi":"10.1109/DCC.2009.50","DOIUrl":"https://doi.org/10.1109/DCC.2009.50","url":null,"abstract":"Much research has been devoted to optimizing algorithms of the Lempel-Ziv (LZ) 77 family, both in terms of speed and memory requirements. Binary search trees and suffix trees (ST) are data structures that have been often used for this purpose, as they allow fast searches at the expense of memory usage.In recent years, there has been interest on suffix arrays (SA), due to their simplicity and low memory requirements. One key issue is that an SA can solve the sub-string problem almost as efficiently as an ST, using less memory.  This paper proposes two new SA-based algorithms for LZ encoding, which require no modifications on the decoder side. Experimental results on standard benchmarks show that our algorithms, though not faster,  use 3 to 5 times less memory than the ST counterparts. Another important feature of our SA-based algorithms is that the amount of memory is independent of the text to search, thus the memory that has to be allocated can be defined a priori. These features of low and predictable memory requirements are of the utmost importance in several scenarios, such as embedded systems, where memory is at a premium and speed is not critical. Finally, we point out that the new algorithms are general, in the sense that they are adequate for applications other than LZ compression, such as text retrieval and forward/backward sub-string search.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128129396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
An Implementable Scheme for Universal Lossy Compression of Discrete Markov Sources 离散马尔可夫源的通用有损压缩实现方案
Pub Date : 2009-01-15 DOI: 10.1109/DCC.2009.72
S. Jalali, A. Montanari, T. Weissman
We present a new lossy compressor for discrete sources. For coding a source sequence $x^n$, the encoder starts by assigning a certain cost to each reconstruction sequence. It then finds the reconstruction that minimizes this cost and describes it losslessly to the decoder via a universal lossless compressor. The cost of a sequence is given by a linear combination of its empirical probabilities of some order $k+1$ and its distortion relative to the source sequence. The linear structure of the cost in the empirical count  matrix allows the encoder to employ a Viterbi-like algorithm for obtaining the minimizing reconstruction sequence simply. We identify a choice of coefficients for the linear combination in the cost function which ensures that the algorithm universally achieves the optimum rate-distortion performance of any Markov source in the limit of large $n$, provided $k$ is increased as $o(log n)$.
提出了一种新的离散源有损压缩器。对于编码源序列$x^n$,编码器首先为每个重构序列分配一定的代价。然后,它找到重构,使该成本最小化,并通过通用无损压缩器将其无损描述给解码器。序列的代价是由它的经验概率(k+1阶)及其相对于源序列的失真的线性组合给出的。经验计数矩阵中代价的线性结构允许编码器采用类似维特比的算法来简单地获得最小重构序列。我们确定了成本函数中线性组合的系数选择,以确保算法在大$n$的极限下普遍实现任何马尔可夫源的最佳率失真性能,前提是$k$增加为$o(log n)$。
{"title":"An Implementable Scheme for Universal Lossy Compression of Discrete Markov Sources","authors":"S. Jalali, A. Montanari, T. Weissman","doi":"10.1109/DCC.2009.72","DOIUrl":"https://doi.org/10.1109/DCC.2009.72","url":null,"abstract":"We present a new lossy compressor for discrete sources. For coding a source sequence $x^n$, the encoder starts by assigning a certain cost to each reconstruction sequence. It then finds the reconstruction that minimizes this cost and describes it losslessly to the decoder via a universal lossless compressor. The cost of a sequence is given by a linear combination of its empirical probabilities of some order $k+1$ and its distortion relative to the source sequence. The linear structure of the cost in the empirical count  matrix allows the encoder to employ a Viterbi-like algorithm for obtaining the minimizing reconstruction sequence simply. We identify a choice of coefficients for the linear combination in the cost function which ensures that the algorithm universally achieves the optimum rate-distortion performance of any Markov source in the limit of large $n$, provided $k$ is increased as $o(log n)$.","PeriodicalId":377880,"journal":{"name":"2009 Data Compression Conference","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128049226","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
期刊
2009 Data Compression Conference
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1