Digital Representation of Perceptual Criteria

Abstract

Information signals are typically intended for human consumption. Human perception therefore contributes directly to fidelity criteria for digital representation. As computational capabilities increase and costs diminish, coding algorithms are able to iiicorporate more of the constraints that characterize perception. The incentive is still-greater economy for digital transmission and storage. Sight and sound are sensory modes favored by the human for information exchange. These modes are presently most central to humadmachine communications and multimedia systems. The intricacies of visual and auditory perception are therefore figuring more prominently in signal coding. For example, taking account of the eye's sensitivity to quantizing noise as a function of temporal and spatial frequencies leads to good-quality coding of color motion images at fractions of a bit per pixel. Similarly, the characteristics of auditory masking, in both time and frequency domains, provide leverage to identify signal components which are irrelevant to perception and which need not consume coding capacity. This discussion draws a perspective on recent coding advances and points up opportunities for increased sophistication in representing perceptual I y imp0 rtan t factors. It also indicates relations hips between economies gained by perceptual coding alone, and those where source coding can trade on signal-specific characteristics to achieve further reductions in bit rate. It COnChdeS with brief consideration of other sensory modalities, such as the tactile dimension, that might contribute to naturalness and ease of use in interactive multimedia information systems.