Frequency component extraction from color images for specific sound transformation and analysis

Abstract

This paper presents a method allowing the conversion of images into sound. Initially, a frequency component extraction is realized from the original image. At this stage, the image is divided into windows in order to represent consecutive different time periods using STFT. Then, the dominant frequencies of each window are mapped into corresponding sound frequencies through Fourier analysis. This procedure is applied twice and two series of sound frequency components are produced: The first is originated from the brightness of the image, the second from the dominant RGB layer. The connection between the visual impression of the image and the psychoacoustic effect of the sound mapping is done by using different musical scales according to the dominant color of the image. The results revealed that the melody extracted from this analysis produces a certain psychoacoustic impression, as it has reported by several volunteers. Despite the fact that volunteers could not always do the association between image and sound, they could hardly believe that the music was produced by an algorithmic procedure.