A center-surround framework for spatial image processing

This paper presents a computational framework inspired by the center-surround antagonistic receptive fields of the human visual system. It demonstrates that, starting from the actual pixel value (center) and a low-pass estimation of the pixel’s neighborhood (surround) and using a mapping function inspired by the shunting inhibition mechanism, some widely used spatial image processing techniques can be implemented, including adaptive tone-mapping, local contrast enhancement, text binarization and local feature detection. As a result, it highlights the relations of these seemingly different applications with the early stages of the human visual system and draws insights about their characteristics. Introduction Center-surround antagonistic Receptive Fields (RFs) are abundant in the Human Visual System (HVS). They have been found in many areas, such as the retina, the Lateral Geniculate Nucleus, V1 or in higher visual areas. It seems that this is a typical strategy that the HVS employs for local signal comparisons, not only in vision but in other sensory areas as well. The RFs of center-surround cells comprise two separate concentric regions sampling the photoreceptor mosaic (namely the center and the surround) that act antagonistically on the final output of the cell. ON center-surround cells exhibit increased output with higher photoreceptor activity on their center and decreased output with increased activity on their surround. Conversely, for OFF center-surround cells, higher photoreceptor activity on the center has a negative impact on their output, whereas, increased photoreceptor activity on the surround increases their output. The size of the two regions defines the spatial frequency of sampling: smaller RF sizes sample finer details from the photoreceptor mosaic, while larger sizes encode coarser scales of the same signal. Center-surround cells are essentially a biological implementation of spatial filtering. Spatial filtering is a very broad term, encompassing any kind of filtering operations that depend on the local content of the signal and are not globally constant. Almost all existing image processing and computational photography techniques include some kind of spatial image processing. Modern denoising, local contrast enhancement, scale decomposition, exposure fusion, HDR tone mapping are some of them. Most of these methods have some common grounds with the basic computational models of the early stages of the HVS. However these similarities are not always so evident. In this paper, we start from the computational model of the first stages of HVS, developed by Grossberg [24], and we adapt it for image processing operations. Explicitly modeling HVS is out of the scope of this paper. We rather draw inspiration from it in order to address real-world imaging problems. More specifically, we define a framework, inspired by Grossberg’s theory, that describes center-surround signal interactions. We show that such a framework can give rise to existing spatial image processing techniques, as many of them are special cases of it. This gives a more unified view between image processing and biological vision models, highlighting their common ground and showing other potential applications that can be developed. Modeling Center-Surround RFs Traditionally, center-surround RFs have been modeled as Difference of Gaussians (DoG) [13]. This linear operator essentially approximates the Laplacian operator, by subtracting two Gaussians of different sigmas, centered in the same position. DoG is at the heart of many computer vision and image processing algorithms, such as edge detection [12], scale-space construction [1] and local feature detectors [11]. Contrary to the linear response of the DoG operator though, the center-surround cells of the HVS exhibit non-linear response in regards to their inputs. Interestingly, their nonlinear response is thought to contribute to illumination invariance and contrast enhancement [24]. According to the standard retinal model [6, 21], the output Vi j of an ON-center OFF-surround cell at grid position (i, j), obeying the membrane equations of physiology is given by dVi j (t) dt = gleak ( Vrest −Vi j ) +Ci j ( Eex−Vi j ) +Si j ( Einh−Vi j )