Spherical Panorama Construction Using Multi Sensor Registration Priors and Its Real-Time Hardware

Abstract

In this work, a novel method is presented to improve the quality of panoramic images on a spherically arranged multi sensor imaging system. The new method is composed of two parts. The first approach proposed is based on mapping the panorama generation problem onto a Markov Random Field (MRF) and then estimating posterior probabilities from initial likelihoods. The novelty of approach is based on extracting the prior evidence from the registration information of multiple cameras and estimating expected value on an undirected graph. The second part of the method is a geometrical approach targeting a better estimation for the initial priors, which is also not applied before. The aim of both approaches is to decrease the parallax errors and ghosting effects which occur due to the nature of multi camera systems. It is shown that instead of directly using independent intensity coefficients extracted from registration information, applying a neighborhood based local probability distribution for each pixel of panorama utilizing the registration information as prior gives better results. Visual comparisons are provided to show the achieved quality enhancement in terms of seamless and more natural panoramic image with less ghosting effects. Since the registration priors are used effectively with a single iteration step in a 4 connected neighborhood, the need for an intensity based loopy and iterative inference method is prohibited. Hence, the proposed methods are suitable for real-time hardware implementation. A hardware implementation of the method for real-time operation is proposed.