Efficient shape modeling using occupancy reasoning with reconstruction scheduling for interactive virtual environments

An interactive virtual environment (IVE) is a world where virtual objects and real world objects communicate interactively with each other like in AR or AV worlds. In terms of interaction, reasonable response time is a key aspect for real-time reaction. To achieve it, reconstruction of real world objects in a virtual world should be done as efficient as possible. The conventional multiple camera based approaches for shape modeling are not proper for a visual sensor network (VSN), because limited resources such as the lack of power, memory capacity, and computing performance were not under the consideration and the space covered by cameras gets even much larger. In IVE with VSNs, two problems for the reconstruction take place as follows. First, performance for reconstruction decreases due to increasing scanning space. Second, energy consumption for communicating silhouette images increases. In order to solve those two problems, first we build up a probability map which represents three-dimensional occupancy of multiple moving objects in the ground plane. This helps restrict the scanning space only within the occupancy region. Second, we make a schedule of reconstruction, which means to decide which cameras reconstruct which objects. With our method, shape modeling in IVE with VSNs can be done in an efficient manner.