A Probabilistic Model of Referring Expressions for Complex Objects

Abstract

This paper presents a probabilistic model both for generation and understanding of referring expressions. This model introduces the concept of parts of objects, modelling the necessity to deal with the characteristics of separate parts of an object in the referring process. This was ignored or implicit in previous literature. Integrating this concept into a probabilistic formulation, the model captures human characteristics of visual perception and some type of pragmatic implicature in referring expressions. Developing this kind of model is critical to deal with more complex domains in the future. As a first step in our research, we validate the model with the TUNA corpus to show that it includes conventional domain modeling as a subset.