Construction of retrieval system for pictorial book of flora

Pattern information and natural language information used together can complement and reinforce each other to enable more effective communication than can either medium alone. A good example is a pictorial book of flora (PBF). In the PBF, readable explanations combine texts and pictures. However, it is difficult to retrieve explanation text and pictures from the PBF when we don't know the names of flowers. To solve this problem, we propose a retrieval method for the PBF using the color feature of each flower and fruit, and construct an experimental retrieval system for the PBF. For obtaining the color feature of each flower and fruit, we analysed the PBF pictures and found several problems as follows: Pictures of the PBF contain many kinds of objects. In addition to flowers and fruits, there are leaves, stems, skies, soils, and sometimes humans in the PBF pictures. The position, size, and direction of flowers and fruits vary quite widely in each picture. Each flower and fruit has its unique shape, color, and texture which are commonly different from those of the others. Because of these problems, it is difficult to build the general and precise model for analyzing the PBF pictures in advance. We propose a method for image analysis using natural language information. Our method works as follows. First, we analyse the PBF explanation texts for extracting the color information on each flower and fruit. Then, we analyse the PBF pictures by using the results of the natural language processing, and finally obtain the color feature of each flower and fruit.