Love is blue: Designing and using a multimodal corpus analysis tool

Abstract

With multimodal visual texts becoming increasingly important, especially online, there is a need for an effective tool to analyze these texts. In this paper, we present the Multimodal Corpus Analysis Tool which uses sets of categories, such as dominant colors and object location, to analyze a corpus of online advertisements. The 12 categories are based on both multimodal theory and advertising theory, and the tool produces frequency counts of codes and likelihood of co-occurrence of two codes using observed-to-expected ratio. We illustrate the use of the tool through three case studies. First, examining the left and right positioning of verbal content and images in horizontal adverts, we find that there is a general preference for language left – picture right adverts, a finding potentially questioning currently accepted theories of information value in multimodal texts. Second, examining color schemes in adverts, we find that most use of dominant colors reflects accepted meanings in advice on multimodal text design, except for blue which is associated with storge love. For brightness, colorfulness and contrast, there are differences between adverts for everyday products and adverts for services and expensive products. Third, focusing on the call to action phrases in the adverts, there are differences in the use of the two most common forms (shop now and learn more) in adverts for products and services. These cases illustrate the potential benefits of developing and using multimodal corpus tools.