The Integration of Speech and Camera Control in Message Transfer TV Conferencing

Abstract

In this study, we examined participants' integration of speech and camera control during message-transfer television conferencing. We obtained participants' free use of speech and camera-control modality when they transmitted 3 types of objects--formed text (FT), mathematical expression (ME), and electronic circuit (EC)-each ranging in size from 1 to 7 elements. Modality use was classified into main-modality use and auxiliary-modality use. Five integration models were proposed based on main-auxiliary combinations. Results indicated that (a) speech was used more frequently for transmitting ME objects, and camera control was used more often for FT and EC objects; (b) speech was gradually replaced by camera control as size increased; (c) when camera control was chosen as main modality, speech was usually also used, as auxiliary modality; and (d) compared to beginners, experienced participants used more speech. A postsession questionnaire was conducted to collect participants' assessment of the correctness, s...