Multisensory Research最新文献

The aim of this study was twofold. First, our objective was to test the influence of an object's actual size (size rank) on the drawn size of the depicted object. We tested the canonical size effect (i.e., drawing objects larger in the physical world as larger) in four drawing conditions - two perceptual conditions (blindfolded or sighted) crossed with two materials (paper or special foil for producing embossed drawings). Second, we investigated whether drawing quality (we analysed both the local and global criteria of quality) depends on drawing conditions. We predicted that drawing quality, unlike drawing size, would vary according to drawing conditions - namely, being higher when foil than paper was used for drawing production in the blindfolded condition. We tested these hypotheses with young adults who repeatedly drew eight different familiar objects (differentiated by size in the real world) in four drawing conditions. As expected, drawn size increased linearly with increasing size rank, whatever the drawing condition, thus replicating the canonical size effect and showing that this effect was not dependent on drawing conditions. In line with our hypothesis, in the blindfolded condition drawing quality was better when foil rather than paper was used, suggesting a benefit from haptic feedback on the trace produced. Besides, the quality of drawings produced was still higher in the sighted than the blindfolded condition. In conclusion, canonical size is present under different drawing conditions regardless of whether sight is involved or not, while perceptual control increases drawing quality in adults.

The quality of a concert hall primarily depends on its acoustics. But does visual input also have an impact on musical enjoyment? Does the color of ambient lighting modulate the perceived music quality? And are certain colors perceived to fit better than others with a given music piece? To address these questions, we performed three within-subjects experiments. We carried out two pretests to select four music pieces differing in tonality and genre, and 14 lighting conditions of varying hue, brightness, and saturation. In the main experiment, we applied a fully crossed repeated-measures design. Under each of the four lighting conditions, participants rated the musical variables 'Harmonic', 'Powerful', 'Gloomy', 'Lively' and overall liking of the music pieces, as well as the perceived fit of music and lighting. Subsequently, participants evaluated music and lighting separately by rating the same variables as before, as well as their emotional impact (valence, arousal, dominance). We found that music and lighting being similarly rated in terms of valence and arousal in the unimodal conditions were judged to match better when presented together. Accordingly, tonal (atonal) music was rated to fit better with weakly saturated (highly saturated) colors. Moreover, some characteristics of the lighting were carried over to music. That is, just as red lighting was rated as more powerful than green and blue lighting, music was evaluated to be more powerful under red compared to green and blue lighting. We conclude that listening to music is a multisensory process enriched by impressions from the visual domain.

We report two experiments designed to investigate whether the presentation of a range of pleasant fragrances, containing both floral and fruity notes, would modulate people's judgements of the facial attractiveness (Experiment 1) and age (Experiment 2) of a selection of typical female faces varying in age in the range 20-69 years. In Experiment 1, male participants rated the female faces as less attractive when presented with an unpleasant fragrance compared to clean air. The rated attractiveness of the female faces was lower when the participants rated the unpleasant odour as having a lower attractiveness and pleasantness, and a higher intensity. In Experiment 2, both male and female participants rated the age of female faces while presented with one of four pleasant fragrances or clean air as a control. Only the female participants demonstrated a crossmodal effect, with the pleasant fragrances inducing an older rating for female faces in the 40-49-years-old age range, whereas a younger rating was documented for female faces in the 60-69-years-old age range. Taken together, these results are consistent with the view that while the valence of fragrance (pleasant versus unpleasant) exerts a robust crossmodal influence over judgements of facial attractiveness, the effects of pleasant fragrance on judgements of a person's age appear to be less reliable. One possible explanation for the differing effect of scent in the two cases relates to the fact that attractiveness judgements are more subjective, hedonic, and/or intuitive than age ratings which are more objective, cognitive-mediated, and/or analytic in nature.

The purpose of this study was to investigate the cue congruency effect of auditory stimuli during visual search in dynamic environments. Twenty-eight participants were recruited to conduct a visual search experiment. The experiment applied auditory stimuli to understand whether they could facilitate visual search in different types of background. Additionally, target location and target orientation were manipulated to clarify their influences on visual search. Target location was related to horizontal visual search and target orientation was associated with visual search for an inverted target. The results regarding dynamic backgrounds reported that target-congruent auditory stimuli could speed up the visual search time. In addition, the cue congruency effect of auditory stimuli was critical for the center of the visual display but declined for the edge, indicating the inhibition of horizontal visual search behavior. Moreover, few improvements accompanying auditory stimuli were provided for the visual detection of non-inverted and inverted targets. The findings of this study suggested developing multisensory interaction with head-mounted displays, such as augmented reality glasses, in real life.

In this review, we discuss how specific sensory channels can mediate the learning of properties of the environment. In recent years, schools have increasingly been using multisensory technology for teaching. However, it still needs to be sufficiently grounded in neuroscientific and pedagogical evidence. Researchers have recently renewed understanding around the role of communication between sensory modalities during development. In the current review, we outline four principles that will aid technological development based on theoretical models of multisensory development and embodiment to foster in-depth, perceptual, and conceptual learning of mathematics. We also discuss how a multidisciplinary approach offers a unique contribution to development of new practical solutions for learning in school. Scientists, engineers, and pedagogical experts offer their interdisciplinary points of view on this topic. At the end of the review, we present our results, showing that one can use multiple sensory inputs and sensorimotor associations in multisensory technology to improve the discrimination of angles, but also possibly for educational purposes. Finally, we present an application, the 'RobotAngle' developed for primary (i.e., elementary) school children, which uses sounds and body movements to learn about angles.

Previous studies have found that semantics, the higher-level meaning of stimuli, can impact multisensory integration; however, less is known about the effect of valence, an affective response to stimuli. This study investigated the effects of both semantic congruency and valence of non-speech audiovisual stimuli on multisensory integration via response time (RT) and temporal-order judgement (TOJ) tasks [assessing processing speed (RT), Point of Subjective Simultaneity (PSS), and time window when multisensory stimuli are likely to be perceived as simultaneous (temporal binding window; TBW)]. Through an online study with 40 participants (mean age: 26.25 years; females = 17), we found that both congruence and valence had a significant main effect on RT (congruency and positive valence decrease RT) and an interaction effect (congruent/positive valence condition being significantly faster than all others). For TOJ, there was a significant main effect of valence and a significant interaction effect where positive valence (compared to negative valence) and the congruent/positive condition (compared to all other conditions) required visual stimuli to be presented significantly earlier than auditory stimuli to be perceived as simultaneous. A subsequent analysis showed a positive correlation between TBW width and RT (as TBW widens, RT increases) for the categories that were furthest from true simultaneity in their PSS (Congruent/Positive and Incongruent/Negative). This study provides new evidence that supports previous research on semantic congruency and presents a novel incorporation of valence into behavioural responses.

Integration of incoming sensory signals from multiple modalities is central in the determination of self-motion perception. With the emergence of consumer virtual reality (VR), it is becoming increasingly common to experience a mismatch in sensory feedback regarding motion when using immersive displays. In this study, we explored whether introducing various discrepancies between the vestibular and visual motion would influence the perceived timing of self-motion. Participants performed a series of temporal-order judgements between an auditory tone and a passive whole-body rotation on a motion platform accompanied by visual feedback using a virtual environment generated through a head-mounted display. Sensory conflict was induced by altering the speed and direction by which the movement of the visual scene updated relative to the observer's physical rotation. There were no differences in perceived timing of the rotation without vision, with congruent visual feedback and when the speed of the updating of the visual motion was slower. However, the perceived timing was significantly further from zero when the direction of the visual motion was incongruent with the rotation. These findings demonstrate the potential interaction between visual and vestibular signals in the temporal perception of self-motion. Additionally, we recorded cybersickness ratings and found that sickness severity was significantly greater when visual motion was present and incongruent with the physical motion. This supports previous research regarding cybersickness and the sensory conflict theory, where a mismatch between the visual and vestibular signals may lead to a greater likelihood for the occurrence of sickness symptoms.

The integration of information from different sensory modalities is crucial for successful navigation through an environment. Among others, self-motion induces distinct optic flow patterns on the retina, vestibular signals and tactile flow, which contribute to determine traveled distance (path integration) or movement direction (heading). While the processing of combined visual-vestibular information is subject to a growing body of literature, the processing of visuo-tactile signals in the context of self-motion has received comparatively little attention. Here, we investigated whether visual heading perception is influenced by behaviorally irrelevant tactile flow. In the visual modality, we simulated an observer's self-motion across a horizontal ground plane (optic flow). Tactile self-motion stimuli were delivered by air flow from head-mounted nozzles (tactile flow). In blocks of trials, we presented only visual or tactile stimuli and subjects had to report their perceived heading. In another block of trials, tactile and visual stimuli were presented simultaneously, with the tactile flow within ±40° of the visual heading (bimodal condition). Here, importantly, participants had to report their perceived visual heading. Perceived self-motion direction in all conditions revealed a centripetal bias, i.e., heading directions were perceived as compressed toward straight ahead. In the bimodal condition, we found a small but systematic influence of task-irrelevant tactile flow on visually perceived headings as function of their directional offset. We conclude that tactile flow is more tightly linked to self-motion perception than previously thought.