Multisensory Research最新文献

There has been a rapid recent growth in academic attempts to summarise, understand, and predict the taste profile matching complex images that incorporate multiple visual design features. While there is now ample research to document the patterns of vision-taste correspondences involving individual visual features (such as colour and shape curvilinearity in isolation), little is known about the taste associations that may be primed when multiple visual features are presented simultaneously. This narrative historical review therefore presents an overview of the research that has examined, or provided insights into, the interaction of graphic elements in taste correspondences involving colour, shape attributes, texture, and other visual features. The empirical evidence is largely in line with the predictions derived from the proposed theories concerning the origins of crossmodal correspondences; the component features of a visual stimulus are observed to contribute substantially to its taste expectations. However, the taste associated with a visual stimulus may sometimes deviate from the taste correspondences primed by its constituent parts. This may occur when a new semantic meaning emerges as multiple features are displayed together. Some visual features may even provide contextual cues for observers, thus altering the gustatory information that they associate with an image. A theoretical framework is constructed to help more intuitively predict and conceptualise the overall influence on taste correspondences when visual features are processed together as a combined image.

Temporal perception in multisensory processing is important for an accurate and efficient understanding of the physical world. In general, it is executed in a dynamic environment in our daily lives. In particular, the motion-binding property is important for correctly identifying moving objects in the external environment. However, how this property affects multisensory temporal perception remains unclear. We investigate whether the motion-binding property influences audiovisual temporal integration. The study subjects performed four types of temporal-order judgment (TOJ) task experiments using three types of perception. In Experiment 1, the subjects conducted audiovisual TOJ tasks in the motion-binding condition, between two flashes, and in the simultaneous condition, in which the two flashes are perceived as simultaneous stimuli without motion. In Experiment 2, subjects conducted audiovisual TOJ tasks in the motion-binding condition and the short and long successive interval condition, in which the two stimuli are perceived as successive with no motion. The results revealed that the point of subjective simultaneity (PSS) and the just-noticeable difference (JND) in the motion-binding condition differed significantly from those in the simultaneous and short and long successive interval conditions. Specifically, the PSS in the motion-binding condition was shifted toward a sound-lead stimulus in which the PSS became closer to zero (i.e., physical simultaneity) and the JND became narrower compared to other conditions. This suggests that the motion-binding property contributes to accurate temporal integration in multisensory processing by precisely encoding the temporal order of the physical stimuli.

The Rubber Hand Illusion (RHI) arises through multisensory congruence and informative cues from the most relevant sensory channels. Some studies have explored the RHI phenomenon on the fingers, but none of them modulated the congruence of visuo-tactile and visuo-proprioceptive information by changing the posture of the fingers. This study hypothesizes that RHI induction is possible despite a partial visuo-proprioceptive or visuo-tactile incongruence. With quantitative and qualitative measures, we observed that gradual induction of the sense of body ownership depends on the congruence of multisensory information, with an emphasis on visuo-tactile information rather than visuo-proprioceptive signals. Based on the overall measures, the RHI observed went from stronger to weaker with full congruence; visuo-proprioceptive incongruence and visuo-tactile congruence; visuo-proprioceptive congruence and visuo-tactile incongruence; full incongruence. Our results confirm that congruent visual and tactile mapping is important, though not mandatory, to induce a strong sense of ownership. By changing index finger and thumb postures rather than the rotation of the whole hand, our study investigates the contribution of visuo-proprioception and postural congruence in the field of RHI research. The results are in favor of a probabilistic multisensory integration theory and do not resonate with rules and constraints found in internal body models. The RHI could be illustrated as a continuum: the more multisensory information is congruent, the stronger the RHI.

Atypical sensory processing is now considered a diagnostic feature of autism. Although multisensory integration (MSI) may have cascading effects on the development of higher-level skills such as socio-communicative functioning, there is a clear lack of understanding of how autistic individuals integrate multiple sensory inputs. Multisensory dynamic information is a more ecological construct than static stimuli, reflecting naturalistic sensory experiences given that our environment involves moving stimulation of more than one sensory modality at a time. In particular, depth movement informs about crucial social (approaching to interact) and non-social (avoiding threats/collisions) information. As autistic characteristics are distributed on a spectrum over clinical and general populations, our work aimed to explore the multisensory integration of depth cues in the autistic personality spectrum, using a go/no-go detection task. The autistic profile of 38 participants from the general population was assessed using questionnaires extensively used in the literature. Participants performed a detection task of auditory and/or visual depth moving stimuli compared to static stimuli. We found that subjects with high-autistic traits overreacted to depth movement and exhibited faster reaction times to audiovisual cues, particularly when the audiovisual stimuli were looming and/or were presented at a fast speed. These results provide evidence of sensory particularities in people with high-autistic traits and suggest that low-level stages of multisensory integration could operate differently all along the autistic personality spectrum.

The mechanisms underlying geocentric (orientations of an object or the body relative to 'gravity') and egocentric estimates (object orientation relative to the 'body') have each been examined; however, little is known regarding the association between these estimates, especially when the body is nearly upright. To address this, we conducted two psychophysical experiments. In Experiment 1, participants estimated the direction of a visual line (subjective visual vertical; SVV) and their own body relative to gravity (subjective body tilt; SBT) and the direction of a visual line relative to the body longitudinal axis (subjective visual body axis; SVBA) during a small-range whole-body roll tilt. We evaluated the correlations between performance on each of these tasks as covariates of actual body tilt angles. Our results showed a significant correlation of performance (estimation errors) on the SVBA task with performance on the SBT task but not performance on the SVV task at the group level after adjusting for the actual body tilt angles, suggesting a link between the estimates for SVBA and SBT tasks. To confirm this relationship, in Experiment 2, we further assessed whether manipulating the subjective direction of the body axis by providing visual feedback in the SVBA task subsequently affected SBT performance. We found that feedback in the SVBA task significantly shifted the SBT angles even when the actual body angles were identical. The observed association between SVBA and SBT performance supports at least a partially shared mechanism underlying body tilt and egocentric estimates when the body is nearly upright.

A growing body of experimental research now demonstrates that neurologically normal individuals associate different taste qualities with design features such as curvature, symmetry, orientation, texture and movement. The form of everything from the food itself through to the curvature of the plateware on which it happens to be served, and from glassware to typeface, not to mention the shapes of/on food product packaging have all been shown to influence people's taste expectations, and, on occasion, also their taste/food experiences. Although the origins of shape-taste and other form-taste crossmodal correspondences have yet to be fully worked out, it would appear that shape qualities are occasionally elicited directly. However, more often, there may be a metaphorical attempt to translate the temporal qualities of taste sensations into a spatial analogue. At the same time, emotional mediation may sometimes also play a role in the affinity people experience between shape properties and taste. And finally, it should be acknowledged that associative learning of the relation between packaging shapes, glassware shapes, logos, labels and iconic food forms that commonly co-occur with specific taste properties (i.e., in the case of branded food products) may also play an important role in determining the nature of shape-taste correspondences. Ultimately, however, any attempt to use such shape-taste correspondences to nudge people's behaviour/perception in the real world is made challenging due to the fact that shape properties are associated with multiple qualities, and not just taste.