Pub Date : 2012-01-01DOI: 10.1163/187847612X646587
F. Pons, D. Lewkowicz
It is known that perception of audio–visual (A–V) temporal relations is affected by the type of stimulus used. This includes differences in A–V temporal processing of speech and non-speech events and of native vs. non-native speech. Similar differences have been found early in life, but no studies have investigated infant response to A–V temporal relations in fluent speech. Extant studies (Lewkowicz, 2010) investigating infant response to isolated syllables have found that infants can detect an A–V asynchrony (auditory leading visual) of 666 ms but not lower. Here, we investigated infant response to A–V asynchrony in fluent speech and whether linguistic experience plays a role in responsiveness. To do so, we tested 24 monolingual Spanish-learning and 24 monolingual Catalan-learning 8-month-old infants. First, we habituated the infants to an audiovisually synchronous video clip of a person speaking in Spanish and then tested them in separate test trials for detection of different degrees of A–V asynchrony (audio preceding video by 366, 500 or 666 ms). We found that infants detected A–V asynchronies of 666 and 500 ms and that they did so regardless of linguistic background. Thus, compared to previous results from infant studies with isolated audiovisual syllables, here we found that infants are more sensitive to A–V temporal relations inherent in fluent speech. Furthermore, given that responsiveness to non-native speech narrows during the first year of life, the absence of a language effect suggests that perceptual narrowing of A–V synchrony detection has not completed by 8 months of age.
{"title":"Infant perception of audiovisual synchrony in fluent speech","authors":"F. Pons, D. Lewkowicz","doi":"10.1163/187847612X646587","DOIUrl":"https://doi.org/10.1163/187847612X646587","url":null,"abstract":"It is known that perception of audio–visual (A–V) temporal relations is affected by the type of stimulus used. This includes differences in A–V temporal processing of speech and non-speech events and of native vs. non-native speech. Similar differences have been found early in life, but no studies have investigated infant response to A–V temporal relations in fluent speech. Extant studies (Lewkowicz, 2010) investigating infant response to isolated syllables have found that infants can detect an A–V asynchrony (auditory leading visual) of 666 ms but not lower. Here, we investigated infant response to A–V asynchrony in fluent speech and whether linguistic experience plays a role in responsiveness. To do so, we tested 24 monolingual Spanish-learning and 24 monolingual Catalan-learning 8-month-old infants. First, we habituated the infants to an audiovisually synchronous video clip of a person speaking in Spanish and then tested them in separate test trials for detection of different degrees of A–V asynchrony (audio preceding video by 366, 500 or 666 ms). We found that infants detected A–V asynchronies of 666 and 500 ms and that they did so regardless of linguistic background. Thus, compared to previous results from infant studies with isolated audiovisual syllables, here we found that infants are more sensitive to A–V temporal relations inherent in fluent speech. Furthermore, given that responsiveness to non-native speech narrows during the first year of life, the absence of a language effect suggests that perceptual narrowing of A–V synchrony detection has not completed by 8 months of age.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X646587","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X646938
A. Crisinel, C. Spence
We report a series of experiments investigating crossmodal correspondences between various food-related stimuli (water-based solutions, milk-based flavoured solutions, crisps, chocolate and odours) and sounds varying in pitch and played by four different types of musical instruments. Participants tasted or smelled stimuli before matching them to a musical note. Our results demonstrate that participants preferentially match certain stimuli to specific pitches and instrument types. Through participants’ ratings of the stimuli along a number of dimensions (e.g., pleasantness, complexity, familiarity or sweetness), we explore the psychological dimensions involved in these crossmodal correspondences, using principal components analysis (PCA). While pleasantness seems to play an important role in the choice of instrument associated with chemosensory stimuli, the pitch seems to also depend on the quality of the taste (bitter, salty, sour or sweet). The level at which such crossmodal correspondences might occur, as well as the potential applications of such results, will be discussed.
{"title":"Crossmodal correspondences between chemosensory stimuli and musical notes","authors":"A. Crisinel, C. Spence","doi":"10.1163/187847612X646938","DOIUrl":"https://doi.org/10.1163/187847612X646938","url":null,"abstract":"We report a series of experiments investigating crossmodal correspondences between various food-related stimuli (water-based solutions, milk-based flavoured solutions, crisps, chocolate and odours) and sounds varying in pitch and played by four different types of musical instruments. Participants tasted or smelled stimuli before matching them to a musical note. Our results demonstrate that participants preferentially match certain stimuli to specific pitches and instrument types. Through participants’ ratings of the stimuli along a number of dimensions (e.g., pleasantness, complexity, familiarity or sweetness), we explore the psychological dimensions involved in these crossmodal correspondences, using principal components analysis (PCA). While pleasantness seems to play an important role in the choice of instrument associated with chemosensory stimuli, the pitch seems to also depend on the quality of the taste (bitter, salty, sour or sweet). The level at which such crossmodal correspondences might occur, as well as the potential applications of such results, will be discussed.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X646938","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X646965
J. Velzen
Experimental evidence has shown that the actions we intend to perform influence the way our visual system processes information in the environment, consistent with the considerable overlap observed between brain circuits involved in action and attention. Conceptual thinking about action-perception links in cognitive science is heavily influenced by earlier work that has established that motor preparation causes a shift of attention to the goal of a movement. This sensory enhancement is characterised on a behavioural level by improved detection and discrimination performance at that location, and neurally by larger responses in visual cortex to stimuli presented there. In a series of experiments we examined electrophysiological visual cortex responses (ERPs) to task-irrelevant visual probe stimuli presented at various locations in movement space during preparation of manual reaching movements. The data from these experiments show simultaneous enhanced visual processing of stimuli at the location of the effector about to perform the movement and at the goal of the movement. Further, our data demonstrates that compared to controls, adults with Developmental Coordination Disorder show a markedly different pattern of enhanced visual processing during preparation of more complex reaching movement, i.e., across the body midline. This suggests a specific difficulty in this group in recruiting appropriate preparatory visual mechanism for manual movements, which may be related to the difficulties this group experiences in their daily life.
{"title":"Spatial and temporal dynamics of visual processing during movement preparation: ERP evidence from adults with and without Developmental Coordination Disorder","authors":"J. Velzen","doi":"10.1163/187847612X646965","DOIUrl":"https://doi.org/10.1163/187847612X646965","url":null,"abstract":"Experimental evidence has shown that the actions we intend to perform influence the way our visual system processes information in the environment, consistent with the considerable overlap observed between brain circuits involved in action and attention. Conceptual thinking about action-perception links in cognitive science is heavily influenced by earlier work that has established that motor preparation causes a shift of attention to the goal of a movement. This sensory enhancement is characterised on a behavioural level by improved detection and discrimination performance at that location, and neurally by larger responses in visual cortex to stimuli presented there. In a series of experiments we examined electrophysiological visual cortex responses (ERPs) to task-irrelevant visual probe stimuli presented at various locations in movement space during preparation of manual reaching movements. The data from these experiments show simultaneous enhanced visual processing of stimuli at the location of the effector about to perform the movement and at the goal of the movement. Further, our data demonstrates that compared to controls, adults with Developmental Coordination Disorder show a markedly different pattern of enhanced visual processing during preparation of more complex reaching movement, i.e., across the body midline. This suggests a specific difficulty in this group in recruiting appropriate preparatory visual mechanism for manual movements, which may be related to the difficulties this group experiences in their daily life.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X646965","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X646677
J. DeSouza, Rachel J. Bar
We were interested in examining the time course of the evolution when beginning to learn a motor habit and it’s associated neural functional changes in the brain. To accomplish this we employed five professional dancers that were scanned using a within subjects design. Each dancer participated in four fMRI (functional magnetic resonance imaging) scanning sessions over the training and learning of a dance to a 1 min piece of music employing a typical blocked design (5 epochs with alternations of a 30-s fixation period). We also tested five control subjects that had dance experience but did not learn the dance to this music. Subjects were asked to visualize dancing while listening to a piece of music. At the first scanning session, only 4 rehearsals of the piece (initial acquisition phase) were studied. The control subjects were also tested at this time period but they had no rehearsals and had no visual exposure to the music before scanning. The second scanning session occurred one week later, after a total of 9 rehearsals. The third scanning session was completed 7 weeks after initial acquisition of the dance (the dance was performed a total of 16 times after initial training). Thus in total there were 22 scanning sessions using 10 subjects. Additionally a control motor scan was performed in each scanning session to activate motor regions that should not change activation patterns across all scanning sessions. Results revealed a significant increase of BOLD signal, across the sessions in a network of brain regions including bilateral auditory cortex to supplementary motor cortex. These results suggest that as we learn a motor sequence from music, greater neuronal activity occurs and we discuss the potential neural network involved in dance and its implications towards alternative neural regions that are potentially recruited during dance therapy.
{"title":"The effects of rehearsal on auditory cortex: An fMRI study of the putative neural mechanisms of dance therapy","authors":"J. DeSouza, Rachel J. Bar","doi":"10.1163/187847612X646677","DOIUrl":"https://doi.org/10.1163/187847612X646677","url":null,"abstract":"We were interested in examining the time course of the evolution when beginning to learn a motor habit and it’s associated neural functional changes in the brain. To accomplish this we employed five professional dancers that were scanned using a within subjects design. Each dancer participated in four fMRI (functional magnetic resonance imaging) scanning sessions over the training and learning of a dance to a 1 min piece of music employing a typical blocked design (5 epochs with alternations of a 30-s fixation period). We also tested five control subjects that had dance experience but did not learn the dance to this music. Subjects were asked to visualize dancing while listening to a piece of music. At the first scanning session, only 4 rehearsals of the piece (initial acquisition phase) were studied. The control subjects were also tested at this time period but they had no rehearsals and had no visual exposure to the music before scanning. The second scanning session occurred one week later, after a total of 9 rehearsals. The third scanning session was completed 7 weeks after initial acquisition of the dance (the dance was performed a total of 16 times after initial training). Thus in total there were 22 scanning sessions using 10 subjects. Additionally a control motor scan was performed in each scanning session to activate motor regions that should not change activation patterns across all scanning sessions. Results revealed a significant increase of BOLD signal, across the sessions in a network of brain regions including bilateral auditory cortex to supplementary motor cortex. These results suggest that as we learn a motor sequence from music, greater neuronal activity occurs and we discuss the potential neural network involved in dance and its implications towards alternative neural regions that are potentially recruited during dance therapy.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X646677","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427548","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X647432
V. Bellan, C. Reverberi, A. Gallace
Several studies have shown that watching one’s own body part improves tactile acuity and discrimination abilities for stimuli presented on that location. In a series of experiments we asked the participants to localize tactile stimuli presented on the left or right arm. In Experiment 1 the participants were not allowed to watch their body, but they could see another person’s left arm via a LCD display. This arm could be touched or not during the presentation of the stimuli. We found that when the participants saw the arm on the screen being touched, their responses to the tactile stimuli presented on the left and on the right arm were faster and more accurate than when the arm on the screen was approached but not touched. Critically, we did not find any illusion of ownership related to the hand seen on the screen. In Experiments 2 and 3 we varied the position of the screen with respect to the participant’s body midline and the image displayed on it (an arm or an object of equal size). The participants gave faster responses when an object rather than a hand was displayed on the screen. Moreover, the responses were slower when the hand on the screen was placed in front of the participants, as compared to any other position. Taken together the results of our experiments would seem to suggest that watching touch activates multisensory mechanisms responsible for alerting people regarding the possible presence of tactile stimuli on the body surface.
{"title":"Watching touch increases people’s alertness to tactile stimuli presented on the body surface","authors":"V. Bellan, C. Reverberi, A. Gallace","doi":"10.1163/187847612X647432","DOIUrl":"https://doi.org/10.1163/187847612X647432","url":null,"abstract":"Several studies have shown that watching one’s own body part improves tactile acuity and discrimination abilities for stimuli presented on that location. In a series of experiments we asked the participants to localize tactile stimuli presented on the left or right arm. In Experiment 1 the participants were not allowed to watch their body, but they could see another person’s left arm via a LCD display. This arm could be touched or not during the presentation of the stimuli. We found that when the participants saw the arm on the screen being touched, their responses to the tactile stimuli presented on the left and on the right arm were faster and more accurate than when the arm on the screen was approached but not touched. Critically, we did not find any illusion of ownership related to the hand seen on the screen. In Experiments 2 and 3 we varied the position of the screen with respect to the participant’s body midline and the image displayed on it (an arm or an object of equal size). The participants gave faster responses when an object rather than a hand was displayed on the screen. Moreover, the responses were slower when the hand on the screen was placed in front of the participants, as compared to any other position. Taken together the results of our experiments would seem to suggest that watching touch activates multisensory mechanisms responsible for alerting people regarding the possible presence of tactile stimuli on the body surface.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X647432","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427651","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X647126
Brendan Cullen, F. Newell
Major findings in attractiveness such as the role of averageness and symmetry have emerged primarily from neutral static visual stimuli. However it has increasingly been shown that ratings of attractiveness can be modulated within unisensory and multisensory modes by factors including emotional expression or by additional information about the person. For example, previous research has indicated that humorous individuals are rated as more desirable than their non-humorous equivalents (Bressler and Balshine, 2006). In two experiments we measured within and cross-sensory modulation of the attractiveness of unfamiliar faces. In Experiment 1 we examined if manipulating the number and type of expressions shown across a series of images of a person influences the attractiveness rating for that person. Results indicate that for happy expressions, ratings of attractiveness gradually increase as the proportional number of happy facial expressions increase, relative to the number of neutral expressions. In contrast, an increase in the proportion of angry expressions was not assocated with an increase in attractiveness ratings. In Experiment 2 we investigated if perceived attractiveness can be influenced by multisensory information provided during exposure to the face image. Ratings are compared across face images which were presented with or without voice information. In addition we provided either an auditory emotional cue (e.g., laughter) or neutral (e.g., coughing) cue to assess whether social information affects perceived attractiveness. Results shows that multisensory information about a person can increase attractiveness ratings, but that the emotional content of the cross-modal information can effect preference for some faces over others.
{"title":"Within and cross-sensory interactions in the perceived attractiveness of unfamiliar faces","authors":"Brendan Cullen, F. Newell","doi":"10.1163/187847612X647126","DOIUrl":"https://doi.org/10.1163/187847612X647126","url":null,"abstract":"Major findings in attractiveness such as the role of averageness and symmetry have emerged primarily from neutral static visual stimuli. However it has increasingly been shown that ratings of attractiveness can be modulated within unisensory and multisensory modes by factors including emotional expression or by additional information about the person. For example, previous research has indicated that humorous individuals are rated as more desirable than their non-humorous equivalents (Bressler and Balshine, 2006). In two experiments we measured within and cross-sensory modulation of the attractiveness of unfamiliar faces. In Experiment 1 we examined if manipulating the number and type of expressions shown across a series of images of a person influences the attractiveness rating for that person. Results indicate that for happy expressions, ratings of attractiveness gradually increase as the proportional number of happy facial expressions increase, relative to the number of neutral expressions. In contrast, an increase in the proportion of angry expressions was not assocated with an increase in attractiveness ratings. In Experiment 2 we investigated if perceived attractiveness can be influenced by multisensory information provided during exposure to the face image. Ratings are compared across face images which were presented with or without voice information. In addition we provided either an auditory emotional cue (e.g., laughter) or neutral (e.g., coughing) cue to assess whether social information affects perceived attractiveness. Results shows that multisensory information about a person can increase attractiveness ratings, but that the emotional content of the cross-modal information can effect preference for some faces over others.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X647126","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X647450
L. Piwek, K. Petrini, F. Pollick
Multimodal perception of emotions has been typically examined using displays of a solitary character (e.g., the face–voice and/or body–sound of one actor). We extend investigation to more complex, dyadic point-light displays combined with speech. A motion and voice capture system was used to record twenty actors interacting in couples with happy, angry and neutral emotional expressions. The obtained stimuli were validated in a pilot study and used in the present study to investigate multimodal perception of emotional social interactions. Participants were required to categorize happy and angry expressions displayed visually, auditorily, or using emotionally congruent and incongruent bimodal displays. In a series of cross-validation experiments we found that sound dominated the visual signal in the perception of emotional social interaction. Although participants’ judgments were faster in the bimodal condition, the accuracy of judgments was similar for both bimodal and auditory-only conditions. When participants watched emotionally mismatched bimodal displays, they predominantly oriented their judgments towards the auditory rather than the visual signal. This auditory dominance persisted even when the reliability of auditory signal was decreased with noise, although visual information had some effect on judgments of emotions when it was combined with a noisy auditory signal. Our results suggest that when judging emotions from observed social interaction, we rely primarily on vocal cues from the conversation, rather then visual cues from their body movement.
{"title":"Auditory signal dominates visual in the perception of emotional social interactions","authors":"L. Piwek, K. Petrini, F. Pollick","doi":"10.1163/187847612X647450","DOIUrl":"https://doi.org/10.1163/187847612X647450","url":null,"abstract":"Multimodal perception of emotions has been typically examined using displays of a solitary character (e.g., the face–voice and/or body–sound of one actor). We extend investigation to more complex, dyadic point-light displays combined with speech. A motion and voice capture system was used to record twenty actors interacting in couples with happy, angry and neutral emotional expressions. The obtained stimuli were validated in a pilot study and used in the present study to investigate multimodal perception of emotional social interactions. Participants were required to categorize happy and angry expressions displayed visually, auditorily, or using emotionally congruent and incongruent bimodal displays. In a series of cross-validation experiments we found that sound dominated the visual signal in the perception of emotional social interaction. Although participants’ judgments were faster in the bimodal condition, the accuracy of judgments was similar for both bimodal and auditory-only conditions. When participants watched emotionally mismatched bimodal displays, they predominantly oriented their judgments towards the auditory rather than the visual signal. This auditory dominance persisted even when the reliability of auditory signal was decreased with noise, although visual information had some effect on judgments of emotions when it was combined with a noisy auditory signal. Our results suggest that when judging emotions from observed social interaction, we rely primarily on vocal cues from the conversation, rather then visual cues from their body movement.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X647450","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427740","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X647298
Ragnhild Eg, D. Behne
Perceived synchrony varies depending on the audiovisual event. Typically, asynchrony is tolerated at greater lead- and lag-times for speech and music than for action events. The tolerance for asynchrony in speech has been attributed to the unity assumption, which proposes a bonding of auditory and visual speech cues through associations in several dimensions. However, the variations in synchrony perception for different audiovisual events may simply be related to their complexity; where speech and music fluctuate naturally, actions involve isolated events and anticipated moments of impact. The current study measured perception of synchrony for long (13 s) and short (1 s) variants of three types of stimuli: (1) action, represented by a game of chess, (2) music, played by a drummer and (3) speech, presented by an anchorwoman in a newscast. The long variants allowed events to play out with their natural dynamics, whereas short variants offered controlled and predictable single actions or events, selected from the longer segments. Results show that among the long stimuli, lead asynchrony was detected sooner for speech than for chess. This contrasts both with previous research and our own predictions, although it may be related to characteristics of the selected chess scene. Interestingly, tolerance to asynchrony was generally greater for short, than for long, stimuli, especially for speech. These findings suggest that the dynamics of complex events cannot account for previously observed differences in synchrony perception between speech and action events.
{"title":"Short and sweet, or long and complex? Perceiving temporal synchrony in audiovisual events","authors":"Ragnhild Eg, D. Behne","doi":"10.1163/187847612X647298","DOIUrl":"https://doi.org/10.1163/187847612X647298","url":null,"abstract":"Perceived synchrony varies depending on the audiovisual event. Typically, asynchrony is tolerated at greater lead- and lag-times for speech and music than for action events. The tolerance for asynchrony in speech has been attributed to the unity assumption, which proposes a bonding of auditory and visual speech cues through associations in several dimensions. However, the variations in synchrony perception for different audiovisual events may simply be related to their complexity; where speech and music fluctuate naturally, actions involve isolated events and anticipated moments of impact. The current study measured perception of synchrony for long (13 s) and short (1 s) variants of three types of stimuli: (1) action, represented by a game of chess, (2) music, played by a drummer and (3) speech, presented by an anchorwoman in a newscast. The long variants allowed events to play out with their natural dynamics, whereas short variants offered controlled and predictable single actions or events, selected from the longer segments. Results show that among the long stimuli, lead asynchrony was detected sooner for speech than for chess. This contrasts both with previous research and our own predictions, although it may be related to characteristics of the selected chess scene. Interestingly, tolerance to asynchrony was generally greater for short, than for long, stimuli, especially for speech. These findings suggest that the dynamics of complex events cannot account for previously observed differences in synchrony perception between speech and action events.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X647298","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64427824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X647397
N. N. Chang, Alex K. Malone, T. Hullar
Imbalance among patients with vestibular hypofunction has been related to inadequate compensatory eye movements in response to head movements. However, symptoms of imbalance might also occur due a temporal mismatch between vestibular and other balance-related sensory cues. This temporal mismatch could be reflected in a widened temporal binding window (TBW), or the length of time over which simultaneous sensory stimuli may be offset and still perceived as simultaneous. We hypothesized that decreased vestibular input would lead to a widening of the temporal binding window. We performed whole-body rotations about the earth-vertical axis following a sinusoidal trajectory at 0.5 Hz with a peak velocity of 60°/s in four normal subjects. Dichotic auditory clicks were presented through headphones at various phases relative to the rotations. Subjects were asked to indicate whether the cues were synchronous or asynchronous and the TBW was calculated. We then simulated decreased vestibular input by rotating at diminished peak velocities of 48, 24 and 12°/s in four normal subjects. TBW was calculated between ±1 SD away from the mean on the psychometric curve. We found that the TBW increases as amplitude of rotation decreases. Average TBW of 251 ms at 60°/s increased to 309 ms at 12°/s. This result leads to the novel conclusion that changes in temporal processing may be a mechanism for imbalance in patients with vestibular hypofunction.
{"title":"Changes in temporal binding related to decreased vestibular input","authors":"N. N. Chang, Alex K. Malone, T. Hullar","doi":"10.1163/187847612X647397","DOIUrl":"https://doi.org/10.1163/187847612X647397","url":null,"abstract":"Imbalance among patients with vestibular hypofunction has been related to inadequate compensatory eye movements in response to head movements. However, symptoms of imbalance might also occur due a temporal mismatch between vestibular and other balance-related sensory cues. This temporal mismatch could be reflected in a widened temporal binding window (TBW), or the length of time over which simultaneous sensory stimuli may be offset and still perceived as simultaneous. We hypothesized that decreased vestibular input would lead to a widening of the temporal binding window. We performed whole-body rotations about the earth-vertical axis following a sinusoidal trajectory at 0.5 Hz with a peak velocity of 60°/s in four normal subjects. Dichotic auditory clicks were presented through headphones at various phases relative to the rotations. Subjects were asked to indicate whether the cues were synchronous or asynchronous and the TBW was calculated. We then simulated decreased vestibular input by rotating at diminished peak velocities of 48, 24 and 12°/s in four normal subjects. TBW was calculated between ±1 SD away from the mean on the psychometric curve. We found that the TBW increases as amplitude of rotation decreases. Average TBW of 251 ms at 60°/s increased to 309 ms at 12°/s. This result leads to the novel conclusion that changes in temporal processing may be a mechanism for imbalance in patients with vestibular hypofunction.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X647397","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64428100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-01-01DOI: 10.1163/187847612X647568
C. Nahanni, J. Deonarine, M. Paré, K. Munhall
The sight of a talker’s face dramatically influences the perception of auditory speech. This effect is most commonly observed when subjects are presented audiovisual (AV) stimuli in the presence of acoustic noise. However, the magnitude of the gain in perception that vision adds varies considerably in published work. Here we report data from an ongoing study of individual differences in AV speech perception when English words are presented in an acoustically noisy background. A large set of monosyllablic nouns was presented at 7 signal-to-noise ratios (pink noise) in both AV and auditory-only (AO) presentation modes. The stimuli were divided into 14 blocks of 25 words and each block was equated for spoken frequency using the SUBTLEXus database (Brysbaert and New, 2009). The presentation of the stimulus blocks was counterbalanced across subjects for noise level and presentation. In agreement with Sumby and Pollack (1954), the accuracy of both AO and AV increase monotonically with signal strength with the greatest visual gain being when the auditory signal was weakest. These average results mask considerable variability due to subject (individual differences in auditory and visual perception), stimulus (lexical type, token articulation) and presentation (signal and noise attributes) factors. We will discuss how these sources of variance impede comparisons between studies.
看到说话人的脸会极大地影响对听觉语言的感知。当受试者在有噪声的情况下接受视听(AV)刺激时，最常观察到这种效应。然而，在已发表的作品中，视觉增加的感知增益的幅度差异很大。在这里，我们报告了一项正在进行的研究的数据，该研究是关于在嘈杂的背景下呈现英语单词时，AV语音感知的个体差异。在AV和纯听觉(AO)两种呈现模式下，以7种信噪比(粉红噪声)呈现大量单音节名词。这些刺激被分成14个块，每块25个单词，每个块使用精妙的数据库被等同于口语频率(Brysbaert and New, 2009)。刺激块的呈现在噪声水平和呈现上是平衡的。Sumby和Pollack(1954)认为，AO和AV的准确度随信号强度的增加而单调增加，听觉信号最弱时视觉增益最大。这些平均结果掩盖了由于主体(听觉和视觉感知的个体差异)，刺激(词汇类型，标记发音)和表示(信号和噪声属性)因素造成的相当大的差异。我们将讨论这些方差来源如何阻碍研究之间的比较。
{"title":"Sources of variance in the audiovisual perception of speech in noise","authors":"C. Nahanni, J. Deonarine, M. Paré, K. Munhall","doi":"10.1163/187847612X647568","DOIUrl":"https://doi.org/10.1163/187847612X647568","url":null,"abstract":"The sight of a talker’s face dramatically influences the perception of auditory speech. This effect is most commonly observed when subjects are presented audiovisual (AV) stimuli in the presence of acoustic noise. However, the magnitude of the gain in perception that vision adds varies considerably in published work. Here we report data from an ongoing study of individual differences in AV speech perception when English words are presented in an acoustically noisy background. A large set of monosyllablic nouns was presented at 7 signal-to-noise ratios (pink noise) in both AV and auditory-only (AO) presentation modes. The stimuli were divided into 14 blocks of 25 words and each block was equated for spoken frequency using the SUBTLEXus database (Brysbaert and New, 2009). The presentation of the stimulus blocks was counterbalanced across subjects for noise level and presentation. In agreement with Sumby and Pollack (1954), the accuracy of both AO and AV increase monotonically with signal strength with the greatest visual gain being when the auditory signal was weakest. These average results mask considerable variability due to subject (individual differences in auditory and visual perception), stimulus (lexical type, token articulation) and presentation (signal and noise attributes) factors. We will discuss how these sources of variance impede comparisons between studies.","PeriodicalId":49553,"journal":{"name":"Seeing and Perceiving","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1163/187847612X647568","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64428211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：