Exp Brain Res (2009) 198:195–208 DOI 10.1007/s00221-009-1877-3
R ES EA R C H A R TI CLE
Audiovisual temporal capture underlies Xash fusion Takahiro Kawabe
Received: 4 August 2008 / Accepted: 21 May 2009 / Published online: 12 June 2009 © Springer-Verlag 2009
Abstract When sequential visual Xashes are accompanied by a lower number of sequential auditory pulses, the perceived number of visual Xashes is lower than the actual number, an illusion termed ‘Xash fusion’. We examined whether temporal capture of Xashes by pulses underlay Xash fusion. One of the visual Xashes was given a luminance increment, and observers reported which Xash had the luminance increment. Results showed that the pulse strongly captured the Xashes in its temporal vicinity, resulting in Xash fusion. Moreover, when one of the successive pulses was given a higher frequency than others, the luminance increment was perceptually paired with the pulse with the higher frequency. The pairing of audiovisual features disappeared when the temporal pattern of the pulse frequency was diYcult for the observer to anticipate. These data indicate that Xash fusion is caused by temporal capture of Xashes by the pulse, and that feature matching between auditory and visual signals also contributes to the modulation of perceived temporal structure of Xashes during Xash fusion. Keywords Audiovisual integration · Flash fusion · Sound-induced Xash illusion · Audiovisual temporal capture
T. Kawabe (&) Kyushu University, 6-19-1 Hakozaki, Higashi-ku, Fukuoka 812-8581, Japan e-mail:
[email protected]
Introduction Audition and vision are not separate modalities (Shimojo and Shams 2001). Previous studies have focused on the interactions of audition and vision in temporal (Kanai et al. 2007; Morein-Zamir et al. 2003; Recanzone 2003; Shipley 1964; Vroomen and de Gelder 2004) and spatial (Bertelson and Aschersleben 1998) dimensions, in the perception of event perception (Kawabe and Miura 2006; Sekuler et al. 1997; Watanabe and Shimojo 2001), and in the perception of apparent motion (Kawabe et al. 2008; MateeV et al. 1985; Soto-Faraco et al. 2002; Zapparoli and Reatto 1969). Such integration of signals from diVerent modalities has been shown to be performed in a statistically optimal fashion (Andersen et al. 2005; Burr and Alais 2006; Sato et al. 2007; Shams et al. 2005). In a seminal study by Shams and colleagues (2000, 2002), it was demonstrated that a single visual Xash was perceived as multiple Xashes when accompanied by multiple auditory pulses. This illusory perception of visual Xashes in the presence of auditory pulses was termed the ‘illusory Xash eVect’ (Shimojo and Shams 2001) or ‘soundinduced Xash illusion’ (Shams et al. 2005). Surprisingly, the sound-induced Xash illusion occurs even when the retinal position of multiple sequential Xashes is laterally shifted across time (Kamitani and Shimojo 2001). The neural correlates of the sound-induced Xash illusion was examined by EEG studies (Bhattacharya et al. 2002; Shams et al. 2001; Mishra et al. 2007) and fMRI (Watkins et al. 2006), which indicated that auditory pulses accompanying the visual Xashes induced the activity modulation of the primary and the extra-striate visual cortex, resulting in the sound-induced illusory Xash. These data imply that sensory modulation, not decision alternation, may underlie the sound-induced Xash illusion, which is supported by recent
123
196
psychophysical studies (Berger et al. 2003; McCormick and Mammassian 2008). In contrast to sound-induced Xash illusion, in the so-called ‘Xash fusion’ (Andersen et al. 2004, 2005; Shams et al. 2005; Mishra et al. 2008; Watkins et al. 2006), the number of sequential visual Xashes is often perceived to be fewer than the actual number when the Xashes are accompanied by sequential auditory pulses of fewer number than the Xashes. Although Xash fusion is as much a vivid phenomenon as sound-induced Xash illusion, the speciWc mechanism underlying Xash fusion remains unclear. To date, mathematical models for the Xash fusion and Wssion illusions have been described (Andersen et al. 2005; Shams et al. 2005). Moreover, using EEG analysis Mishra and colleagues (2008) suggested that a dynamic interplay between the visual and the polysensory cortex underlie the Xash fusion. However, the kind of processing underlying Xash fusion has not been assessed. In my hypothesis, Xash fusion occurs because temporal separation between Xashes is perceptually compressed by audiovisual temporal capture, and hence the perceptual system cannot individuate each Xash. Audiovisual signals are integrated within a narrow temporal window (i.e., 0.05) both when the luminance increment was added to the second and third Xash. In the H-L-L condition, but not the L-H-H condition, the proportion of trials in which the luminance increment was perceived in the Wrst Xash was signiWcantly diVerent from chance level when the luminance increment was added to the second Xash. Discussion In this experiment, feature-based audiovisual temporal capture during Xash fusion disappeared (except in one case, H-L-L pulse condition when the second Xash had the luminance increment) when the frequency pattern of pulses could not be anticipated, suggesting that the expectation for feature pairs plays a critical role in the feature matching in Xash fusion. These data support the previous Wnding that feature-based audiovisual matching requires knowledge of the pairing of audiovisual features (Fujisaki and Nishida 2008), and indicate that the results of Experiment 2 did not arise simply from the bias to report the frequency patterns of pulses as the luminance pattern of Xashes. Even though the feature matching between the luminance increment of the Xash and frequency increment of the pitch was not observed, the Xash fusion itself was strongly observed, indicating that feature matching was an additional phenomenon modulating the matching of audiovisual signals during Xash fusion. Thus, irrespective of the frequency of the pulses, the temporal capture of two Xashes by a single pulse may cause Xash fusion.
Exp Brain Res (2009) 198:195–208
203
Fig. 4 Experiment 3 results. The proportions of trials in which the luminance increment was perceived at the Wrst, second, and third illusory Xashes were each calculated separately for each of the pulse conditions (LHH, LHL, HLL, and HLH). The proportions obtained in the condition where the luminance increment was presented at the second physical Xash were separately calculated from those obtained in the condition where the luminance increment was presented at the third physical Xash. The former and latter proportions are plotted in a and b, respectively
Experiment 4 The aim of this experiment was to examine another example of feature-based temporal capture during Xash fusion. A newly devised pattern of visual Xashes and auditory pulses was used: two luminance patterns of Xashes, D-D-B-B and B-B-D-D, and three pitch patterns of pulses, H-H-L, L-H-H, and L-L-L (Fig. 5a). In this case, if the feature-based temporal capture worked in the Xash fusion, D-D-B-B and B-B-D-D Xashes should be perceived as (1) D-D-B and B-B-D when H-H-L pulses were accompanied by the
Fig. 5 a The temporal properties of stimuli used in Experiment 4. ‘D’ and ‘B’ indicate the dark and bright Xashes, respectively. There were two conditions of luminance patterns of the Xashes: ‘D-D-B-B’ indicates that the Wrst and second Xashes were dark and the third and the forth Xashes were bright. ‘B-B-D-D’ indicates that the Wrst and second
Xashes, and (2) D-B-B and B-D-D when L-H-H pulses were accompanied by the Xashes (Fig. 2c). Materials and methods Observers Four observers including the author (TK) participated in this experiment. Except for the author, the observers were not aware of speciWc purposes of the experiment. All participants reported they had normal or corrected-to-normal
Xashes were bright and the third and the forth Xashes were dark. b Experiment 4 results. The proportions of trials in which the two dark and one bright Xashes were perceived were separately calculated for each of the pulse conditions (HHL, LLL, and LHH), and are plotted separately for the two luminance patterns of Xashes
123
204
visual acuity, and normal hearing ability. Participants other than the author were paid ¥800 for their participation. Stimuli The stimuli were essentially identical to those used in Experiment 3, except for the following. Two diVerent luminance patterns of the Xashes were used: D-D-B-B and B-BD-D, where D and B indicate the dark and bright Xashes, respectively (Fig. 5a). The luminance values of the dark and bright Xashes were 4.0 cd/m2 and 25.4 cd/m2, respectively. Three kinds of frequency patterns were also used: H-H-L, L-L-L, and L-H-H. H indicates the high-frequency pulse (3,000 Hz) and L indicates the low-frequency pulse (500 Hz). Procedure The procedures were identical to those in Experiment 3, except for the following. The two luminance patterns of the Xashes and three frequency patterns of pulses were factorially paired, and one of the pairs was presented on each trial. The observers were Wrst required to report whether the number of Xashes presented was three or four. In the trial where they reported three Xashes, they were further asked to report which of dark or bright Xashes was presented twice. Each observer performed 120 trials consisting of 2 (luminance patterns) £ 3 (pulse frequency patterns) £ 20 (repetitions). The order of trials was randomized across observers. Results The proportion of trials in which the three Xashes were perceived was 99%, robustly showing the Xash fusion. For the trials in which observers reported three Xashes, the proportion of trials in which dark Xashes were presented more frequently than the bright Xashes can be seen in Fig. 5b. Using a two-way ANOVA for the proportions with the luminance pattern and frequency pattern as factors, there was a signiWcant interaction between luminance patterns of Xashes and frequency patterns of pulses [F(2,6) = 27.174, p < 0.0001]. The simple main eVect of pitch pattern was signiWcant when the luminance pattern of Xashes was D-D-B-B [F(2,12) = 22.376, p < 0.0001] or B-B-D-D [F (2, 12) = 17.721, p < 0.0003]. The multiple comparison tests (Ryan’s method) for the simple main eVect of luminance pattern D-D-B-B showed that the proportion in the H-H-L pitch pattern was signiWcantly larger than that in the L-H-H pitch pattern (p < 0.05), and that the proportion in the L-L-L pitch pattern was signiWcantly larger than that in the L-H-H pitch pattern (p < 0.05). The multiple comparison tests (Ryan’s method) for the simple main eVect of the
123
Exp Brain Res (2009) 198:195–208
luminance pattern B-B-D-D showed that the proportion in the H-H-L pitch pattern was signiWcantly smaller than that in the L-H-H pitch pattern (p < 0.05), and that the proportion in the LHH pitch pattern was signiWcantly smaller than that in the L-L-L pitch pattern (p < 0.05). Discussion Interestingly, these data suggest that the perceived luminance pattern in Xash fusion was strongly related to the auditory pulse pattern, and therefore that the sensory system seems to match the temporal pattern of visual feature with the one of auditory feature. Unlike Experiment 3, in this experiment the observers could anticipate the frequency of the third pulse when they were presented the second pulse, allowing room for the top-down processing to work eVectively. Thus, these data support the concept that feature-based temporal capture contributes to Xash fusion.
Experiment 5 Although observers perceived Xash fusion vividly in the present study, alternatively, it is possible that observers failed to tell three from four Xashes, and judgment of the number of Xashes was simply inXuenced by the number of pulses. Furthermore, as there were dual tasks required of the observers in the previous experiments (i.e., the Wrst task was to report the number of Xashes and the second task was to report the position of luminance increment), it may have been diYcult to discriminate the number of Xashes under such a high attentional load condition. Thus, to examine these possibilities, the present experiment used three and four Xashes without luminance increment, and three kinds of pulse conditions (zero, three, and four pulses). The Xashes and pulses were alternatively presented as in the previous experiments. The observers’ task was to determine whether the number of Xashes presented was three or four. If the observers could not discriminate three from four Xashes, the reported number of Xashes in zero pulse condition would be similar between the three and four Xash condition. Furthermore, if the frequent Xash fusion observed in the previous experiments was due to the dual task, the Xash fusion would be less frequent in this experiment as the task of observers was changed from dual to single. Materials and methods Observers Five observers including the author (TK) participated in this experiment. Except for the author, the observers were not aware of speciWc purposes of the experiment.
Exp Brain Res (2009) 198:195–208
205
All participants reported they had normal or correctedto-normal visual acuity, and normal hearing ability. Participants other than the author were paid ¥800 for their participation. Stimuli The stimuli were essentially identical to those used in Experiment 2, except for the following. In this experiment, three or four Xashes were presented in each trial without luminance increment. Moreover, those Xashes were accompanied with zero, three, and four auditory pulses. The frequency of pulses was 500 Hz and their duration was 5 ms. As in the previous experiment, Xashes and pulses were presented in temporal alternation. Procedure The procedure was identical to in Experiment 1, except for the following. The task of the observers was to simply report whether the number of the Xashes presented was three or four. Each observer performed 120 trials consisting of 2 (numbers of Xashes) £ 3 (numbers of pulses) £ 15 (repetitions). Results The proportions of trials in which the observers reported three Xashes can be seen in Fig. 6. Using a two-way ANOVA for the proportions with the number of Xashes and the number of pulses as factors, there was a signiWcant main eVect of the number of Xashes [F(1,4) = 14.912, p < 0.02] and the number of pulses [F(2,8) = 22.139, p < 0.001]. The interaction between the two factors was also signiWcant [F(2,8) = 9.449, p < 0.01]. The simple main eVects of the number of Xashes were signiWcant when the number of pulses was zero and four (p < 0.05), but not when the number of pulses was three (p > 0.05). When the number of Xashes was three, the proportion in the threepulse condition was signiWcantly diVerent from the proportion in the four-pulse condition (p < 0.05). When the number of Xashes was four, the proportion in the three-pulse condition was signiWcantly diVerent from the proportion in the zeroand four-pulse conditions (p < 0.05). Discussion The results of this experiment indicate that at least in the stimulus setting of this study, observers could clearly discriminate the number of Xashes between three and four. Furthermore, a strong Xash fusion was observed even under the single task of reporting the number of Xashes; the proportion in which the number of Xashes was three
Fig. 6 Experiment 5 results. The proportions of trials in which three Xashes were perceived were separately calculated for each of pulse number conditions (zero, three, and four pulses) and each of Xash number conditions. The error bars denote the standard errors of the mean
was high (92%) when actual number of Xashes was four and the number of pulses was three. Therefore, these data suggest that the Xash fusion observed in the previous experiments were free from an inability to discriminate three from four Xashes, and from the artifact due to the dual-task setting. The incidence of Xash fusion in the present study was relatively high (92%) compared with the previously described 42% (Watkins et al. 2006) and 57% (Andersen et al. 2004). This discrepancy is likely due to the diVerence in visual stimuli, as the luminance contrast of Xash in the present study (25.4 cd/m2 Xash against a 9.8 cd/m2 background; 44% of Michelson contrast) was far smaller than that used in previous studies (148 cd/m2 Xash against a 2.07 cd/m2 background, leading to 97% of Michelson contrast, in Andersen et al. 2004; and 420 cd/m2 Xash against a 30 cd/m2 background, leading to 87% Michelson contrast, in Watkins et al. 2006). Thus, the visibility of the Xash in the present study was likely lower than that in the previous reports. When the visibility/reliability of Xashes is low, the weight of auditory signals in audiovisual interaction gets larger (Battaglia et al. 2003; Burr and Alais 2006), suggesting that the stimuli with lower visibility in the present study was likely inXuenced by auditory pulses, resulting in the higher incidence of Xash fusion. Moreover, inconsistent with previous studies, the rotating Xash was used in the present study, and this might also have contributed to increased fusion eVects compared to previous literature.
123
206
General discussion The results of the present study are consistent with the concept that Xash fusion occurs as a result of temporal capture. Although previous studies have examined the stimulus conditions (Andersen et al. 2004), neurophysiological substrates (Bhattacharya et al. 2002; Mishra et al. 2007, 2008; Shams et al. 2001; Watkins et al. 2006), and computational aspects (Shams et al. 2005) for the Xash fusion, they lacked a speciWc mechanism of audiovisual integration contributing to the Xash fusion. The present study demonstrated that the temporal capture of the visual Xash by auditory pulse in the temporal vicinity contributes to the Xash fusion (Experiment 1), that the appearance of the luminance pattern during Xash fusion is consistent with feature-based audiovisual temporal capture (Experiments 2, 3, and 4), and that an inability to discriminate the number of Xashes was not involved in these results (Experiment 5). In Experiment 1, the perceived position of luminance increment was ambiguous when the luminance increment was added to second Xash, and even in this situation, a strong Xash fusion was observed. In this case, multiple temporal capture likely occurred between the Wrst and second pulses or between the second and third pulses. That is, unless the auditory pulse had the feature change which could interact with the visual feature change, the visual feature change (such as luminance increment) is unlikely perceived separately in the rapid succession of Xashes without feature change, and consequently, is also subject to Xash fusion due to temporal capture based on temporal proximity. Likewise, in the non-unique condition of Experiment 2, the luminance increment in the second Xash was perceived to be in the Wrst Xash more frequently than in the second and third Xashes, and the luminance increment in the third Xash was perceived to be in the third Xash more frequently than in the Wrst and second Xashes. These data suggest that when no unique feature was given to the auditory pulse, multiple temporal capture occurs even between the Xashes with and without luminance increment. In Experiment 1, it is also possible that the Wrst auditory pulse did not capture the Wrst Xash, while the second auditory pulse captured the second and third Xashes, or, that the Wrst auditory pulse captured the Wrst and second Xashes, while the second auditory pulse did not capture the third pulse. Flash fusion is expected to occur even in these cases, and as such these possibilities cannot be excluded. However, previous studies have shown that temporal capture did occur even when the visual and auditory signals were separated from each other by 100 ms (Morein-Zamir et al. 2003; Vroomen and de Gelder 2004), and aVected the visibility of target under the object substitution masking (Vroomen and Keetels 2009). In the present study, the temporal separation between the Xash and the pulse was Wxed at 60 ms, and it is
123
Exp Brain Res (2009) 198:195–208
likely that in this temporal course the visual Xash was eVectively captured by the auditory pulse. It was previously reported in some studies that Xash fusion occurs even when one of the Xashes was temporally synchronized with one of the pulses. For example, Andersen et al. (2004, 2005) and Watkins et al. (2006) reported that Xash fusion occurred even when the initial pulse and the initial Xash were physically synchronized, suggesting that the explanation for Xash fusion with audiovisual temporal capture is inappropriate. However, in those studies the stimulus onset asynchrony between Xashes (and pulses) was 67 ms (Andersen et al. 2004) and 46 ms (Watkins et al. 2006), and as audiovisual temporal capture can be induced when temporal separation between auditory and visual signals is less than 100 ms (Morein-Zamir et al. 2003; Vroomen and de Gelder 2004), it can be assumed that audiovisual temporal capture underlies Xash fusion even when the initial Xash is synchronized with the initial pulse. Sound-induced illusory Xash refers to the phenomenon in which the number of successive Xashes is perceptually increased when the Xashes are accompanied with successive auditory pulses which are larger in number than the number of Xashes (Shams et al. 2000, 2002). Soundinduced illusory Xash has also been reported even when the Xash is sequentially moved (Kamitani and Shimojo 2001). Moreover, Kamitani and Shimojo (2001) showed that when two successive Xashes were presented in a spatiotemporally discrete manner, and were accompanied by the three successive auditory pulses, the observers reported that the location of the illusory Xash (i.e., the perceptually second Xash) was distorted in accordance with the temporal interval between auditory pulses. That is, when the temporal interval between the Wrst and second pulses was shorter than the interval between the second and third pulses, the spatial interval between the Wrst and second ‘illusory’ Xashes was perceptually shorter than the interval between the second ‘illusory’ and the third read Xashes. Recently, we reported that the temporal interval between auditory pulses can also distort perceived spatial interval between real Xashes (Kawabe et al. 2008). SpeciWcally, when the temporal interval between the Wrst and second pulses was shorter than the interval between the second and third pulses, the spatial interval between the Wrst and second real Xashes was perceptually shorter than the interval between the second and the third read Xashes. In that study, we suggested that the temporal interval between Xashes was captured by the temporal interval between pulses, and consequently the distorted temporal interval of Xashes also distorted the spatial interval between Xashes through the Tau eVect (Goldreich 2007). The pattern of spatial distortion in sound-induced Xash illusion is similar to that in audiovisual tau eVect (Kawabe et al. 2008). Thus, speculatively, the audiovisual temporal
Exp Brain Res (2009) 198:195–208
capture may also underlie sound-induced illusory Xash. The idea that the sound-induced illusory Xash and Xash fusion have the same origin is parsimonious and has also been validated in two mathematical models (Andersen et al. 2005; Shams et al. 2005). On the other hand, by measuring event-related potentials, Mishra et al. (2008) showed that sound-induced Xash illusion and Xash fusion had diVerent neural correlates. Future studies are required to further address the similarity and diVerence between Xash fusion and sound-induced illusory Xash by determining the eVect of spatial and temporal parameters on these illusory phenomena. Finally, there is also a potential role of audiovisual temporal capture in the ‘pulse fusion’ reported by Andersen et al. (2004). In that study, the number of pulses perceptually decreased when the pulses were accompanied with a lesser number of Xashes. However, the pulse fusion was induced only when the SPL of the pulses was reduced to the level of the observers’ threshold (i.e., 10 dB). Since the temporal resolution of auditory processing is extremely high, then temporal capture of audition by vision should not occur easily. On the other hand, it was previously shown that temporal capture of auditory timing by transient visual Xash did occur (Fendrich and Corballis 2001). Thus, it is reasonable to hypothesize that temporal capture of pulses by Xashes underlies the pulse fusion. This issue will be addressed in future studies by employing near-threshold pulse stimuli in which one of the pulses has a unique frequency, and asking observers to report the temporal location of the pulse with the unique frequency when the pulses are accompanied with supra-threshold successive Xashes, which are fewer in number than the number of pulses.
References Andersen TS, Mamassian P (2008) Audiovisual integration of stimulus transients. Vis Res 48:2537–2544 Andersen TS, Tiippana K, Sams M (2004) Factors inXuencing audiovisual Wssion and fusion illusion. Cogn Brain Res 21:301–308 Andersen TS, Tiippana K, Sams M (2005) Maximum likelihood integration of rapid Xashes and beeps. Neurosci Lett 380:301–308 Battaglia PW, Jacobs RA, Aslin RN (2003) Bayesian integration of visual and auditory signals for spatial localization. J Opt Soc Am A 20:1391–1397 Berger TD, Martelli M, Pelli DG (2003) Flicker Xutter: is an illusory event as good as the real thing? J Vis 3:406–412 Bertelson P, Aschersleben G (1998) Automatic visual bias of perceived auditory locations. Psychon Bull Rev 5:482–489 Bhattacharya J, Shams L, Shimojo S (2002) Sound-induced illusory Xash perception: role of gamma band responses. Neuroreport 13:1727–1730 Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436 Burr D, Alais D (2006) Combining visual and auditory information. Prog Brain Res 155:243–258 Fendrich R, Corballis PM (2001) The temporal cross-capture of audition and vision. Percept Psychophys 63:719–725
207 Fujisaki W, Nishida S (2007) Feature-based processing of audiovisual synchrony perception revealed by random pulse trains. Vis Res 47:1075–1093 Fujisaki W, Nishida S (2008) Top-down feature-based selection of matching features for audio-visual synchrony discrimination. Neurosci Lett 433:225–230 Goldreich D (2007) A Bayesian perceptual model replicates the cutaneous rabbit and other tactile spatiotemporal illusions. PLoS ONE 2:e333. doi:10.1371/journal.pone.0000333 Kamitani Y, Shimojo S (2001) Sound-induced visual “rabbit”. J Vis 1:478 Kanai R, Sheth BR, Verstraten FAJ, Shimojo S (2007) Dynamic perceptual changes in audiovisual simultaneity. PLoS ONE 2:e1253 Kawabe T, Miura K (2006) EVects of the orientation of moving objects on the perception of streaming/bouncing motion displays. Percept Psychophys 68:750–758 Kawabe T, Miura K, Yamada Y (2008) Audiovisual tau eVect. Acta Psychol 128:249–254 Keetels M, Stekelenburg J, Vroomen J (2007) Auditory grouping occurs prior to intersensory pairing: evidence from temporal ventriloquism. Exp Brain Res 180:449–456 MateeV S, Hohnsbein J, Noack T (1985) Dynamic visual capture: apparent auditory motion induced by a moving visual target. Perception 14:721–727 McCormick D, Mammassian P (2008) What does the illusory-Xash look like? Vis Res 48:63–69 Mishra J, Martinez A, Sejnowski TJ, Hillyard SA (2007) Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. J Neurosci 27:4120–4131 Mishra J, Martinez A, Hillyard SA (2008) Cortical processes underlying sound-induced Xash fusion. Brain Res 1242:102–115 Morein-Zamir S, Soto-Faraco S, Kingstone A (2003) Auditory capture of vision: examining temporal ventriloquism. Cogn Brain Res 17:154–163 Nishida S, Johnston A (2002) Marker correspondence, not processing latency, determines temporal bindings of visual attributes. Curr Biol 12:359–368 Pelli DG (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat Vis 10:437–442 Recanzone GH (2003) Auditory inXuences on visual temporal rate perception. J Neurophysiol 89:1078–1093 Sato Y, Toyoizumi T, Aihara K (2007) Bayesian inference explains perception of unity and ventriloquism aftereVect: identiWcation of common sources of audiovisual stimuli. Neural Comput 19:3335– 3355 Sekuler R, Sekuler AB, Lau R (1997) Sound alters visual motion perception. Nature 385:308 Shams L, Kamitani Y, Shimojo S (2000) Illusions: what you see is what you hear. Nature 408:788 Shams L, Kamitani Y, Thompson S, Shimojo S (2001) Sound alters visual evoked potentials in humans. Neuroreport 12:3849–3852 Shams L, Kamitani Y, Shimojo S (2002) Visual illusion induced by sound. Cogn Brain Res 14:147–152 Shams L, Ma WJ, Beierholm U (2005) Sound-induced Xash illusion as an optimal percept. Neuroreport 16:1923–1927 Shimojo S, Shams L (2001) Sensory modalities are not separate modalities: plasticity and interactions. Curr Opin Neurobiol 11:505–509 Shipley T (1964) Auditory Xutter-driving of visual Xicker. Science 145:1328–1330 Soto-Faraco S, Lyons J, Gazzaniga M, Spence C, Kingstone A (2002) The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. Cogn Brain Res 14:139–146 Vroomen J, de Gelder B (2004) Temporal ventriloquism: Sound modulates the Xash-lag eVect. J Exp Psychol Hum Percept Perform 30:513–518
123
208 Vroomen J, Keetels M (2009) Sounds change four-dot masking. Acta Psychol 130:58–63 Watanabe K, Shimojo S (2001) When sound aVects vision: eVects of auditory grouping on visual motion perception. Psychol Sci 12:109–111 Watkins S, Shams L, Tanaka S, Haynes J-D, Rees G (2006) Sound alters activity in human V1 in association with illusory visual perception. Neuroimage 31:1247–1256
123
Exp Brain Res (2009) 198:195–208 Zapparoli GC, Reatto LL (1969) The apparent movement between visual and acoustic stimulus and the problem of intermodal relations. Acta Psychol 29:256–267
