NeuroImage 59 (2012) 3604–3610
Contents lists available at SciVerse ScienceDirect
NeuroImage journal homepage: www.elsevier.com/locate/ynimg
Single-trial magnetoencephalography signals encoded as an unfolding decision process Nikolaos Smyrnis a, b,⁎, Dimitris S. Mylonas c, Roozbeh Rezaie b, Constantinos I. Siettos c, Errikos Ventouras e, Periklis Y. Ktonas d, Ioannis Evdokimidis a, Andrew C. Papanicolaou b a
Cognition and Action Group, Neurology Department, National and Kapodistrian University of Athens, Medical School, Aeginition Hospital, 72 V. Sofias Ave., Athens 11528, Greece Department of Pediatrics, University of Texas Health Sciences Center, 1333 Moursund St. Ste H114, Houston, TX 77030, USA School of Applied Mathematics and Physical Sciences, National Technical University of Athens, 9 Heroon Polytechniou str. St., Athens 157 80, Greece d Psychiatry Department, National and Kapodistrian University of Athens, Medical School, Aeginition Hospital, 72 V. Sofias Ave., Athens 11528, Greece e Department of Medical Instrumentation Technology, Technological Educational Institute of Athens, Ag. Spyridonos St., Athens 12210, Greece b c
a r t i c l e
i n f o
Article history: Received 13 May 2011 Revised 8 August 2011 Accepted 28 October 2011 Available online 4 November 2011 Keywords: Reaction time Voluntary movement Visuomotor ICA Premotor potential Visual evoked potential
a b s t r a c t The model of a stochastic decision process unfolding in motor and premotor regions of the brain was encoded in single-trial magnetoencephalographic (MEG) recordings while ten healthy subjects performed a sensorimotor Reaction Time (RT) task. The duration of single-trial MEG signals preceding the motor response, recorded over the motor cortex contralateral to the responding hand, co-varied with RT across trials according to the model's prediction. Furthermore, these signals displayed the same properties of a “rising-to-afixed-threshold” decision process as posited by the model and observed in the activity of single neurons in the primate cortex. The present findings demonstrate that non-averaged, single-trial MEG recordings can be used to test models of cognitive processes, like decision-making, in humans. © 2011 Elsevier Inc. All rights reserved.
Introduction A fundamental goal of cognitive neuroscience is the specification of brain mechanisms mediating decisions. This goal has been pursued in animal and human studies utilizing overt Reaction Time (RT) to test predictions of alternative models (Carpenter and Williams, 1995; Ratcliff, 2006; Stone, 1960; Taylor et al., 2006) of the decision process that begins with the encoding of task-relevant stimuli and terminates with the motor response. This process is typically viewed as a stochastic integration of the sensory input signals reaching a fixed response threshold. The neuronal processes corresponding to this decision mechanism have been the focus of many recent studies using single-neuron recordings in behaving animals. The basic finding in these studies has been that the activation of neurons in parietal and prefrontal cortical areas follows this pattern of stochastic integration over time until a fixed threshold is reached. The final level of neuronal activity predicts the response choice (whether right or wrong) of the animal performing a sensorimotor task, while the varying time to the
⁎ Corresponding author at: Psychiatry Department, National & Kapodistrian University of Athens, Medical School, Aeginition Hospital, 72 V. Sofias Ave., Athens, GR-11528, Greece. Fax: +30 2107216424. E-mail address:
[email protected] (N. Smyrnis). 1053-8119/$ – see front matter © 2011 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2011.10.091
activation threshold predicts RT (Kim and Shadlen, 1999; Salzman and Newsome, 1994; Shadlen and Newsome, 1996). In the context of some models (Carpenter et al., 2009; Redii, 2001), the decision process in simple sensorimotor tasks is decomposed into two stages, each potentially contributing to the ubiquitous trial-to-trial variability in RT. The first stage involves processing of the afferent signals evoked by the stimulus to be responded to. Across-trial variability in detecting these signals, due to the presence of noise, is one source of the observed RT variability. This source is most prominent with stimuli near sensory threshold and can be effectively eliminated through the presentation of unambiguous suprathreshold stimuli. The second stage involves a “rising-to-a-fixedthreshold” stochastic process leading to the motor response, which also varies in duration across trials, constituting another source of variability in RT. In a simple visuomotor task, single neurons in the eye movement motor areas, such as the frontal eye field (Hanes and Schall, 1996) and the superior colliculus (Glimcher and Sparks, 1992) of the rhesus monkey, exhibit such “ramping” activity to a fixed threshold. In this experiment we validate the two-stage decision model explaining the trial-to-trial RT variability, by further developing previously reported techniques to analyze single-trial MEG recordings (Jung et al., 2001). Specifically, we tested the following hypotheses based on predictions from the model: a) the visual stimulus-specific
N. Smyrnis et al. / NeuroImage 59 (2012) 3604–3610
single-trial MEG signal reflecting stimulus detection processes does not contribute to RT variability because we use highly suprathreshold stimuli; b) the single-trial MEG signal preceding the movement onset is a significant contributor to RT variability; this signal displays the same properties as the “rising-to-a-fixed-threshold” decision signal of the model depicted in Fig. 1. Accordingly, we hypothesize that it is the trial-to-trial variation of the slope of the MEG signal (corresponding to the rate of rise of the decision signal), and not its peak amplitude (corresponding to the threshold of the decision signal), that is related to RT variability. Materials and methods Participants Ten right-handed volunteers (8 females) participated in the study (age range: 22–30 years; mean/standard deviation: 26.4 ± 5.3 years). Participants had no history of neurological and psychiatric disorders and had normal to corrected-to-normal vision. The study was performed in accordance with the Declaration of Helsinki and was approved by the Institutional Review Board of the University of Texas Health Sciences Center, Houston. Participants provided informed consent and were financially compensated. Experimental procedures Utilizing a variable inter-stimulus interval of 2500–3500 ms, a visual stimulus (high-contrast, black-white checkerboard) was presented for 2000 ms on a back-projection screen (viewing angles of approximately 28.6° horizontally and 16.2° vertically). Upon presentation of the highly supra-threshold visual stimulus, subjects were instructed to squeeze a silicon ball as fast as possible. The silicon ball was attached to a fiber-optic response pad, which recorded the subject's reaction time (RT). Each subject performed 120 trials of the task. Data acquisition and preprocessing Continuous MEG recordings were obtained with a whole-head magnetometer array (4D Neuroimaging, San Diego, CA), that consisted of 248 first-order axial gradiometer coils, housed in a magnetically shielded chamber and arranged to cover the entire head surface. The magnetic flux measurements were digitized at 508 Hz, and subjected to a noise-reduction algorithm that is part of the 4D-Neuroimaging software.
Fig. 1. This is a schematic representation of a theoretical model for a simple rise-tothreshold decision process explaining RT variation in a simple sensorimotor task. A decision signal, depicted by the heavy black line in the figure, evolves linearly from a starting point (S0), after stimulus presentation, to a criterion point (ST) where a decision is made to respond to the stimulus. The rate of this linearly increasing decision process r varies from trial to trial and this variation is modeled as a normal distribution with mean μ and variance σ (shaded area around μ). The RT for a particular trial is equal to (ST − S0) / r (modified from Carpenter and Williams 1995).
3605
Subsequent preprocessing of the neuromagnetic signals was performed using the Fieldtrip toolbox (http://www.ru.nl/fcdonders/ fieldtrip) running in Matlab 7.7. Continuous signals were segmented into epochs of 2-sec time windows (1 sec pre-stimulus period). The baseline of each trial signal was computed from the first 200 ms of the data epoch. Trials with RT larger than 600 ms were considered as late responses and were excluded; trials with RT less than 100 ms were considered as predictive responses and were also excluded. The data were visually inspected on a single-trial basis to identify and discard the epochs and recording channels that were contaminated by muscle artifacts or SQUID jumps. The data were also cleaned of electrocardiographic and electrooculographic artifacts using the Independent Component Analysis algorithm that is incorporated in the Fieldtrip software (Makeig et al., 1996). For each subject, one to four recording channels, and approximately 15% of the trials, were discarded from the analysis. In total, 86% (1032) of 1200 trials (120 trials × 10 subjects) were included in the analysis. Detection of visual stimulus-specific and pre-movement response-specific activity in single-trial MEG time series The procedure used can be summarized in the following steps: Step 1: Computation of the average signal both after the stimulus onset and before the movement onset. The average visual stimulus-specific activity was derived for each subject by averaging the single-trial signals aligned to stimulus onset over a time window defined by the longest RT from all trials. In the case of the average pre-movement response-specific activity, the single-trial signals were averaged aligned to movement onset over a time window defined again by the subject's longest RT plus 200 ms after the movement onset. Step 2: Identification of statistically significant changes in the average MEG activity. At this step, two tasks were performed: one for the visual and one for the pre-movement activity. In the case of the average visual activity, we performed a statistical test to check if there was a significant increase or decrease in the MEG averaged signal intensity within a time window of 60–130 ms after the stimulus onset from the mean MEG signal intensity of the first 40 ms after the stimulus onset. In the case of the average pre-movement activity, we searched for a significant difference in the average MEG signal's intensity from zero that should have ended no sooner than 50 ms before movement onset. For both cases we performed a repeated twotailed t-test with the significance level set at 0.01. Step 3: Identification of statistically significant activity (components) in single-trial MEG signals. The aim, at this step, was to examine if there was significant single-trial MEG activity (components) within a certain time interval, depending on the sought response. For the visual activity, we searched within a time interval that was longer than 6 samples (~ 12 ms). This time interval effectively discarded high-frequency noise that was present in the raw MEG signal. For the pre-movement activity, the duration of the significant signal (component) should have been again longer than 6 samples and should have ended no sooner than 50 ms from the movement onset. The identification was based on the concept of the template-function approach (Rowley and Marsden, 2000). For our analysis we used a zero-mean Gaussian-shaped template-function, ! 1 −i2 g ðiÞ ¼ pffiffiffiffiffiffi exp ; 2d2 2πd
3606
N. Smyrnis et al. / NeuroImage 59 (2012) 3604–3610
motor response. The visual stimuli were black dots presented at one of 8 possible locations around a central fixation dot. The subject was instructed to press a button with his/her right index finger to indicate the presence of the visual stimulus. We used data from 2 subjects, each performing 1200 trials of this task (150 trials per each stimulus location). The preprocessing of the data was identical to the one described in this manuscript and the method for detecting single-trial visual stimulus-evoked MEG signals and pre-movement activity-related MEG signals was identical to the one previously described. The application of the method was highly successful, as shown in supplementary Figs. 1 and 2; the resulting detection of the visual and pre-movement-related single-trial signals was verified in all channels where such signals were detected in the average evoked response.
whose width was set equal to the width of the significant average visual or pre-movement activity, detected in Step 2, according to the sought task. We also repeated the analysis using a triangular and a step pulse as template functions and the results were very similar. We then computed the Pearson correlation coefficient between the single-trial MEG signal y(i) and the template function g(i). The correlation coefficient N∑g ðiÞyði−lÞ−∑g ðiÞ∑yði−lÞ r gy ðlÞ ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiqffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi N∑g ðiÞ2 −ð∑g ðiÞÞ2 N∑yði−lÞ2 −ð∑yði−lÞÞ2 was computed for l = −N/2 to N/2, shifting y(i) by one sample step, where N was the length of y(i). In order to check if there was a statistically significant singletrial visual or pre-movement MEG activity, we performed a repeated two-tailed t-test with a significance level set at 0.01 for each value of the Pearson's correlation coefficient, rgy(l). In addition to the criteria that were used to detect the significant visual and pre-movement motor activity in the average signal, we set the following two restrictions: (a) in the case of the visual activity, the end of the significant signal should not occur after the movement onset and (b) in the case of the pre-movement activity, the significant signal should begin after the presentation of the stimulus. For the single-trial MEG signal during the time of statistically significant components, given by the correlation process, we recorded the time of onset and calculated the duration, the peak absolute amplitude and the linear slope (computed using the onset time and the peak absolute amplitude). We should note that, in using the MEG, the surface-recorded brain activity may be, in principle, represented as a positive or a negative magnetic field fluctuation, depending on the relative position of the recording channel to the underlying neural sources. Thus, a single channel should record the same kind of magnetic field fluctuation (positive or negative) as a result of the same activity of the same sources. A detection of single-trial visual or pre-movement activity in a recording channel was accepted only if the raw signal fluctuation in the given channel was in the same direction (either positive or negative) with the average detected signal. Step 4: Final validation of the detected signals A new averaging was performed for each recording channel where significant visual or pre-movement activity was detected in at least 50 trials. The alignment in time was the same as in the first averaging (Step 1), for both the visual and pre-movement activity. We used the same two-tailed t-test to detect significant differences of the average signal from the baseline activity. This procedure was performed in order to verify that the new average signal from all detected single-trial signals (components) had larger or at least equal peak amplitude as the original average signal produced by averaging all trials. The above four-step procedure was applied to all recording channels for all subjects. The identification of the duration of the significant visual or pre-movement activity in the single-trial MEG signal was checked by plotting the singletrial signals following the method introduced by Jung et al. (2001). In the vast majority of the cases, the identified signals matched perfectly those identified by inspection of the single-trial signal plots. In order to provide further validation for this method, we applied the same method to a new set of data recorded in the MEG laboratory for another experiment. In that experimental set up, visual stimuli were used again to trigger a
Analysis of the single-trial signal component parameters with respect to RT As mentioned before, we computed the following parameters for each detected significant component in the single-trial MEG signal, for the visual and pre-movement activity: (a) time of onset from stimulus presentation (in ms), (b) duration (in ms), (c) slope (in Tesla/s) and (d) peak absolute amplitude (in Tesla). These measures for each detected component were obtained for all active channels (channels with 50 or more detected single-trial signals) and for each one of the 10 subjects. In order to derive a single value for each parameter, for each trial and for each subject, we computed the average parameter values across all active channels. Table 1 presents the means and standard deviations of the derived singletrial signal parameters for each subject and for all detected trials, separately for both the visual and the pre-movement activity. It should be noted that subject 4 did not have any valid channels with 50 or more detected components for pre-movement data. Therefore, the single-trial parameter data for the pre-movement activity were derived from 9 of the 10 subjects. Overall, 967 single-trial parameter values were obtained for the visual activity for all subjects and 945 single-trial parameter values were obtained for the pre-movement activity for 9 subjects. In order to study the effect of each single-trial parameter on RT, we used a random-effect general linear model analysis. The independent variable of the model was each one of the single-trial parameters characterizing the visual or the pre-movement activity, respectively, while RT was the dependent variable. The subject served as an independent random factor in this analysis. The resulting F and P values for the significance of the parameter effect on RT are reported in Results section. The F values for the random subject effect and the interaction effect of subject by single-trial MEG parameter are not reported since they are not relevant to the scope of this study. We also used a simple linear regression model in which each single-trial MEG parameter predicted RT. The regression was run separately for each subject and a resulting beta value was estimated. Then a t-test was used to estimate if the mean beta value for all subjects was significantly different from zero. The resulting mean beta value, the t value and its significance are reported for each parameter effect in Results section. The statistical analysis was performed using the STATISTICA 7.0 software (StatSoft Inc. 1984–2004). Results The simple visuomotor task performance resulted in two average magnetic evoked responses that were clearly spatially dissociated. The first one was related to the visual stimulus presentation and the second one was related to the upcoming movement (Fig. 2).
N. Smyrnis et al. / NeuroImage 59 (2012) 3604–3610
3607
Table 1 Single-trial signal characteristics. Number of trials with detected signals A S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 B S1 S2 S3 S5 S6 S7 S8 S9 S10
RT (ms)
Time of signal onset (ms)
Single-trial visual-activity-related signal 103 317 (103) 106 (12) 111 259 (46) 115 (6) 102 341 (66) 83 (8) 83 473 (66) 105(14) 69 419 (75) 122 (17) 95 330 (82) 118 (18) 110 293 (66) 115 (10) 78 337 (79) 112 (20) 108 302 (41) 111 (15) 108 305 (45) 120 (16) Single-trial pre-movement-activity-related signal 102 316 (102) 278 (104) 111 259 (46) 214 (41) 102 341 (66) 266 (67) 93 423 (77) 308 (71) 97 331 (82) 262 (79) 110 293 (66) 230 (63) 105 350 (80) 276 (70) 114 303 (40) 236 (34) 111 306 (45) 247 (46)
Signal duration (ms)
Slope of signal rise (10− 13 × T/s)
Signal peak absolute amplitude (10− 13 × T)
25 32 35 32 56 33 34 31 35 32
93 64 76 36 37 51 70 46 56 46
(38) (12) (33) (11) (18) (17) (19) (18) (22) (18)
1.2 1.0 1.3 0.6 1.0 0.8 1.2 0.7 1.0 0.7
(0.5) (0.2) (0.6) (0.2) (0.6) (0.3) (0.4) (0.3) (0.4) (0.3)
60 47 19 14 26 36 15 38 29
(18) (8) (4) (4) (8) (8) (6) (10) (9)
1.5 1.5 0.6 0.9 0.9 1.2 0.6 1.0 0.8
(0.5) (0.2) (0.2) (0.3) (0.3) (0.3) (0.2) (0.2) (0.3)
(3) (4) (5) (8) (25) (11) (6) (12) (15) (12)
35 (12) 42 (11) 62 (16) 108 (34) 63 (17) 57 (14) 65 (24) 55 (13) 50 (24)
This table presents the number of detected significant single-trial signals (components) per subject for the visual (A) and pre-movement (B) activity as well as means and standard deviations in parentheses for RT and signal parameters measured for these trials. Note that subject 4 is missing from the pre-movement activity data set because there were no recording sites with more than 50 detected single-trial signals for this subject's pre-movement activity.
Both the evoked visual stimulus-specific MEG signal (Fig. 3a), and the MEG signal preceding the movement response (Fig. 3b) could be visualized in single trials (Figs. 3c, d). By cross-correlating the singletrial MEG signal with a Gaussian function, serving as a signal identification template (Materials and methods), we were able to successfully
detect and quantify single-trial MEG signals related to visual and premovement activity (Figs. 3e, f). A map of the active recording channels (corresponding to sensors) was constructed for each subject, separately for the visual and the pre-movement activity as shown in Figs. 3g and h, respectively. It is clear from viewing these maps that the active
Fig. 2. This figure presents grand average MEG signals aligned to the presentation of the visual stimulus (left side plots, stimulus presentation marked with black vertical line) and to the movement onset (right side plots, movement onset marked with magenta vertical line). A clear visual evoked response starting at the earliest 60 ms after stimulus presentation and peaking at about 130 ms was present for occipitoparietal recording sites. Also, a clear pre-movement activity starting at about 100 ms before movement onset and peaking at movement onset was present for frontal recording sites with a left hemispheric predominance.
3608
N. Smyrnis et al. / NeuroImage 59 (2012) 3604–3610
channels showing single-trial MEG visual activity were located in the posterior cortical areas bilaterally; the active channels showing singletrial MEG pre-movement activity were located in central and frontal cortical areas contralateral to the right arm (that was used for these movements). Single-trial MEG signals from all active channels during visual and pre-movement-related activity for each subject were used to derive estimates for four signal parameters, namely time of onset, duration, linear slope of signal rise, and peak absolute amplitude (see Materials and methods). Consistent with our first hypothesis, none of the single-trial visual activity-related MEG signal parameters had a significant effect on RT as verified by the random effects ANOVA
analysis (see Materials and methods) (time of onset: F1,956 = 0.03, P = 0.86; duration: F1,956 = 2.37, P = 0.12; slope: F1,956 = 0.23, P = 0.63; peak amplitude: F1,956 = 1.04, P = 0.31). The linear regression model analysis (see Materials and methods) verified the same non-significant effects (time of onset: mean beta value = 0.0008, SD = 0.15, t9 = 0.02, P = 0.98; duration: mean beta value = 0.09, SD = 0.15, t9 = 1.87, P = 0.09; slope: mean beta value = − 0.03, SD = 0.11, t9 = − 0.1, P = 0.9; peak amplitude: mean beta value = 0.02 SD = 0.13, t9 = 0.57, P = 0.58). In accordance with our second hypothesis, we observed that the duration of the single-trial pre-movement activity-related MEG signal was a significant predictor of RT. The time of onset of the pre-
Fig. 3. a: Average visual activity-related MEG signal aligned on stimulus presentation (black line) recorded from a single channel (subject 2). b: Average pre-movement-related MEG signal aligned on the onset of the movement (magenta line) recorded from a single channel (subject 2). c: Single-trial visual activity-related MEG signals aligned on stimulus presentation (black line; recording channel as in a). Magenta lines mark movement onset for each trial. d: Single-trial pre-movement-related MEG signals aligned on movement onset (magenta line; recording channel as in b). Black lines mark stimulus presentation. e, f: Same plots as in c, d respectively, with the addition of black boxes illustrating the quantification of the single-trial visual activity-related/pre-movement-related MEG signal duration. g, h: Maps of the active channels (subject 2) defined as having more than 50 single-trial visual activity-related or pre-movement-related signals detected, respectively. Non-active channels are colored grey, while active channels are colored according to the number of detected single-trial signals (bar below map).
N. Smyrnis et al. / NeuroImage 59 (2012) 3604–3610
3609
movement activity-related MEG was also a highly significant predictor of RT. These significant effects were observed using the random effect ANOVA analysis (time of onset: F1,935 = 8400, P b 10 − 6 ; duration: F1,935 = 80.5, P b 10 − 6) and linear regression analysis (time of onset: mean beta value = 0.93, SD = 0.05, t8 = 51.8, P b 10 − 6 ; duration: mean beta value = 0.31, SD = 0.2, t8 = 4.5, P = 0.002). It should be noted here that the time of onset of the premovement-related activity was, by definition, the remaining time interval when subtracting the pre-movement activity duration from RT. Thus, any variation in RT that was not related to the premovement activity per se would be reflected in this time interval. This explains the very strong and extremely highly significant covariation of this time interval with RT. Our hypothesis was that the duration of the pre-movement related activity makes an independent contribution to the variability of RT. This was definitively confirmed by performing a joint analysis of both variables (time of onset and duration of the pre-movement related activity) which accounted for all the variance in RT, by using a random effect ANOVA and showing that their effects on RT were indeed independent (time of onset: F1,934 = 97367, P b10 − 6; duration: F1,934 = 9759, P b 10 − 6). As predicted by the “rising-to-a-fixed-threshold” decision model depicted in Fig. 1, the duration of the pre-movement activity-related MEG signal increased with increasing RT (Fig. 4a). Furthermore, there was no effect of peak MEG signal amplitude on RT (Fig. 4b) (F1,935 = 1.04, P = 0.31), and RT decreased with increasing slope of the premovement-related MEG signal (Fig. 4c) (F1,935 = 14.48, P b 10 − 3). The same results were also confirmed using the linear regression analysis (peak amplitude: mean beta value = 0.02, SD = 0.21, t8 = 0.33, P = 0.75; slope: mean beta value = −0.20, SD = 0.17, t8 = −3.4, P = 0.009). A more subtle justification of the model by our data relates to the fact that, in the model, the slope of the decision signal increases non-linearly relative to RT, but linearly relative to the reciprocal of RT. Our data agree with the above as can be seen by the fitted regression lines in Figs. 4c and d.
Discussion Our results clearly demonstrate that the single-trial MEG signal prior to a simple motor response to a visual stimulus carries important information about the timing of this response. More specifically, these results validate a two-stage decision model positing that the visuomotor task consisted of two independent processes, each potentially contributing to RT variability. There are two basic prerequisites to obtaining such evidence in humans non-invasively: (a) recording of noiseless brain signals on a single-trial basis and, (b) recording separately brain signals encoding the stimulus and brain signals related to the preparation to execute the response, i.e. signals corresponding to the two stages of the decision process. Jung et al. (2001) used independent component analysis to derive noiseless single-trial signals and were able to visualize sensory evoked responses as well as premotor activity in the single-trial EEG and MEG signal. In this study, we capitalized on that finding and were able to quantify these single-trial signals, deriving estimates of their duration as well as slope and peak amplitude. The use of MEG also allowed the clear temporal and spatial separation of the single-trial signals to those corresponding to the stimulus encoding (visual activity) and those corresponding to the preparation of the movement.
Fig. 4. Scatter plots showing the relation of RT to the single-trial pre-movement-related MEG signal parameters (a: duration, b: peak absolute amplitude, c: slope) pooled together for all subjects. A linear regression fit of each parameter predicting RT was performed and the resulting fitted regression line (black) and regression coefficient r are plotted only for the cases where the ANOVA analysis has shown a significant effect of the parameter on RT. d: Same as c but, instead of RT, the reciprocal of RT is used in x axis.
3610
N. Smyrnis et al. / NeuroImage 59 (2012) 3604–3610
Previous studies using EEG (Kutas et al., 1977; McCarthy and Donchin, 1981) showed that manipulation of the sensory stimuli affects both RT and the magnitude of the (averaged across trials) P300 response, but did not show any separate features of the brain activity relating to stimulus parameters versus that relating to the response. Furthermore, those and more recent studies investigating the relation of cortical potentials preceding saccades to RT (Everling et al., 1997; Papadopoulou et al., 2010) were based on averaging and could not demonstrate any variation in the duration of brain activity across individual trials to be related to the variation in RT. A recent fMRI study (Binder et al., 2004) showed changes in the magnitude of the hemodynamic response in auditory areas of the brain contingent on the accuracy of discriminating between two auditory stimuli, and in prefrontal cortex contingent on variations in RT. However, this study was also based on averaging across trials. Furthermore, the hemodynamic response cannot provide an accurate estimate of the timing of neuronal activation in fast RT tasks. Finally, Ratcliff et al. (2009) recorded EEG in a perceptual discrimination task and used a logistic regression model to define two EEG amplitude components (one early and one late) that could discriminate the two categories of perceptual stimuli. They showed that the trial-totrial variability of the amplitude of the late but not the early EEG component predicted the quality of the categorical decision based on a theoretical diffusion model for decision making. A major difference between those studies and the current one is the use of MEG that clearly dissociated visual stimulus-related activity from the activity related to movement preparation and execution both in time and in space. Furthermore, the present study showed a direct correlation of the MEG single-trial signals to behavior. Despite their very low signal-to-noise ratio, these signals, nevertheless, seem to carry specific timing information that may be lost by averaging. The relation of both the peak amplitude and slope of the rising pre-movement-related MEG signal to RT provides evidence that the single-trial MEG signal can in fact be used to model a decision signal as predicted by previous psychophysical and neural recording studies. The particular form of the relation of RT to both the MEG signal slope and peak amplitude found in this study matches the prediction of the linear “rise-to-a-fixed-threshold” model that is presented in Fig. 1. Such evidence was also provided in a study recording the activity of single neurons in the primate Frontal Eye Field preceding visually guided saccadic eye movements (Hanes and Schall, 1996). In that study, the activation of single neurons increased to a fixed threshold. The rise of this activity varied from trial to trial and this variation predicted RT variability as predicted by the model in Fig. 1. Thus, our study generalizes this previous evidence to a different motor task, and, more importantly, from the single neuron activity to the activation signal arising from large brain networks (MEG signal). In conclusion, this study demonstrates that the single-trial MEG signal can be directly encoded as a decision process unfolding in the
motor areas of the human brain. This finding raises the possibility of investigating the neural substrate of more complex decision processes, as well as of pathological states affecting decision making in humans, using non-invasive MEG brain imaging. Supplementary materials related to this article can be found online at doi:10.1016/j.neuroimage.2011.10.091. Acknowledgments This research has been supported in part by the European Union (European Social Fund—ESF) and by Greek National Funds through the Operational Program “Education and Lifelong Learning” of the National Strategic Reference Framework (NSRF)—Research Funding Program: Heracleitus II. Investing in knowledge society through the European Social Fund (D. Mylonas and C. Siettos). References Binder, J., Liebenthal, E., Possing, E., Medler, D., Ward, B., 2004. Neural correlates of sensory and decision processes in auditory object identification. Nat. Neurosci. 7, 295–301. Carpenter, R.H.S., Williams, M.L.L., 1995. Neural computation of log likelihood in the control of saccadic eye movements. Nature 265, 59–65. Carpenter, R.H.S., Reddi, B.A.J., Anderson, A.J., 2009. A simple two-stage model predicts reaction time distributions. J. Phys. 587 (16), 4051–4062. Everling, S., Krappmann, P., Spantekow, A., Flohr, H., 1997. Influence of pre-target cortical potentials on saccadic reaction times. Exp. Brain Res. 115, 479–484. Glimcher, P.W., Sparks, D.L., 1992. Movement selection in advance of action in the superior colliculus. Nature 355, 542–545. Hanes, J.P., Schall, J.D., 1996. Neural control of voluntary movement initiation. Science 274, 427–430. Jung, T.-P., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., Sejnowski, T.J., 2001. Analysis and visualization of single-trial event related potentials. Hum. Brain Mapp. 14, 166–185. Kim, J., Shadlen, M.N., 1999. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci. 2, 176–185. Kutas, M., McCarthy, G., Donchin, E., 1977. Augmenting mental chronometry: the P300 as a measure of stimulus evaluation time. Science 197, 792–795. Makeig, S., Bell, A.J., Jung, T.-P., Sejnowski, T.J., 1996. In: Touretzky, D., Mozer, M., Hasselmo, M. (Eds.), Advances in Neural Information Processing Systems. MIT Press, Cambridge, MA, pp. 145–151. McCarthy, G., Donchin, E., 1981. A metric for thought: a comparison of P300 latency and reaction time. Science 211, 77–80. Papadopoulou, M., Evdokimidis, I., Tsoukas, E., Mantas, A., Smyrnis, N., 2010. Eventrelated potentials before saccades and antisaccades and their relation to reaction time. Exp. Brain Res. 205, 521–531. Ratcliff, R., 2006. Modeling response signal and response time data. Cogn. Psychol. 53, 195–237. Ratcliff, R., Philistiades, M.G., Sajda, P., 2009. Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proc. Natl. Acad. Sci. U. S. A. 106, 6539–6544. Redii, B.A.J., 2001. Decision making: the two stages of neuronal judgment. Curr. Biol. 11, 603–606. Rowley, C.W., Marsden, J.E., 2000. Reconstruction equations and the Karhunen–Loève expansion for systems with symmetry. Phys. D 142, 1–19. Salzman, C.D., Newsome, W.T., 1994. Neural mechanisms for forming a perceptual decision. Science 264, 231–237. Shadlen, M.N., Newsome, W.T., 1996. Motion perception: seeing and deciding. Proc. Natl. Acad. Sci. U. S. A. 93, 628–633. Stone, M., 1960. Models for choice reaction time. Psychometrica 25, 251–260. Taylor, M.J., Carpenter, R.H.S., Anderson, A.J., 2006. A noisy transform predicts saccadic and manual reaction times to changes in contrast. J. Phys. 573 (3), 741–751.
