ALCOHOLISM: CLINICAL AND EXPERIMENTAL RESEARCH
Vol. 31, No. 5 May 2007
Ethanol-Associated Cues Produce General PavlovianInstrumental Transfer Laura H. Corbit and Patricia H. Janak
Background: Conditioned stimuli are thought to play an important role in maintaining ethanol use and inducing relapse. Therefore, understanding the mechanisms through which such stimuli trigger ethanol seeking is of interest in the study and treatment of alcoholism. Methods: This series of experiments examined the impact of ethanol-associated cues on ethanolseeking behavior using a Pavlovian-instrumental transfer design. Rats received Pavlovian training in which an auditory stimulus predicted ethanol (10%) delivery. In a separate instrumental training phase, animals were trained to press a lever for ethanol. In the test phase, the impact of the stimulus on instrumental performance was assessed in extinction by presenting the stimulus while animals were free to perform the lever-press response. Experiment 2 assessed the selectivity of the transfer effect; rats received training with 2 auditory stimuli which predicted either ethanol or sucrose (2%) delivery and were trained to perform 2 instrumental responses, one earning ethanol and the other earning sucrose. Finally, Experiment 3 examined the selectivity of PIT using 2 natural rewards (sucrose and polycose). Results: The results from Experiment 1 show that ethanol supports excitatory conditioning and that ethanol-associated cues facilitate instrumental performance for ethanol. When the selectivity of the transfer effect was examined in Experiment 2, the ethanol-paired stimulus was found to have a general excitatory effect on reward-seeking behavior, affecting both ethanol-directed and sucrosedirected responding equally. In contrast, the sucrose-paired stimulus had a selective effect, elevating sucrose-directed responding only. Experiment 3 confirms that selective transfer is observed when 2 natural rewards are used to reinforce responding. Conclusions: These data provide further evidence that ethanol-associated cues can drive ethanolseeking behaviors. Because ethanol-associated cues also enhanced seeking behavior for a nonalcohol reward, these results additionally suggest that the modulation of reward-directed behaviors by cues associated with ethanol versus natural rewards may rely on different behavioral and neural mechanisms. Key Words: Pavlovian-Instrumental Transfer, Alcohol, Reward, Addiction, Learning.
C
ONTEMPORARY THEORIES OF addiction attribute a significant role to drug-associated stimuli in producing craving, supporting compulsive drug taking, and inducing relapse (Everitt and Robbins, 2005; O’Brien et al., 1998); therefore, defining the mechanisms through which these stimuli trigger drug seeking behavior is critical for understanding and ultimately treating drug abuse and relapse. Environmental stimuli can gain control of drug seeking through Pavlovian conditioning when such
From the Department of Neurology, Ernest Gallo Clinic and Research Center, University of California, San Francisco, California. Received for publication September 20, 2006; accepted January 17, 2007. The research reported in this paper was supported by funds from the state of California for medical research on alcohol and substance abuse through the University of California, San Francisco. Reprint requests: Laura H. Corbit, PhD, Department of Neurology, Ernest Gallo Clinic and Research Center, 5858 Horton St. Suite 200, Emeryville, CA 94608; Fax: 510-985-3101; E-mail:
[email protected]. edu Copyright r 2007 by the Research Society on Alcoholism. DOI: 10.1111/j.1530-0277.2007.00359.x 766
stimuli are repeatedly paired with the pharmacological effects of the drug. Studies of the role of drug-predictive stimuli, or conditioned stimuli (CSs), in the control of drug-seeking behaviors have largely used procedures such as conditioned reinforcement where response-contingent CSs act to support or maintain responding associated with drug delivery (e.g., Di Ciano and Everitt, 2005; Nie and Janak, 2003). It is likely that drug-associated CSs can also act before the drug-seeking response to enhance drug-seeking, and thus it is important to understand the effects that such stimuli have on response selection and initiation, although these areas are relatively less studied. The Pavlovian-instrumental transfer (PIT) paradigm can be used to examine the modulatory role of CSs on performance of independently acquired reward-seeking behaviors. In this task, a stimulus such as a tone is first paired with reward delivery. Subjects then receive instrumental training where performance of a particular response earns reward. Finally, in the test phase, responding is assessed in the presence and absence of the previously trained CS. Results generally show that Alcohol Clin Exp Res, Vol 31, No 5, 2007: pp 766–774
CUE-INDUCED ETHANOL SEEKING
presentation of reward-predictive stimuli will enhance performance of previously trained instrumental responses directed toward that same reward. An important issue that arises when assessing the behavioral function of drug-associated stimuli is that Pavlovian CSs can elevate instrumental responding through at least 2 processes. Stimuli may increase appetitive arousal generally and thus have an activating, but potentially indiscriminant effect on compatible appetitive responses (e.g., Rescorla and Solomon, 1967). Additionally, presentation of a CS may activate a specific priming or retrieval process where presentation of a CS signals or produces the expectancy of a particular reward and thus selectively elevates performance of responses also associated with that unique reward (Kruse et al., 1983; Overmier and Lawry, 1979). This distinction may be important when considering drug or ethanol (EtOH) reward systems. Stimuli associated with drugs may exert a more powerful influence over reward directed behaviors than stimuli associated with natural rewards, and this motivational activation may generalize to behaviors that produce a variety of rewards. Alternatively, one might predict a highly selective effect of drug-related cues given the characteristic narrowing of an addict’s behavioral repertoire to focus on obtaining and using drugs. More complex versions of the PIT task can examine this issue; the selectivity of any stimulus can be tested by conditioning and comparing multiple stimuli and instrumental responses and then examining their interactions (Colwill and Motzkin, 1994; Corbit and Balleine, 2005; Corbit et al., 2001; Trapold and Overmier, 1972). A recent study examining PIT with EtOH reward found that cues associated with either EtOH or a nondrug reward had a nonselective excitatory effect on reward-seeking behaviors and concluded that EtOH-directed responses are influenced by the general excitatory or arousing properties of reward-related cues rather than being outcome specific, i.e., based on knowledge of the particular consequences of a given action (Glasner et al., 2005). However, in this study, responding for the nondrug reward was also elevated by cues in a nonselective fashion. This result is somewhat surprising because previous work with natural rewards has shown outcome-selective transfer effects (e.g., Colwill and Motzkin, 1994; Corbit and Balleine, 2005; Kruse et al., 1983). It is possible that differences in the training parameters rather than something particular about EtOH reward led to the development of associations that support general versus outcome-specific PIT. For example, Glasner et al. (2005) used sweetened EtOH as well as food restriction which may have led the rats to attend to the common features of the 2 outcomes (e.g., calories), which later biased the form of transfer seen at test. The aim of the current study was to further examine the ability of EtOH-associated cues to modulate reward-directed behavior by testing the ability of an EtOH-associated cue to invigorate responding for EtOH
767
and by assessing the selectivity of the effects of an EtOH CS when animals were trained to respond for both EtOH and a nondrug reward. MATERIALS AND METHODS Experiment 1—Pavlovian-Instrumental Transfer and EtOH Reward Subjects and Apparatus. Ten experimentally naı¨ ve male LongEvans rats (Harlan, Indianapolis, IN) weighing approximately 350 g at the beginning of the experiment served as subjects. The rats were singly housed and had free access to food and water in the home cage except as described below. All procedures were approved by the Institutional Animal Care and Use Committee of the Ernest Gallo Clinic and Research Center at the University of California, San Francisco. Training and testing took place in Med Associates (East Fairfield, VT) operant chambers housed within sound-attenuating and light-attenuating shells. Each chamber was equipped with 2 pumps, each of which was fitted with a syringe that delivered a set volume of solution into a recessed magazine in the chamber when activated. The chambers contained retractable levers that could be inserted to the left and right of the magazine. The boxes also contained a sonalert that delivered a 3 kHz tone, a white noise generator, and a solanoid that, when activated, delivered a 5 Hz clicker stimulus. A 3 W, 24 V houselight mounted on the top-center of the wall opposite the levers and magazine provided illumination. Computers equipped with MED-PC software controlled the equipment and recorded magazine entries and lever-press responses. EtOH Acclimation. It has been extensively reported that because of the aversive taste of EtOH rats are initially relatively unwilling to consume EtOH. To ensure that the rats experienced the pharmacological effects of EtOH they were initially were given 3 days of forced EtOH; water bottles were removed and the rats had free access to 10% EtOH (10E; v/v) in tap water. Following this initial exposure, to develop voluntary self-administration of EtOH, we used a modified sucrose fading procedure (e.g., Nie and Janak, 2003; Samson, 1986; Tolliver et al., 1988). The water bottles were returned and the animals were given a solution containing 10% EtOH and 10% sucrose (10S/10E). This solution was available 24 hours a day for 14 days. Next the 10S/10E solution was changed to 1 containing only 10E. Animals had 24 hour access to this solution for 14 days. As the self-administration sessions would occur during the animals’ light phase and they would only have limited access (i.e., 1 hr session) during this training, for the final 7 days of acclimation, the 10E was made available for 1 hour per day at the same time that the training session would occur. The rats had free access to water throughout acclimation with the exception of the initial 3 days of forced consumption. Rats were weighed daily and EtOH consumption was recorded to calculate grams per kilogram of EtOH. Pavlovian Training. The rats received eight sessions of Pavlovian conditioning. Two auditory stimuli [tone (T) and clicker (C)] served as CSs. One of these stimuli (CS1) was paired with EtOH delivery while the other stimulus (CS ) had no programmed consequences. For half of the subjects, the tone was paired with EtOH delivery, whereas the clicker was paired with EtOH delivery for the remaining subjects. Six presentations of each stimulus were given in each session in random order interspersed with periods in which no stimuli were present. The length of the intertrial intervals varied, but on average, these intervals were 5 minutes. The stimulus presentations were 2 minutes long. During the CS1, EtOH was delivered on a random time (RT) 30 sec schedule. An RT schedule was used to prevent the precise timing of reward delivery which has been shown to affect the nature of the conditioned response (CR) that develops as a result of conditioning (e.g., Holland, 1980). Each delivery was 0.2 mL and while the schedule of delivery was random and thus the number of outcomes varied across sessions, on average the animals received 4.8
CORBIT AND JANAK
768
mL of EtOH across the 75 minutes session. The number of magazine entries during each stimulus as well as in a prestimulus interval of equal length (2 minutes) was measured. In the final session, additional probe trials were included within the session. An additional nonreinforced probe trial of each the CS1 and CS were included and magazine entries within these trials were recorded separately. Instrumental Training. The animals were next trained to selfadminister EtOH. For half of the animals, responding on the left lever (active) produced 0.1 mL of 10E whereas responding on the right lever had no programmed consequences (inactive). The remaining animals received the opposite assignments. Initially the animals received 1 day of training with the active lever alone. Responses on this lever produced EtOH on a continuous reinforcement schedule. Thereafter, both levers were present though only responding on the active lever was rewarded. The animals received 5 days of a continuous reinforcement schedule and were then shifted to a random ratio (RR) 2 schedule where, on average, every second response was rewarded. After 5 days on the RR2 schedule, the animals were shifted to a RR5 schedule for an additional 5 days of training. Sessions were 60 minutes in duration. Pavlovian-Instrumental Transfer Test. The excitatory impact of the stimuli on instrumental performance was assessed in extinction. Both levers were present during the test and each stimulus was presented 3 times interspersed with intervals of no stimulus (+). The test was 30 minutes in duration and contained 12, 2-minutes bins (3 tone trials and 3 clicker trials alternated with six + bins. The first bin was a + bin followed by stimulus bins in the following order: T,C,C,T,T,C; thus, for half of the animals the first stimulus presentation was the CS1 whereas for the remaining animals it was the CS ). Each stimulus bin was separated from the subsequent baseline (+) bin by 1 minute.
Instrumental Training. The animals were next trained to respond on the 2 levers to self-administer EtOH or sucrose. For half of the animals, responding on the left lever produced 0.1 mL of 10E whereas responding on the right lever delivered 0.1 mL of 2S. The remaining animals received the opposite assignments. Training for the 2 levers occurred independently on alternating days and all sessions were 60 minutes in duration. Initially the animals received 2 days of training for each outcome in which responding was reinforced on a continuous reinforcement schedule; they were then shifted to a random ratio (RR) 2 schedule. After 3 days of training on the RR2 schedule for each outcome, the animals were shifted to a RR4 schedule for an additional 3 days of training. Pavlovian-Instrumental Transfer Test. The animals received 2 extinction tests (1 for each lever) 1 day apart. During each of the tests, 1 lever was available and each stimulus was presented 3 times interspersed with intervals of no stimulus (+). The test was 30 minutes in duration and contained 12, 2-minutes bins (starting with a baseline (+) bin and followed by 3 white noise trials and 3 clicker trials alternated with + trials in the following order: N,C,C,N,N,C. Because of the counterbalancing during conditioning, half of the animals were tested with the EtOH lever first and the sucrose lever second. Additionally, the first stimulus presentation in each test was the EtOH stimulus for half of the animals and the sucrose stimulus for the remaining animals. Each stimulus bin was separated from the subsequent baseline (+) bin by 1 minute. Single-lever tests were used because this is most consistent with the training conditions with which the animals were familiar. This allows us to assess the effect of the stimuli without the added complexity of a novel choice situation. Additionally our procedures were based on several reports in the literature (e.g., Rescorla, 1994) in which single manipulandum tests following training with multiple actions were used to avoid individual response biases or outcome preferences.
Experiment 2—Selectivity of EtOH Pavlovian-Instrumental Transfer Subjects and Apparatus. Sixteen experimentally naı¨ ve male Long Evans rats served as subjects. The housing conditions and apparatus were identical to those described above. The acclimation to EtOH was similar except that, since animals would ultimately receive training with sucrose reward as an alternative reward to EtOH, sucrose was never added to the EtOH solution. The rats were initially given 3 days of forced EtOH (10E), followed by 2 weeks in which the animals had free access to both water and the 10E solution 24 hours a day. Next 10E was made available for 1 hour per day at the time that the training sessions would occur for an additional 2 weeks. The rats had free access to water throughout this phase. In an attempt to match the extensive pre-exposure to EtOH, before training, animals were also pre-exposed to a 2% sucrose solution (2S; wt/v), which would serve as the other reward. Animals received 2 exposures to 100 mL of the sucrose solution in a drinking bottle overnight in the home cage which they readily consumed; water was also available. This amount was chosen to roughly match the total volume of EtOH consumed during pre-exposure. Pavlovian Conditioning. The Pavlovain training was similar to that of the first experiment, apart from the changes described below. The rats received 10 sessions of Pavlovian conditioning. Two auditory stimuli served as CSs. The clicker described above was used and the tone was replaced with a white noise stimulus. One of these stimuli (E1) was paired with EtOH delivery while the other stimulus (S1) was paired with sucrose (counterbalanced). Six presentations of each stimulus were given in each session in random order interspersed with periods in which no stimuli were present. The average length of the variable intertrial interval was 5 minutes. The stimulus presentations were 2 minutes long. During each stimulus, 0.2 mL of the appropriate outcome (10E or 2S) was delivered on a random time (RT) 30 seconds schedule. The number of magazine entries during each stimulus as well as in a pre-stimulus interval of equal length (2 min) was measured.
Experiment 3—Selectivity of Pavlovian-Instrumental Transfer With Natural Rewards Subjects and Apparatus. Sixteen experimentally naı¨ ve Long Evans rats served as subjects. The housing conditions and training apparatus were identical to those described above. All behavioral training and testing procedures were also identical to those described for Experiment 2 above, except that EtOH was not used as a reward; instead, a solution of 5% polycose plus 0.9% sodium chloride (wt/v) and the same 2S were the 2 rewards for both the Pavlovian and instrumental training phases. Statistics Data were analyzed with repeated measures analysis of variance (ANOVA) and paired t-tests were used to further assess significant main effects and interactions.
RESULTS
Experiment 1—Pavlovian-Instrumental Transfer and EtOH Reward Home-Cage EtOH Consumption. The average consumption for the final 3 days of limited (1 hr) access was 4.1 mL (SE 0.44 mL), which corresponds to a calculated mean EtOH intake of 0.82 g/kg (SE 0.08) confirming that the rats would voluntarily consume unsweetened EtOH in a 1-hour period. Pavlovian Training. The training data are displayed in Fig. 1A, inspection of which indicates that animals entered the magazine more during the CS1 than during the CS
CUE-INDUCED ETHANOL SEEKING
769
or pre-stimulus interval. An ANOVA comparing magazine entries during the CS1, CS , and pre-stimulus interval across days of training revealed an effect of stimulus [F(2, 18) 5 68.8, po0.01], no effect of training day [F(7, 63) 5 0.46, p40.05], and no interaction between these factors [F(14, 126) 5 1.7, p40.05]. Post hoc analyses confirmed that animals made more entries during the CS1 A
50 CS+ CS– pre-CS
Mean Magazine Entries
40
30
20
10
0 1
2
3
4
5
6
7
8
Day B 140 active lever inactive lever
Mean Lever Presses
120 100 80 60 40 20 0 1
2
3
4
5
6
7
8 RR 2
RR 1
9 10 11 12 13 14 15 RR 5
Day
Mean Lever Presses (CS - baseline)
C
8 CS+ CS– 6
4 * 2
0 Active
Inactive Lever
than during either the CS [t(9) 5 10.4, po0.01] or prestimulus interval [t(9) 5 9.95, po0.01], which did not differ from each other [t(9) 5 0.4, p40.05]. Probe trial data indicate that the animals developed an excitatory relationship between the CS1 and ethanol delivery: the rats made more entries during the CS1 (16.5 entries) than during the prestimulus interval (5.25) or CS (6.5 entries) even when the US was not present. Instrumental Training. As shown in Fig. 1B, the rats acquired the lever-press response for EtOH. Responding increased across days of training [F(14, 126) 5 5.3, po0.01] and the rats discriminated between the 2 levers, performing more active than inactive lever responses [F(1, 9) 5 13.0, po0.01]. This difference emerged across training days as evidenced by the interaction between training day and lever [F(14, 126) 5 7.1, po0.01]. As the ratio requirement increased, the animals performed more responses but the number of outcomes earned remained relatively stable. The average volume of EtOH consumed for the final 3 days of instrumental training was 3.8 mL (SE: 0.41), which corresponds to a mean EtOH intake of 0.64 g/kg (SE: 0.07). Transfer Test. Pavlovian-instrumental transfer examines the ability of reward cues to modulate instrumental performance. In this case, it was predicted that the EtOHpaired stimulus should invigorate responding; therefore, for the purpose of analysis, the number of responses made in the pre-stimulus or baseline period was subtracted from the number of responses made during the CS1 and CS . As such, positive numbers indicate positive transfer whereas numbers close to zero indicate no transfer effect relative to baseline. Presentation of the CS1 had a clear excitatory effect, enhancing responding on the active lever (Fig. 1C). This effect was specific to the CS1 as the CS failed to alter responding from baseline. Further, neither stimulus affected performance on the inactive lever. While the analysis revealed no effect of lever [F(1, 9) 5 4.6, p 5 0.06] or of stimulus [F(1, 9) 5 4.0, p40.05], there was a significant interaction between these factors [F(1, 9) 5 5.2, po0.05]. Post hoc analyses indicated that responding on the active lever was elevated more during the CS1 than the CS [t(9) 5 2.3, po0.05], and that the effect of the CS1 was specific to the active lever, as the CS1 elevated responding more on the active than the inactive lever [t(9) 5 2.4, Fig. 1. Presentation of an EtOH-paired stimulus elevates performance of EtOH directed responding. (A) Mean magazine entries during presentations of the CS1, CS , and pre-stimulus baseline across days of Pavlovian training. Rats learned the association between presentation of the CS1 and EtOH delivery as indicated by the increase in magazine entries during CS1 presentations relative to CS and pre-CS intervals. (B) Mean lever presses across days of instrumental training. The rats acquired the lever-press response for EtOH and increased responding on the active lever across days of instrumental training and as the random ratio (RR) response requirement increased. (C) Mean lever presses during presentation of the CS1 and CS relative to baseline in the Pavlovian-instrumental transfer test. Presentation of the CS1 selectively elevated responding on the active lever. There was no effect on responding on the inactive lever and the CS did not affect responding on either lever. Significant differences between stimuli (po0.05).
CORBIT AND JANAK
770
Fig. 2. Presentation of an EtOH-paired stimulus elevates responding for EtOH and sucrose suggesting a general enhancement of reward-seeking behavior. (A) Mean magazine entries during presentation of the EtOH and sucrose-paired stimuli across days of Pavlovian training. The rats increased magazine entries during both the E1 and the S1 relative to the pre-CS interval. (B) Mean lever presses for EtOH and sucrose across days of instrumental training. The rats acquired the lever-press responses delivering both EtOH and sucrose. (C) Mean lever presses during presentation of the EtOH-paired and sucrose-paired stimuli in the Pavlovian-instrumental transfer test. Responding on the EtOH lever was selectively elevated during presentations of the E1 but not S1 whereas presentation of both stimuli elevated responding on the sucrose lever to a similar degree suggesting that the E1 had a general excitatory effect on reward-directed behaviors. Significant differences between stimuli (po0.05).
A 100 E+ S+ pre-CS
80 Mean Magazine Entries
Home-Cage EtOH Consumption. The average consumption for the final 3 days of limited (1 hour) access was 3.9 mL (SE: 0.37 mL), which corresponds to a calculated mean EtOH intake of 0.75 g/kg (SE: 0.07). Pavlovian Training. Across days of training, the rats made more entries into the magazine during both the E1 and S1 than during the pre-stimulus baseline (Fig. 2A). The analysis confirmed an increase in entries across days [F(9, 135) 5 5.7, po0.01]. There was also an effect of stimulus [F(2, 30) 5 26.8, po0.01] indicating that the rats made fewer entries during the pre-stimulus interval than during presentation of the stimuli; further, there was an interaction between day and stimulus, indicating that this difference increased as training progressed [F(18, 270) 5 3.2, po0.01]. Instrumental Training. The rats acquired the lever-press responses for both EtOH and sucrose and responding increased across days [Fig. 2B; F(7, 105) 5 10.9, po0.01]. While the rats responded somewhat more on the lever delivering sucrose, this difference was not significant [F(1, 15) 5 3.8, p 5 0.07] and there was no interaction between training day and lever [F(7, 105) 5 1.5, p40.05]. The average volume of EtOH consumed for the final 3 days of instrumental training was 3.2 mL (SE: 0.52), which corresponds to a mean EtOH intake of 0.54 g/kg (SE: 0.08). Pavlovian-Instrumental Transfer. The data from the transfer test are displayed in Fig. 2C. There was no difference in the baseline responding on the ethanol (2.75 0.54) versus sucrose (3.75 0.78; Fo1) lever. Test data are presented as the responding during the stimuli with the baseline responding subtracted. There was no effect of lever (EtOH vs sucrose) [F(1, 13) 5 0.6, p40.05] but there was a significant effect of stimulus, indicating that the EtOH stimulus enhanced responding to a greater extent than the sucrose stimulus [F(1, 13) 5 5.9, po0.05].
60
40
20
0 1
2
3
4
5
6
7
8
9
10
Day B 300 sucrose EtOH
250 Mean Lever Presses
Experiment 2—Selectivity of EtOH Pavlovian-Instrumental Transfer
Additionally there was an interaction between stimulus and lever [F(1, 13) 5 5.5, po0.05]. Post hoc analyses confirmed that, for the EtOH lever, the E1 elevated responding to a greater extent than the S1 [t(13) 5 4.0, po0.01]. In contrast, for the sucrose lever, the E1 and S1 had a similar effect on responding [t(13) 5 0.3, p40.05].
200 150 100 50 0 1
2
3
RR 1
4
5
RR 2
6
7
8
RR 4
Day C 12 Mean Lever Presses (CS - baseline)
po0.05]. Responding on the inactive lever did not differ as a result of the stimulus presentations [t(9) 5 0.35, p40.05], and the effect of the CS was not different for the 2 levers [t(9) 5 1.1, p40.05].
E+ S+
10 8 *
6 4 2 0 E tO H
S uc ro se Lever
CUE-INDUCED ETHANOL SEEKING
A
Further, there was no difference in responding on the 2 levers during the E1 [t(13) 5 0.9, p40.05] but responding during the S1 was greater on the sucrose lever than on the EtOH lever [t(13) 5 2.43, po0.05].
100 P+ S+ pre-CS
80 Mean Magazine Entries
771
60
Selectivity of Pavlovian-Instrumental Transfer With Natural Rewards
40
Pavlovian Training. Over training, the rats increased magazine entries during the stimulus presentations (Fig. 3A). Specifically, the animals made more entries as training progressed indicated by the effect of training day [F(9, 135) 5 5.2, po0.01]. Further, the rats made more entries during the stimuli than the pre-stimulus interval [F(2, 30) 5 40.7, po0.01], and there was an interaction between stimulus and day indicating that the difference in the number of magazine entries during the stimuli compared with the baseline period increased across days [F(18, 270) 5 6.4, po0.01]. Instrumental Training. The rats acquired the lever-press responses for both polycose and sucrose and increased the number of responses for each across days of training [Fig. 3B; F(7, 105) 5 23.6, po0.01]. The animals made more responses on the polycose lever than on the sucrose lever [F(1, 15) 5 21.8, po0.01] and this difference increased across days resulting in an interaction between lever and training day [F(7, 105) 5 4.9, po0.05]. Pavlovian-Instrumental Transfer. The data from the transfer test are displayed in Fig. 3C. Statistical analysis revealed that there was no effect of either lever [F(1, 15) 5 1.97, p40.05] or stimulus [F(1, 15) 5 0.22, p40.05] but there was a significant interaction between these factors [F(1, 15) 5 16.23, po0.01]. Post hoc analyses revealed that responding on the polycose lever was elevated more by presentation of the polycose stimulus (P1) than by presentation of the sucrose stimulus (S1) [t(15) 5 2.8, po0.01]. Similarly, presentation of S1 enhanced responding on the sucrose lever to a greater extent that presentation of P1 [t(15) 5 2.9, po0.01]. Additionally, responding during the P1 was greater for the polycose lever than for the sucrose lever [t(15) 5 2.8, po0.05]. Responding during the S1 was numerically higher on the sucrose lever than the polycose lever but this difference failed to reach statistical significance [t(15) 5 1.1, p40.05]. However, more importantly, despite somewhat lower responding on the sucrose lever in general, a selective transfer effect was still seen (i.e., as noted above there was significantly more responding on the sucrose lever in the presence of the S1 than during the P1).
20
0 1
2
3
4
5
6
7
8
9
10
Day B 300 Polycose Sucrose
Mean Lever Presses
250 200 150 100 50 0 1
2
3
4
RR 1
5
6
RR 2
7
8
RR 4
Day
Mean Lever Presses (CS - base line )
C 12 P+ S+
10 8
*
6
*
4 2 0 Polycose
Sucrose Lever
Fig. 3. Pavlovian-instrumental transfer with natural rewards is outcome-selective. (A) Mean magazine entries during presentation of the sucrose-paired and polycose-paired stimuli across days of Pavlovian training. The rats increased magazine entries during both the P1 and S1 relative to the prestimulus interval. (B) Mean lever presses for sucrose and polycose across days of instrumental training (C) Mean lever-press responses during presentation of the polycose- and sucrose-paired stimuli in the Pavlovianinstrumental transfer test. Transfer was selective for both responses as responding on the polycose lever was elevated by the P1 to a greater extent than by the S1 whereas responding on the sucrose lever was elevated by the S1 to a greater extent than by presentation of the P1. Significant differences between stimuli (po0.05).
DISCUSSION
One of the central challenges to drug-abuse treatment is that even after long drug-free periods, the risk of relapse remains high. Understanding the factors that precipitate such relapse is critical if long-term treatment is to be successful and abstinence is to be achieved. A growing
772
number of studies suggest that environmental stimuli paired with previous drug use are importantly involved in maintaining performance of drug-seeking responses and may produce craving and subsequent relapse in abstinent drug users (Everitt and Robbins, 2005; Field and Duka, 2002; Heinz et al., 2004; Ludwig et al., 1974; Reid et al., 2006). However, attempts to assess cue-induced craving in human alcoholics or other drug users have produced mixed results, suggesting that the precise role of such cues is likely more complex than originally envisioned (Carter and Tiffany, 1999). The current series of experiments examined the ability of EtOH-predictive CSs to enhance EtOH-seeking using the PIT paradigm. The data from our initial experiment demonstrated that EtOH-paired cues invigorate performance of EtOH seeking responses. Experiment 2 examined the selectivity of the excitatory impact of an EtOH-paired stimulus by comparing it to a stimulus paired with sucrose reward. In this experiment, to further explore the specificity of any transfer effects, animals were trained to perform 2 responses, one earning EtOH and the other sucrose. The data from Experiment 2 indicated that a stimulus paired with a nondrug reward (sucrose) has a selective excitatory effect on responses that have previously delivered that same reward. In contrast, when the effects of a stimulus paired with EtOH delivery were examined, this stimulus was found to have a more general effect, elevating responding for both sucrose and EtOH. Our third experiment confirmed that selective transfer was observed when animals were trained with 2 natural rewards, suggesting that something unique about EtOH reinforcement, rather than the specific training parameters employed, produced the general PIT effect observed in Experiment 2. In many other paradigms used to examine the behavioral effects of drug-associated CSs, CS presentation is contingent upon performance of some response, which means that behavior must be initiated through some other mechanism and then may be maintained through conditioned reinforcement. The current result indicates the feasibility of using PIT for examining the motivating effect of an EtOH CS when the presentation of the CS occurs before the response. It is likely that EtOH-associated CSs can act both preceding and following the drug-seeking response to enhance drug-seeking behavior (Everitt and Robbins, 2005). One of the advantages of the PIT paradigm is that the association between the CS and reward is established independently from acquisition of the drug-seeking response. As such, this allows the examination of the effects of cue presentation on response initiation, choice between alternative responses, and levels of performance. As animals were trained with a single excitatory stimulus and single rewarded response in our initial experiment, we could not assess whether the elevated responding observed was due to the specific predictive relationship between the CS and EtOH, or to the general excitatory impact of this
CORBIT AND JANAK
stimulus following association with reward. The design of our second experiment allowed us to evaluate the selectivity of the excitatory impact of the EtOH-paired stimulus by comparing it with a stimulus paired with sucrose reward. Interestingly, the data from that experiment suggest that EtOH-paired stimuli have a general effect on reward-seeking behaviors as the excitatory impact of the EtOH CS extended beyond responses required for EtOH delivery. Specifically, the excitatory effect of the EtOH-paired stimulus also increased performance of a response that had previously earned a nondrug reward, consistent with the findings of Glasner et al. (2005). It is unlikely that these effects are the result of some detail of the training parameters employed because the sucrose-paired CS exerted a selective effect on responding for sucrose in this experiment, and because outcomeselective PIT was observed when 2 nondrug rewards were used, instead of EtOH, in our third experiment. The selective effect of cues paired with nondrug rewards observed in the current study is similar to multiple previous reports (Colwill and Motzkin, 1994; Corbit et al., 2001; Kruse et al., 1983). Overall, these results suggest that EtOH-paired CSs and natural reward-paired CSs may impact instrumental behavior through distinct mechanisms. In considering the reasons for this distinction, it may be important to consider what impact pre-exposure to EtOH may have on the subsequent conditioning. It is common in studies using EtOH reinforcement in animals for there to be substantial home cage exposure to EtOH before training for animals to acclimate to the taste of EtOH and become familiar with EtOH’s pharmacological effects. However, this exposure may lead to neural changes, which could affect subsequent conditioning. For example, studies have shown that long-term EtOH exposure can result in down-regulation of dopamine D2 receptors (e.g., Heinz et al., 2004; Volkow et al., 1996; Weiss et al., 2001) and given the proposed role for dopamine in reward-related learning (e.g., Schultz et al., 1997) dysregulation of this system as a result of EtOH exposure may alter later learning. In the current study, it is not clear whether the moderate pre-exposure was sufficient to alter dopamine systems or whether any alterations would be sufficient to change later learning. Another possible concern when evaluating the general excitatory effects of the EtOH stimulus is that the results may somehow depend upon the higher level of responding during the instrumental phase for sucrose reward than for EtOH reward, which may indicate a preference for sucrose over EtOH. Direct comparisons of EtOH and sucrose preference are difficult to derive from reinforced responding as the rats respond for and consume EtOH until a desired pharmacological effect is achieved whereas responding for sucrose is not similarly constrained. Regardless, we believe it is unlikely that any outcome preference can account for the differences in the transfer effects seen for EtOH and sucrose. First, if animals preferred sucrose one might expect that the stimulus associated
CUE-INDUCED ETHANOL SEEKING
with that outcome would have a stronger and more general effect on reward-directed responding. In fact, the opposite effect was actually observed; the EtOH-paired stimulus had a greater facilitatory effect on responding than the sucrose-paired stimulus. Second, in Experiment 3, the animals responded more for polycose than for sucrose during the reinforced instrumental training phase and yet a highly selective transfer was observed for each stimulusoutcome pair. This suggests that outcome preference based on amount of responding is not sufficient for producing a general transfer effect. The current findings are consistent with previous reports that Pavlovian cues for EtOH can contribute to EtOH consumption (Krank, 1989), can produce sign-tracking behavior (Krank, 2003), and can reinstate EtOH-seeking responses (Katner et al., 1999; Liu and Weiss, 2002; Nie and Janak, 2003; Zironi et al., 2006; Ciccocioppo et al., 2001). In particular, a previous study showed that an EtOH-paired stimulus enhanced responding for EtOH using a PIT design (Krank, 2003). Our findings extend these results in several ways by using an unsweetened EtOH solution, by using nonlocalized auditory stimuli that do not allow for simultaneous expression of sign-tracking behavior, and most importantly by testing whether EtOH-paired stimuli have general or outcome-specific effects on responding. Interestingly, the PIT effects reported herein differ from previous findings with discriminative stimuli for EtOH availability. Specifically, auditory stimuli have previously been shown to be relatively ineffective in reinstating EtOH-seeking (e.g., Katner et al., 1999). One notable difference in that study was that the olfactory or auditory stimuli were present throughout the self-administration session whereas our 2-minute stimuli have higher temporal contiguity with ethanol delivery, which may contribute to the ability to facilitate responding in the transfer tests. The distinction between general and outcome-specific arousal mechanisms may relate to the varied results reported in the human craving literature. For example, Carter and Tiffany (1999) describe how not all relapsing addicts report that subjective experiences of craving precipitated their relapse. However, Tiffany has suggested that craving may not always be conscious and thus may not necessarily be accurately captured by self-report (Tiffany, 1990, 1999). For example, abstinent alcoholic individuals may deny craving as part of a conscious decision for abstinence despite strong arousing or motivational effects of EtOH-related stimuli (Grusser et al., 2002). If drug and EtOH-related cues activate systems that increase arousal or general motivation rather than those that are involved in more specific cuing processes, then the presentation of these cues may have an activating effect without producing craving that is specific enough to be easily articulated by the relapsing addict. This is also consistent with the view that EtOH-seeking is a habitual behavior (Dickinson et al., 2002) mediated by automatic
773
action schemas which are elicited by cues associated with previous drug use (Tiffany, 1990) and thus is less readily under conscious control. Finally, the generality of the effects of the EtOH stimulus may relate to the observation that abstinent alcoholic individuals may increase their intake of other substances including sweets, caffeine, and nicotine as though craving for alcohol is transferred to other substances, although further study is clearly needed to investigate this intriguing possibility more completely (Junghanns et al., 2000, 2005). In summary, we found that an EtOH-paired CS can enhance EtOH-seeking behavior but can also generalize to other reward-seeking behaviors, and that the mechanisms that underlie the motivating effects of the EtOH-paired stimulus may be distinct from those mediating the impact of cues associated with nondrug rewards. Studies on the neurobiological bases of this effect will further clarify this possible distinction. In addition, studies on the role of cues paired with other drugs of abuse will shed light on whether our results reflect a unique property of EtOH reinforcement or whether drug-associated cues in general drive behavior through mechanisms distinct from those mediating the effects of cues predictive of natural rewards. ACKNOWLEDGMENTS
We are grateful to Michael Gill for his assistance and helpful comments on an earlier version of this manuscript. REFERENCES Carter BL, Tiffany ST (1999) Meta-analysis of cue-reactivity in addiction research. Addiction 94:327–340. Ciccocioppo R, Angeletti S, Weiss F (2001) Long-lasting resistance to extinction of response reinstatement induced by ethanol-related stimuli: role of genetic ethanol preference. Alcohol: Clin Exp Res 25: 1414–1419. Colwill RM, Motzkin DK (1994) Encoding of the unconditioned stimulus in Pavlovian conditioning. Anim Learn Behav 22:384–394. Corbit LH, Balleine BW (2005) Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. J Neurosci 25:962–970. Corbit LH, Muir JL, Balleine BW (2001) The role of the nucleus accumbens in instrumental conditioning: evidence for a functional dissociation between core and shell. J Neurosci 21:3251–3260. Di Ciano P, Everitt BJ (2005) Neuropsychopharmacology of drug seeking: insights from studies with second-order schedules of drug reinforcement. Eur J Pharmacol 526:186–198. Dickinson A, Wood N, Smith JW (2002) Alcohol seeking by rats: action or habit? Quart J Exp Psychol 55B:331–348. Everitt BJ, Robbins TW (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 8: 1481–1489. Field M, Duka T (2002) Cues paired with a low dose of alcohol acquire conditioned incentive properties in social drinkers. Psychopharmacology 159:325–334. Glasner SV, Overmier JB, Balleine BW (2005) The role of Pavlovian cues in alcohol seeking in dependent and nondependent rats. J Stud Alcohol 66:53–61. Grusser SM, Heinz A, Raabe A, Wessa M, Podschus J, Flor H (2002) Stimulus-induced craving and startle potentiation in abstinent alcoholics and controls. Eur Psychiatry 17:188–193.
774
Heinz A, Siessmeier T, Wrase J, Hermann D, Klein S, Grusser S, Flor H, Braus D, Buchholz HG, Grunder G, Schreckenberger M, Smolka M, Rosch F, Mann K, Bartenstein P (2004) Correlation between dopamine D2 receptors in the ventral striatum and central processing of alcohol cues and craving. Am J Psychiatry 161:1783–1789. Holland PC (1980) CS-US interval as a determinant of the form of Pavlovian appetitve conditioned responses. J Exp Psychol: Anim Behav Process 6:155–174. Junghanns K, Backhaus J, Tietz U, Lange W, Rink L, Wetterling T, Driessen M (2005) The consumption of cigarettes, coffee and sweets in detoxified alcoholics and its association with relapse and a family history of alcholism. Eur Psychiatry 20:451–455. Junghanns K, Veltrup C, Wetterling T (2000) Craving shift in chronic alcoholics. Eur Addict Res 6:64–70. Katner SN, Magalong JG, Weiss F (1999) Reinstatement of alcohol-seeking behavior by drug-associated discriminative stimuli after prolonged extinction in the rat. Neuropsychopharmacology 20: 471–479. Krank MD (1989) Environmental signals for ethanol enhance freechoice ethanol consumption. Behav Neurosci 103:365–372. Krank MD (2003) Pavlovian conditioning with ethanol: sign-tracking (autoshaping), conditioned incentive, and ethanol self-administration. Alcohol Clin Exp Res 27:1592–1598. Kruse JM, Overmier JB, Konz WA, Rokke E (1983) Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific. Learn Motivat 14:165–181. Liu X, Weiss F (2002) Reversal of ethanol-seeking behavior by D1 and D2 antagonists in an animal model of relapse: differences in antagonistpotency in previously ethanol-dependent versus nondependent rats. J Pharmacol Exp Ther 300:882–889. Ludwig AM, Wikler A, Stark LH (1974) The first drink: psychobiological aspects of craving. Arch Gen Psychiatry 30:539–547. Nie H, Janak PH (2003) Comparison of reinstatement of ethanol- and sucrose-seeking by conditioned stimuli and priming injections of allopregnanolone after extinction in rats. Psychopharmacology 168: 222–228. O’Brien CP, Childress AR, Ehrman R, Robbins SJ (1998) Conditioning factors in drug abuse: can they explain compulsion? J Psychopharmacol 12:15–22.
CORBIT AND JANAK
Overmier JB, Lawry JA (1979) Pavlovian conditioning and the mediation of behavior, in The Psychology of Learning and Motivation (Bower GH ed), pp 1–55. Academic Press, New York. Reid MS, Flammino F, Starosta A, Palamar J, Franck J (2006) Physiological and subjective responding to alcohol cue exposure in alcoholics and control subjects: evidence for appetitive responding. J Neural Transmiss 113:1519–1535. Rescorla RA (1994) Control of instrumental performance by Pavlovianand instrumental stimuli. J Exp Psychol AnimBehav Process 20: 44–50. Rescorla RA, Solomon RL (1967) Two-process learning theory: relationship between Pavlovian conditioning and instrumental learning. Psychol Rev 74:151–184. Samson HH (1986) Initiation of ethanol reinforcement using a sucrosesubstitution procedure in food- and water-sated rats. Alcohol Clin Exp Res 10:436–442. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599. Tiffany ST (1990) A cognitive model of drug urges ad drug-use behavior: role of automatic and nonautomatic processes. Psychol Rev 97: 147–168. Tiffany ST (1999) Cognitive concepts of craving. Alcohol Res Health 23:215–224. Tolliver GA, Sadeghi KG, Samson HH (1988) Ethanol preference following the sucrose-fading initiation procedure. Alcohol 5:9–13. Trapold MA, Overmier JB (1972) The second learning process in instrumental learning, in Classical Conditioning II: Current Research and Theory (Black AA, Prokasy WF eds), pp 427–452. Appleton-CenturyCrofts, New York. Volkow ND, Wang GJ, Fowler JS, Logan J, Hitzmann R, Ding YS, Pappas N, Shea C, Piscani K (1996) Decreases in dopamine receptors but not in dopamine transporters in alcoholics. Alcohol Clin Exp Res 20:1594–1598. Weiss F, Ciccocioppo R, Parsons LH, Katner A, Liu X, Zorrilla EP, Valdez GR, Ben-Shahar O, Angeletti S, Richter RR (2001) Compulsive drug-seeking behavior and relapse. Neuroadaptation, stress, and conditioning factors. Ann NY Acad Sci 937:1–26. Zironi I, Burattini C, Aicardi G, Janak PH (2006) Context is a trigger for relapse to alcohol. Behav Brain Res 167:150–155.
