Do Clinicians Follow a Risk-Sensitive Model of CapacityDetermination? An Experimental Video Survey SCOTT Y.H. KIM, M.D., PH.D., ERIC D. CAINE, M.D. JEFFREY G. SWAN, M.A., PAUL S. APPELBAUM, M.D.
The authors asked whether clinicians use a risk-sensitive model for decisional-capacity determinations; that is, whether a higher degree of capacity was required in higher-risk situations. The respondents were randomly assigned to view a videotaped “capacity” interview of a medicationrandomized clinical trial scenario (N⳱52) or a neurosurgical clinical trial scenario (N⳱47). A significant scenario effect was mediated by the respondents’ perception of scenario-specific risk. Respondents showed considerable disagreement within each scenario that was not explained by clinician-specific factors. Thus, clinicians, in fact, use the normative risk-sensitive model for capacity, but there remains considerable unexplained variability in their judgments. (Psychosomatics 2006; 47:325–329)
P
sychiatrists and other mental health professionals are often asked to assess the decision-making capacity of psychiatric and medical patients in treatment and research settings. Considerable research over the past two decades has begun to characterize the degree and nature of decisionmaking impairment in persons with schizophrenia,1–3 Alzheimer’s disease (AD),4–6 depression,1,7,8 and other medical disorders,9 and guidance has been provided to evaluators on the process of conducting an examination, including the functions that should be assessed because their impairment may result in decisional incapacity.10 However, the capacity-determination process involves more than an application of empirical evidence. It requires an integration of empirical factors (e.g., “How impaired is the patient?”) with a normative judgment (i.e., “Given the level of impairment, should this person be allowed to exercise his or her own choice?”). For many years, it has been argued that thresholds for categorical determinations of capacity should be sensitive to the risk/benefit profile of the decision-making situation.10 Although this risk-sensitive model is disputed by some philosophers, most authoritative sources endorse it.11 The model requires higher levels of decisional capacity when the risks are greater and the probability of benefit is less. The justification for what is somePsychosomatics 47:4, July-August 2006
times called a “sliding-scale” approach to decisional-capacity determinations has rested on a variety of considerations. Although everyone acknowledges that fully capable persons have the right to make whatever decisions they choose—even decisions that may be deleterious to their well-being—when persons show some degree of impairment, there may be reason for greater caution. Thresholds that are sensitive to the risk/benefit ratio favor the presumption that patients should receive beneficial treatment and avoid harmful options.10 Also, they offer greater assurance that persons who are faced with riskier choices in fact have sufficient capacity to make decisions that deserve to be respected. In this study, we examined whether the normative assumption that thresholds for decisional capacity should be based, in part, on the degree of risk inherent in the decision is reflected in the judgments of experienced clinicians who Received July 12, 2005; revised October 17, 2005; accepted October 20, 2005. From the Dept. of Psychiatry, the Bioethics Program, and the Center for Behavioral and Decision Sciences in Medicine, Univ. of Michigan; the Dept. of Psychiatry, Univ. of Rochester; and the Dept. of Psychiatry, Columbia Univ. Medical School. Send correspondence and reprint requests to Dr. Kim, 300 North Ingalls St., 7C27, Ann Arbor, MI 481090429. e-mail:
[email protected] Copyright 䉷 2006 The Academy of Psychosomatic Medicine
http://psy.psychiatryonline.org
325
Models of Capacity-Determination regularly perform capacity evaluations. Our survey focused on the research-consent context as the vehicle for testing the assumption. This issue is particularly relevant for research-consent decisions, since the goal of research is not the direct benefit of participants; this makes the assessment of the risks involved extremely important. We conducted an experimental survey of experienced clinicians, providing them with a rich source of information (a videotaped interview of a patient undergoing an often-used measure of decision-making abilities) and then eliciting their views. METHOD Subject Recruitment The subjects were recruited via the Academy of Psychosomatic Medicine (APM)’s mailing list. APM is the major professional organization for psychiatrists involved in consultation–liaison psychiatry, a group often called upon in medical settings to perform decisional-capacity assessments. A total of 629 letters of invitation were sent, with an enclosed postcard for response. The letter explained that the purpose of the survey was to find out “how various characteristics of the evaluation process (the level of patient performance, nature of the risk/benefit situation, the experience and background of the psychiatrist, etc.) affect the outcome of psychiatrists’ judgments” and described the randomized, experimental design of the survey. Twenty-four letters were “returned to sender;” 239 postcards were returned by the recipients; of these, 160 respondents were willing to participate in the study and were randomized into one of two groups (see below) and sent survey materials. Up to two reminders were sent to maximize responses; a total of 99 subjects returned the survey. Measures and Procedure The respondents were randomized to view one of two versions of a semistructured capacity-assessment interview (utilizing the MacArthur Competence Assessment Tool– Clinical Research12). The scripts of the two interviews were based on an actual patient interview from a previous study4 and were identical to each other, except that one version portrayed a randomized, placebo-controlled trial for a new medication for Alzheimer’s disease, whereas the other version portrayed a placebo-surgery controlled, randomized trial testing a neurosurgical cell transplant for AD. The scripts were acted out by a research assistant, who played the evaluator, and an actor, who portrayed a potential research subject. Using an actor allowed us to match 326
http://psy.psychiatryonline.org
the frequency and the manner of errors portrayed so that the subject’s performance level appeared identical in the two versions. The portrayed performance of the subject on the MacCAT–CR instrument was as follows (as scored by JS and SYHK): Understanding: 18/26, Appreciation: 4/6, Reasoning: 6/8, and Choice: 2/2. On the basis of previous experience, this represents a borderline level of performance on the MacCAT–CR in the context of a medication randomized controlled trial.4 This case was chosen because it is precisely in these “gray-area” cases where the risk/ benefit profile may make a significant difference in capacity determinations. Each video was approximately 20 minutes long and was sent to each respondent on a CD, along with study forms. After watching the interview, the respondents completed a written survey that had two parts: 1) questions regarding the respondents’ demographic characteristics, professional background, and training regarding capacity assessments (Table 1); and 2) specific questions regarding the decision-making abilities and capacity of the subject portrayed to provide informed consent for research participation, scenario-specific perceptions of risk and benefit, and a question regarding whether the clinicians felt the video provided a sufficient basis for their judgments (quality of video question; Table 2). The survey is available from the first author. Statistical Analysis Parametric (t-tests) and nonparametric (Mann-Whitney or chi-square) tests, depending on the nature and distribution of the data, were used to compare the baseline characteristics and responses of the two groups of respondents. To determine whether a significant scenario effect, if one exists, is due to the difference in the perception of risk, we conducted a mediation analysis by examining the relationships among the scenario variable, the respondents’ scenario-specific risk perceptions, and the overall capacity ratings. To explore the considerable within-group variation in the clinicians’ judgments that was found, post-hoc logistic-regression models were constructed, controlling for scenario, with demographic and professional background variables as predictors. This survey was deemed exempt from review by the institutional review boards of the University of Rochester, University of Michigan, and the University of Massachusetts Medical School. RESULTS The professional background information of the respondents is summarized in Table 1. Fifty-two clinicians anPsychosomatics 47:4, July-August 2006
Kim et al. swered the medication survey, and 47 answered the neurosurgical survey. The randomization was successful; the two groups were similar in demographics, professional background, and training in capacity evaluations.
TABLE 1.
Table 2 summarizes the responses of the two groups of psychiatrists on the basis of a review of their respective capacity interviews. The group that viewed the surgical version clearly de-
Respondents’ Demographic, Professional, and Training History Drug RCT (Nⴔ52)
Surgical RCT (Nⴔ47)
p
47.4 (10.7) 18 (34.6) 48 (92.3)c 14.9 (11.4)
48.8 (12.5) 13 (27.7) 47 (100) 16.8 (11.8)
0.55 0.46 0.15 0.44
31 (59.6) 12 (23.1) 15 (28.8) 14 (26.9) 28 (53.8) Median: 13.5 Mean: 31.6 SD: 52.6
26 (55.3) 12 (25.5) 16 (34.0) 8 (17.0) 31 (66.0) Median: 15.0 Mean: 33.1 SD: 50.9
0.67 0.78 0.58 0.24 0.22 0.89
2.4 (1.8) 3.1 (2.2) 2.3 (1.0)
2.8 (1.8) 3.3 (2.1) 2.7 (1.1)
0.26 0.74 0.16
Mean age,a years (SD) Women,b N (%) Professional background: psychiatrist,b N (%) Years in practice,a mean (SD) Practice setting,d N (%) Consultation–Liaison Inpatient Private outpatient Hospital outpatient Academic Capacity evaluations per yeare
Training in capacity assessments,e mean (SD) Number of lectures Number of supervised cases Quality (scale: 1–4; 1: poor; 2: adequate; 3: good; 4: excellent)
RCT: randomized, controlled trial; SD: standard deviation. a t-test. b chi-square test. c The other respondents in this group were three psychologists and one nurse. d The percentages exceed 100 because some respondents have more than one practice setting. Chi-square tests were used to compare the two groups for each practice setting. e Mann-Whitney test.
TABLE 2.
Comparison of Responses Regarding Decision-Making Abilities and Decision-Making Capacity for the Two Response Groups
Question
Medication RCT Version (Nⴔ52)
Surgical RCT Version (Nⴔ47)
pa
2.10 (0.57)
3.60 (0.54)
⬍0.001
2.15 (0.64) 5.52 (2.04) 5.75 (2.3) 5.04 (2.25) 6.85 (2.06) 5.52 (2.26)
1.91 (0.58) 4.54 (1.92) 4.24 (1.95) 4.11 (1.95) 7.20 (2.53) 4.36 (2.07)
0.04 0.02 0.001 0.04 NS (0.24) 0.01 0.022c
Level of risk to subject in the clinical trial (1: minimal; 2: minor increase over minimal; 3: moderate; 4: high) Likelihood of benefit to subject (1: none or little; 2: some; 3: moderate; 4: high) Understanding b Appreciation b Reasoning b Choice b Overall Capacity Rating b Categorical capacity judgment, N (%) Definitely not capable Probably not capable Probably capable Definitely capable Video gave sufficient information for capacity judgment? (1: strongly disagree – 5: strongly agree)
5 16 22 9
(9.6) (30.8) (42.3) (17.3)
3.48 (1.18)
12 20 11 3
(26.1) (43.5) (23.9) (6.5)
3.30 (1.06)
NS (0.34)
RCT: randomized, controlled trial. a Mann-Whitney test, unless otherwise indicated. b 1: absolutely incapable – 10: perfectly capable. c chi-square test.
Psychosomatics 47:4, July-August 2006
http://psy.psychiatryonline.org
327
Models of Capacity-Determination scribed a worse risk/benefit profile than the group viewing the medication version; the difference in risk-perception was large and significant, and the difference in the perception of potential benefit was smaller but still significant. On ratings of each decision-making ability, both groups gave similarly high ratings for ability to choose; on every other ability, those in the medication group gave a higher ability rating. This difference was reflected in the categorical capacity judgment as well, with 60% of the medication group respondents judging the subject in the demonstration to be probably or definitely capable, as compared with 30% of the surgical group. When asked whether the video provided a sufficient basis for making their capacity judgments, there was no difference between the two groups’ responses; both groups felt, on average, somewhere between “neither agree nor disagree” and “agree.” On mediation analysis using regression models, the scenario variable (i.e., medication versus surgical) predicted risk perception (b⳱1.51; standard error [SE]⳱0.112; p⬍0.001); risk perception predicted overall capacity rating (b ⳱ –0.75; SE⳱0.230; p⳱0.002), and the scenario variable predicted overall capacity rating (b ⳱ –1.11; SE⳱0.44; p⳱0.01). When the risk perception variable was added to the third model (scenario predicting overall capacity rating), the scenario variable no longer predicted outcome (p⳱0.91), whereas risk perception was a significant predictor (p⳱0.05). Given the experimental between-subject design of this survey, this analysis confirms the mediating role of risk perception in the main scenario effect. A similar analysis, testing the mediating effect of perception of benefit, was not significant. To better understand why, even within groups, there was such a divergence of opinions on the capacity status of the subject portrayed in the videos, we performed a forward, stepwise logistic-regression analysis, controlling for scenario group, with the following predictors: age, gender, years in practice, number of capacity evaluations per year, practice setting, training background, and perception of quality of the video as a basis for capacity judgments. Categorical capacity judgment (dichotomized for the logistic regression by collapsing the Probably and Definitely categories) was the dependent variable. None of the variables proved to be a significant independent predictor of capacity judgments. DISCUSSION Our findings confirm that clinicians with experience in capacity evaluations are clearly influenced by the risk/benefit 328
http://psy.psychiatryonline.org
profile of the patient’s decision-making situation. Several factors support the internal validity of this finding. First, by using an experimental between-subjects design, we avoided inducing an effect that could have resulted from a within-subject design. Second, the quality of stimulus provided to the respondents was high, as we used a 20-minute videotaped interview of a widely-used measure for assessing abilities relevant for informed consent to research. Third, we were able to confirm that the scenario effect is mediated through the respondents’ perception of scenariospecific risk. Fourth, although the respondents were heterogeneous, on average they had many years of clinical experience and conducted a median of over 13 capacityevaluations per year. There was considerable diversity among the experts’ views regarding the subject’s decision-making capacity. Those assigned to the medication scenario were split 40% incapable versus 60% capable, whereas those assigned to the surgical were split 70% versus 30%, respectively. We were unable to find plausible clinician-related factors that could explain the difference in capacity judgments. Being restricted to viewing a video, our respondents were unable to frame their own questions or follow up on what might have been ambiguous responses; such a procedure might have reduced the variability in responses. It is also possible that there are other clinician-related factors (such as their personal philosophical views on the how to balance patient autonomy and patient welfare) that could explain the discrepancy in scores. Furthermore, it is also possible that capacity-determinations are inherently imprecise judgments when the subject exhibits a performance level that falls in the “gray area” between capacity and incapacity. Since the case we presented to the clinicians was not a typical case, it should not be inferred that clinicians, in general, will have such disparity of judgments in most cases. Finally, some of the variability is likely due to the fact that although consultation-liaison psychiatrists are experienced in determining capacity in the treatment context, few have experience in determining capacity for research consent. The generalizability of this study is somewhat limited by the self-selected nature of the respondents. Just as a drug that shows efficacy in an randomized controlled trial of a highly self-selected sample of patients sometimes may not work as well when applied in the general clinic setting, it is possible that the risk-sensitivity of capacity judgments of this group of psychiatrists may not generalize well. Furthermore, this study focused on risks and potential benefits of participating in research studies testing treatment interPsychosomatics 47:4, July-August 2006
Kim et al. ventions. It is possible that the effect of the risk/benefit situation on the judgments of capacity evaluators may be different for other decision situations, for example, in the treatment context or in a different research context. However, the fact remains that these data indicate that riskperception is an important factor in judgments of capacity. What are the implications of this finding? These data offer the first evidence of what has long been assumed by theorists of decisional capacity: that evaluators’ judgments of capacity are responsive to the risk/benefit profiles inherent in the decisions. This may reflect a “common-sense” approach to capacity determinations that could be taken to support the rationale for a risk/benefit-sensitive capacity threshold. Since the ultimate legal determination of capacity, which occurs in that small minority of cases that enter the courts, is made by judges, a similar study with judges as the subject population could test whether they, too, follow the same risk/benefit-sensitive rule. Second, it may be possible, by studying the judgment patterns of clinician experts, especially if their reliability
can be enhanced, to infer generalizable capacity thresholds for stereotyped risk/benefit situations, such as research protocols that tend to have similar risk/benefit profiles for those eligible to enroll in the protocol. Such data could form the basis of an efficient and valid screening procedure that could be conducted by research assistants for potential research subjects’ decisional capacity. Finally, the basis for the substantial disagreement in judgments among clinicians needs to be further studied if we are to optimize the reliability of clinician judgments of capacity. This may form one part of a line of investigation to improve the capacity-judgments of clinicians by making them more reliable and transparent and reflective of societally-endorsed values. The authors thank those members of the Academy of Psychosomatic Medicine who generously participated in this study. This work was supported by a grant from the NIMH to Dr. Kim (MH64172).
References
1. Grisso T, Appelbaum PS: The MacArthur Treatment Competence Study, III: abilities of patients to consent to psychiatric and medical treatments. Law Hum Behav 1995; 19:149–174 2. Carpenter WT Jr, Gold J, Lahti A, et al: Decisional capacity for informed consent in schizophrenia research. Arch Gen Psychiatry 2000; 57:533–538 3. Palmer BW, Dunn LB, Appelbaum PS, et al: Correlates of treatment-related decision-making capacity among middle-aged and older patients with schizophrenia. Arch Gen Psychiatry 2004; 61:230–236 4. Kim SYH, Caine ED, Currier GW, et al: Assessing the competence of persons with Alzheimer disease in providing informed consent for participation in research. Am J Psychiatry 2001; 158:712–717 5. Marson DC, Ingram KK, Cody HA, et al: Assessing the competency of patients with Alzheimer disease under different legal standards: a prototype instrument. Arch Neurol 1995; 52:949–954 6. Karlawish JHT, Casarett DJ, James BD, et al: The ability of per-
Psychosomatics 47:4, July-August 2006
7.
8. 9.
10.
11. 12.
sons with Alzheimer disease (AD) to make a decision about taking an AD treatment. Neurology 2005; 64:1514–1519 Appelbaum PS, Grisso T, Frank E, et al: Competence of depressed patients for consent to research. Am J Psychiatry 1999; 156:1380– 1384 Lapid M, Rummans T, Poole K, et al: Decisional capacity of severely depressed patients requiring ECT. J ECT 2003; 19:67–72 Moser DJ, Schultz SK, Arndt S, et al: Capacity to provide informed consent for participation in schizophrenia and HIV research. Am J Psychiatry 2002; 159:1201–1207 Grisso T, Appelbaum PS: Assessing Competence to Consent to Treatment: A Guide for Physicians and Other Health Professionals. New York, Oxford University Press, 1998 National Bioethics Advisory Commission (NBAC): Research Involving Persons With Mental Disorders That May Affect Decision-Making Capacity, Volume 1. Rockville, MD, NBAC, 1998 Appelbaum P, Grisso T: MacCAT-CR: MacArthur Competence Assessment Tool for Clinical Research. Sarasota, FL, Professional Resource Press, 2001
http://psy.psychiatryonline.org
329
