Adolescent coping scales: A critical psychometric review

Scandinavian Journal of Psychology, 2008, 49, 533–548

Blackwell Publishing Ltd

DOI: 10.1111/j.1467-9450.2008.00669.x

Personality and Social Sciences

Adolescent coping scales: A critical psychometric review SIGRUN SVEINBJORNSDOTTIR1 and EINAR BALDVIN THORSTEINSSON2 1

University of Akureyri, Iceland University of New England, Australia

2

Sveinbjornsdottir, S. & Thorsteinsson, E. B. (2008). Adolescent coping scales: A critical psychometric review. Scandinavian Journal of Psychology, 49, 533–548. Individual coping is identified as an important factor in relation to health and well-being. Although several coping scales have been developed, key terms of coping such as nature and a number of primary and secondary factors (dimensions) are obscure. Coping scales, such as those that have been developed through exploratory factor analysis (EFA), have been criticized for poor psychometric properties, yet the critique so far does not evaluate development of the scales against best test-theoretical practice. The present study reviews six adolescent coping scales against ten detailed psychometric criteria in relation to statistical choices throughout the process of scale development. All six scales measured poorly on several criteria. Best practice had not been followed throughout their development and they suffered serious psychometric limitations. These findings indicate that there still is empirical research to be pursued in search of latent constructs and possible dimensions of coping through the implementation of EFA. Key words: Adolescence, review, coping scale, validity, reliability, best test-theoretical practice. Dr Einar B. Thorsteinsson, Psychology School of Behavioural, Cognitive and Social Sciences, University of New England, NSW 2351, Australia. Tel: + 61 2 6773 2587; fax: + 61 2 6773 3820; e-mail: [email protected]

Diverse psychosocial disorders (e.g., antisocial behavior, drug use, depression, eating disorders) in the Western world occur frequently in adolescence (Rutter, 1995; Smith & Rutter, 1995). The growing concern for adolescents who experience adjustment problems emphasizes the need for identification of protective and risk factors, and development of evidence-based preventions. Different types of individual coping strategies are identified as either protective or risk factors in relation to health and well-being (Lazarus, 1993; Masten, Best & Garmezy, 1990; Seiffge-Krenke, 1998). For instance, in adolescence, individual coping strategies are found to interact with well-being and adjustment (Jorgensen & Dusek, 1990; Tolor & Fehon, 1987), success at school (Kristensson & Öhlund, 2005; Rijavec & Brdar, 1997; Whatley, Foreman & Richards, 1998), and physical and mental health (Graue, Hanestad, Wentzel-Larsen, Sövik & Bru, 2004; Reid, Dubow & Carey, 1995; Reid, Dubow, Carey & Dura, 1994; Seiffge-Krenke & Klessinger, 2000; Smith, Smoll & Ptacek, 1990; Steiner, Erickson, Hernandez & Pavelski, 2002; Zeidner & Hammer, 1990). Lazarus’s cognitive transactional theory of coping (Lazarus & Folkman, 1984) has been a major inspiration for coping research during the last two decades. Based on that theory, a number of coping measures for adults (e.g., Billings & Moos, 1984; Carver, Scheier & Weintraub, 1989; Endler & Parker, 1990; Folkman & Lazarus, 1985, 1988; McCrae, 1984; McCrae & Costa, 1986; Pearlin & Schooler, 1978; Stone & Neale, 1984) and adolescents (e.g., Brodzinsky, Elias, Steiger, Simon & Gill, 1992; Dise-Lewis, 1988; Ebata & Moos, 1991; Fanshawe & Burnett, 1991; Frydenberg & Lewis, 1993; Seiffge-Krenke, 1989) have been developed. Researchers generally agree on the following issues related to coping: coping strategies (items)

cluster into primary coping factors; coping strategies may be of either a cognitive or behavioral nature; individuals either approach a problem or avoid it; and there is an interaction between coping strategies and situation characteristics. However, confusion still exists around other key issues within the field, and consensus on how to define and hence measure coping has not as yet been reached. For instance, Steger (2006) noted a different factor structure for each of the numerous post hoc factor analytical studies conducted between 1987 and 2000 on the Folkman and Lazarus scale of adult coping. Adolescent coping scales are also varied. For instance, while one researcher reports 80 strategies clustering into 18 primary factors (Frydenberg & Lewis, 1993), another suggests 20 strategies clustering into three factors (Seiffge-Krenke, 1989). Furthermore, conceptualization of second-order factors of coping is inconsistent in the literature. Lazarus and Folkman (1984) suggested that coping (adult) is either problem-focused or emotion-focused, the former aiming at changing the situation or managing the problem, and the latter at controlling the emotions related to the problem. Although Lazarus and Folkman did not present that dichotomy as statistically derived second-order factors, some researchers on adult coping have done so, and have suggested second-order factors relating either to the focus (e.g., Billings & Moos, 1981; Ebata & Moos, 1991, 1994; Pearlin & Schooler, 1978) or to the outcome of coping (i.e., positive or negative) (e.g., McCrae & Costa, 1986; Parkes, 1986). However, distinction between problem- and emotion-focused secondorder factors has not yet been statistically verified for adult or adolescent coping (Ficková, 1998; Jorgensen & Dusek, 1990; Tobin, Holroyd, Reynolds & Wigal, 1989; Watson, Willson & Sinha, 1998).

© 2008 The Authors. Journal compilation © 2008 The Scandinavian Psychological Associations. Published by Blackwell Publishing Ltd., 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. ISSN 0036-5564.

534 S. Sveinbjornsdottir and E. B. Thorsteinsson Researchers who use exploratory factor analysis (EFA) in psychological research are frequently found to make poor statistical decisions throughout the process of EFA (Conway & Huffcutt, 2003; Fabrigar, Wegener, MacCallum & Strahn, 1999; Ford, MacCallum & Tait, 1986), leading to spurious outcomes. Furthermore, psychological measures have been found to be rationally derived rather than developed through systematic quantitative content validation (Haynes, Richard & Kubany, 1995). Concerns about the methodological soundness of coping scales have been raised (Benson & Hagtvet, 1996; Compas, Connor-Smith, Saltzman, Thomsen & Wadsworth, 1999), and coping scales of children, adolescents, and adults have been reviewed (e.g., Compas, Connor-Smith, Saltzman, Thomsen & Wadsworth, 2001; Schwarzer & Schwarzer, 1996). Despite the fact that most coping scales are developed according to classical test-theory and through EFA, review of the development of coping scales in relation to test-theoretical criteria and the implementation of EFA has, to our knowledge, not yet been performed. In a recent exhaustive review of coping assessments and the search for the structure of coping, it is suggested that confirmatory rather than exploratory factor analyses be used for construction of primary coping factors, and confirmatory hierarchical analyses be used for determination of higher order factors or dimensions of coping (Skinner, Edge, Altman & Sherwood, 2003). However, we argue an investigation of the foundations of the scales is needed, requiring an analysis of the quality of the EFA procedures throughout scale development. Our hypothesis is that poor quality EFA procedures, as earlier detected in psychological research (Conway & Huffcutt, 2003; Fabrigar, Wegener, MacCallum & Strahn, 1999; Ford, MacCallum & Tait, 1986), might similarly be found in the development of coping scales. The aim of the present review is to critically examine statistical choices throughout the development of adolescent coping scales as suggested by classical best testtheoretical practice. A brief presentation of psychometric issues and criteria used for the present review is outlined below. See the summary in Table 1.

Best-practice as suggested by classical test-theory for development of a psychological test Item selection is the base of content validity. If a literature search provides limited knowledge, information from informed individuals is needed (Haynes et al., 1995; Kline, 1986). Test items must be clear and easily understood, and must be kept short, containing only one point or statement (DeVellis, 1991). For selection of the most salient items, up to twice as many items as will be included in the final version are needed in the initial stage as suggested by Kline (1986). To support replicability of a test, different numbers of items per factor have been suggested. MacCallum, Widaman, Zhang, and Hong (1999) have suggested a minimum of three to five items per factor, Fabrigar et al. (1999) have suggested four, while Gorsuch (1983) has suggested six items per factor. Four items per factor is the criterion for the present review.

Scand J Psychol 49 (2008)

Size and representativeness of the construction sample is vital for construct validity, and gender, age, socioeconomic background, and ethnicity must be known. The ratio of participants per test item affects the standard error, yet there is no clear rule to follow. Ten participants to each item has been considered safe (Child, 1990), and actually recommended (Nunnally, 1978). Yet some consider a ratio of five to one sufficient (Gorsuch, 1983; Tabachnick & Fidell, 1989) or even three to one for each gender (Kline, 1993). A recent study on best practice using EFA reported an error rate of 30% at a ratio of 20:1 on a sample size of 260 (Costello & Osborne, 2005), indicating an even greater importance of having a high ratio of participants per item than previously believed. The criteria for the present review of scale construction are a minimum of 10 participants per item (five or more per gender) and a minimum of 500 participants. Choices of EFA procedures concern (a) model of analysis, (b) factor rotation, and (c) factor extraction method. Model of analysis is most commonly a choice between common Factor Analysis (FA) and Principal Components Analysis (PCA), the default extraction method in SPSS. PCA has proven to obscure outcomes because of inflation of factor loadings, especially those of lower or moderate value, where FA has not (Fabrigar et al., 1999; Gorsuch, 1983; Widaman, 1993). Furthermore, there is no prior knowledge of unique variance within the correlation matrix when a test is being developed, and to uncover latent factors that are as uncontaminated by that kind of variability as possible, the choice of model of analysis should be FA rather than PCA (Conway & Huffcutt, 2003). The criterion for the present review is FA. Of a number of analytic rotation methods, the most essential distinction is between orthogonal and oblique rotation. Oblique rotation allows correlations between factors, and since characteristics of human behavior are believed to be interrelated (Child, 1990), the oblique factor rotation has been suggested as a first choice. The combination of oblique rotations and FA yields the simplest structure when compared with: oblique rotation and PCA; orthogonal rotation and PCA; and orthogonal rotation and FA (Fabrigar et al., 1999). This lends further support for the choice of FA and oblique rotation. Factor extraction is a challenge: extracting too many factors (overfactoring) or too few factors (underfactoring) leads to error. Two commonly cited extraction methods, often used in combination, are the latent root criterion (eigenvalue greater than one or Kaiser’s criterion) and Cattell’s scree test (Child, 1990; Tabachnick & Fidell, 1989). Kaiser’s criterion is generally not found to be a satisfactory method (Conway & Huffcutt, 2003; Fabrigar et al., 1999; Velicer & Jackson, 1990). Parallel Analysis (PA) is considered more accurate (Fabrigar et al., 1999; Hayton, Allen & Scarpello, 2004), especially when used in combination with a scree test. However, PA is not currently available in commonly used statistical programs like SPSS, although it is possible to conduct it stepwise (Hayton et al., 2004). Therefore, a scree test has been suggested by some as the best choice for the time being (Costello & Osborne, 2005), and is the criterion for the present review. For future development

© 2008 The Authors. Journal compilation © 2008 The Scandinavian Psychological Associations.

Criteria

Coping Across Situations Questionnaire (CASQ)

Coping Scale for Children and Youth (CSC-Y)

Adolescent Coping Scale (ACS) Long Form

Coping Responses Inventory, Youth Form (CRI-Y)a

0

0

–

–

√

–

√ √

√ –

√ √

√ √

√ √

√ NA

0

√

√

√

0

√

0 0

√ 0

√ √

√ 0

√ √

0 0

0

√b

√

√

0

0

0

0

0

0

0

NP

0 0

√ –

0 √

0 √

√ –

NA NA

0

0

√

√

–

NA

–

–

√

√

–

NA

0 NP

0 NP

√ NPc

√ NP

0 NP

0 NP

– – NP NP

0 0 √ –

– – NP NP

√ 0 0 √

√ – 0 0d

√ – √ –

0

NP

NP

√

NP

NP

0 –

– –

– –

√ –

– –

– –

NP – –

√ √ –

NP – –

√ √ 0

NP – –

√ √ –

535

Life Events and Coping Inventory (LECI)

Review of adolescent coping scales

Criterion 1 Items 1.1 Item selection for the scale representative (derived from literature and appropriate “focus” groups) 1.2 Each item one statement 1.3 Proportion of items in original pool/final test, if selected through EFA (less than 75%) 1.4 Ratio items to factor (4:1 or higher) Criterion 2 Sample 2.1 Size (500+) 2.2 Representativeness (i.e., age, gender, socio-economic background, ethnicity 2.3 Ratio participants to item (5+ per gender to item) Criterion 3 EFA procedure 3.1 Factor model (use FA unless small error variance then either FA or PCA) 3.2 Factor rotation (Oblique) 3.3 Factor extraction method (scree test, PA or both) Criterion 4 factors 4.1 Cut-off limit for item loadings (0.40 for PCA and 0.30 for FA if N < 1,000; 0.25 to 0.30 if N > 1,000 and participants to item >10:1) 4.2 Purity of primary factors (Each item to load above criterion on one factor only) 4.3 Internal consistency (0.70+) 4.4 Mean inter-item correlation (0.20 to 0.40) Criterion 5 Test-retest reliability 5.1 Sample size (100+) 5.2 Sample representativeness 5.3 Time interval (4 weeks+) 5.4 Correlation (0.70 if interval is ≤4 weeks, more than 4 weeks and less than 10 week 0.60, 10 weeks or more 0.50) Criterion 6 Evidence of concurrent validity 6.1 Correlation with test that measures same concept (r > 0.70) poor tests (r = 0.30 to 0.50) 6.2 Sample size (100+) 6.3 Sample representativeness Criterion 7 Evidence of construct validity 7.1 Correlation with unrelated test (≤0.30) 7.2 Sample size (100+) 7.3 Sample representativeness

Adolescent Coping Orientation (A-COPE)

Scand J Psychol 49 (2008)

© 2008 The Authors. Journal compilation © 2008 The Scandinavian Psychological Associations.

Table 1. Assessment summary of adolescent coping scales

Criteria Criterion 8 Cross-validation 8.1 Sample size (500+) 8.2 Sample representativeness 8.3 Ratio participants to item (5+ for gender) 8.4 At least 90% of items are replicated Criterion 9 Second-order factors in same/construction study 9.1 Factor model (FA used; PCA if error variance low) 9.2 Factor rotation (Oblique) 9.3 Factor extraction method (scree plot, PA or both) 9.4 Cut-off limit for primary factor loading on secondary order factors (0.40 for PCA and 0.30 for FA if N < 1,000; 0.25 to 0.30 if N > 1,000 and participants to item >10:1) 9.5 Purity of second-order factors (Primary factor should only load above criterion on one second-order factor) Criterion 10 Test practicality 10.1 Test length (50 items or less) 10.2 Reading level appropriate

Adolescent Coping Orientation (A-COPE)

Life Events and Coping Inventory (LECI)

Coping Across Situations Questionnaire (CASQ)

Coping Scale for Children and Youth (CSC-Y)

Adolescent Coping Scale (ACS) Long Form

Coping Responses Inventory, Youth Form (CRI-Y)a

√ – √ 0

– – – NP

√ – √ 0

– – – NP

√ 0 √ 0

0 – 0 0

NP NP NP NP

NP NP NP NP

NP NP NP NP

NP NP NP NP

0 √ √ √

NP NP NP NP

NP

NP

NP

NP

0

NP

0 √

√ √

√ √

√ √

0 √

√ √

Scand J Psychol 49 (2008)

Note: ACS-Long Form covers both Specific and General Form. NP = not performed; √ = Criterion met; 0 = Criterion not met; – = Information not sufficient or information totally missing; NA = Not applicable. a Factors constructed based on theory. b The exact number of males and females is not known; assumed to be equal. c Average item total correlations were reported, which were 0.57 for Factor 1, 0.59 for Factor 2, and 0.53 for Factor 3. d Correlation coefficients for the factors, General Form, were below 0.70 for 8 factors and above 0.70 for 10 factors. For the Specific Form, 10 factors had correlation coefficient below 0.70, and 8 factors had correlation coefficients above 0.70.

536 S. Sveinbjornsdottir and E. B. Thorsteinsson

© 2008 The Authors. Journal compilation © 2008 The Scandinavian Psychological Associations.

Table 1. Continued

Review of adolescent coping scales

Scand J Psychol 49 (2008)

of tests, PA combined with a scree test seems likeliest to be recommended as best practice. Item loadings indicate factors, but test developers need to be cautious. First, different models of analysis may produce loading results that differ for the same sample and variables. PCA may produce higher loadings than FA (Gorsuch, 1983), and the inflation is stronger for lower and moderate loadings, 0.40 or less, than for loadings of 0.80 or higher (Fabrigar et al., 1999; Sveinbjornsdottir, 2001; Widaman, 1993). Second, factor loadings are partly due to sample size, and correlations for larger samples are usually lower than for smaller samples (Tabachnick & Fidell, 1989). Therefore, a salient loading for one study may be inappropriate for another. For the present review, if FA is employed and the number of participants is at least 500, the loading criterion is 0.30 and as low as 0.25 if the number of participants exceed 1,000. If PCA is employed, the criterion for minimum loading is 0.40 for 500 participants or more. An item is expected to load saliently on one factor only, and a minimum internal consistency for a factor is 0.70 (Kline, 1993; Nunnally, 1978). However, since high internal consistency may carry a risk of items being only slight variations of each other (Child, 1990; Kline, 1993, 1994; Tabachnick & Fidell, 1989), inter-item correlation for a factor is optimal when ≥ 0.20 and ≤0.40 (Briggs & Cheek, 1986). Minimum criteria for test-retest analysis are a correlation of 0.70 (Kline, 1986) or even 0.80 (Kline, 1993), 100 participants, and a three-month interval. However, over a three-month period a significant change in cognition may occur, affecting the adolescents’ coping behavior. Therefore, with a minimum sample of 100 the correlation criterion for the present review is 0.70 for a four-week interval, 0.60 for a four- to ten-week interval, and 0.50 for an interval longer than ten weeks. For further evidence of validity, administered simultaneously to the same minimum sample of 100, the correlation between an examined scale and another one measuring the same variable (concurrent), is expected to be at least 0.70 unless one or both scales are of poor quality then only a moderate correlation of 0.50 is expected (Kline, 1986). Low correlation is expected (