Journal of Counseling Psychology 2004, Vol. 51, No. 2, 139 –150
Copyright 2004 by the American Psychological Association 0022-0167/04/$12.00 DOI: 10.1037/0022-0167.51.2.139
Complementary Approaches to Individual Differences Using Paired Comparisons and Multidimensional Scaling: Applications to Multicultural Counseling Competence Maria Darcy, Debbiesiu Lee, and Terence J. G. Tracey Arizona State University Multicultural research has traditionally involved normative methodology and definitions of individual differences. To further our understanding of multicultural concerns, the authors urge researchers to broaden the repertoire of methods used in these inquiries. First, the authors highlight the differences among normative, idiographic, and idiothetic approaches. Then, the authors introduce the use of paired comparison methods and multidimensional scaling techniques for use within these approaches. Last, examples of research using idiographic and idiothetic approaches with multicultural counseling competence as the focus are provided.
proaches to assist in multicultural inquiries. As a way to bridge the normative–idiographic divide, we then introduce the use of idiothetic approaches (Klinger, 1995; Lamiell, 1981), in which both the group and individual level of analysis are the foci. First, we highlight the differences between normative, idiographic, and idiothetic approaches. We proceed by positing that the following two main issues may explain why researchers have not generally adopted idiographic and idiothetic approaches to explore multicultural concerns: (a) a lack of familiarity with these approaches and (b) a lack of knowledge of the analytic tools that could be used. We then introduce paired comparison and multidimensional scaling (MDS) as examples of assessment and analytic tools associated with idiographic and idiothetic studies. In the last section, we provide some examples of research applications in the multicultural area. Although we see our argument appropriate to all counseling research, we focus specifically on using examples in the multicultural counseling competence area because (a) this is a multicultural content area that has several normative studies and no quantitative idiographic or idiothetic research, (b) the nature and measurement of the common constructs used continues to be debated, and (c) social desirability and the issue of bias are particularly salient in this domain.
Several leaders in multicultural research have underscored the importance of nontraditional methods of inquiry to explore multicultural concerns (Fuertes, Bartolomew, & Nichols, 2001; Helms, 2002; Ponterotto, 2002; Ponterotto & Alexander, 1996). They argue that reliance on traditional quantitative methods and definitions of individual differences may limit the understanding of multicultural concerns. Traditional definitions of individual differences involve the assessment of individuals on a common construct and then the comparison of scores relative to others in the sample or population, that is, the normative approach. Seeking to expand the range of approaches used to explore multicultural issues, some researchers call for the use of less traditional quantitative methods (e.g., Helms, 2002), whereas others champion more qualitative methods (e.g., Ponterotto, 2002), which eschew the common normative definition of individual differences. Despite these calls for alternative approaches, an overwhelming reliance has been placed on more traditional quantitative methods that focus on normative differences. Although certainly such inquiries have enhanced our understanding of multicultural concerns, we too suggest that expanding our methodological approaches and analytical tools will result in a more complex understanding of multicultural issues as they pertain to counseling. Moreover, despite our present focus on the multicultural domain, we suggest that research and practice in counseling psychology broadly would benefit from similar applications. Echoing Allport (1937) in the personality realm, we argue for the complementary adoption of normative and idiographic ap-
Editor’s Note. article.—JCH
Overview of Individual-Differences Approaches The normative–idiographic distinction is often characterized as one of amount versus type difference. Normative approaches focus on the extent different individuals score on a common construct (e.g., racial identity; Helms, 1990). Idiographic approaches focus on delineating how individuals uniquely construe constructs (Allport, 1937; Epstein, 1983; Falk, 1956). For example, in an idiographic approach, two individuals may construe acculturation very differently (e.g., one may use implicit values endorsed and another may use behaviors engaged in). It is thus impossible to compare these individuals with respect to the amount of acculturation, as each person views the construct in a nonoverlapping manner. Normative approaches thus facilitate generalizations across people, as there is a common construct, whereas idiographic ap-
Donald Atkinson served as the action editor for this
Maria Darcy, Debbiesiu Lee, and Terence J. G. Tracey, Division of Psychology in Education, Arizona State University. Correspondence concerning this article should be addressed to Terence J. G. Tracey, Division of Psychology in Education, Arizona State University, 302 Payne Hall, P. O. Box 870611, Tempe, AZ 85287-0611. E-mail:
[email protected] 139
DARCY, LEE, AND TRACEY
140
proaches highlight individual uniqueness in construal. This focus on more individual aspects of construal is why more idiographic approaches are being adopted in cognitive psychology (e.g., McFall, Treat, & Viken, 1997; Nosofsky, 1992; Treat et al., 2002). However, other than the application of Kelly’s (1955) conceptions of personal construct, there has been relatively little application of
idiographic approaches in psychology broadly, much less in the multicultural area. The only applications in the multicultural domain remain qualitative case studies. Combining aspects of normative and idiographic methodologies, an idiothetic approach allows differences on a common construct to be examined. In addition, an idiothetic approach allows assessment of the extent to
Table 1 Description of Normative, Idiothetic, and Idiographic Approaches Along With Exploratory and Confirmatory Examples Variable Focus Construct level Individual differences Advantages
Disadvantages
Normative
Idiothetic
Idiographic
Group Seeks common construct or principles across individuals. Construct amount differences. Scores relative to others on a common construct. Ability to generalize. Ability to compare. Ability to develop normative principles. Hard to apply to individual cases. No unique structural information.
Group and individual Seeks group construct and individual fit to that construct. Construct fit differences. Extent of individual fit to common construct. Ability to generalize and to provide more unique individual information than in normative approach.
Individual Provides information on how individuals uniquely construe the world. Construct type differences. Unique set of constructs for each individual. Rich information on the individual.
No unique structural information on individual’s idiosyncratic construals.
Unable to generalize beyond the individual. Unable to directly compare.
Exploratory examples
Approach
Analysis
How do clients think about the counseling relationship with an ethnically different therapist? Ask clients to write about their understanding of the counseling relationship. Identify themes and and design items based on those. Ask group to rate items on a Likert scale in terms of how the items relate to their experience of counseling relationships. Conduct an exploratory factor analysis on the ratings to uncover underlying common constructs.
How do clients think about the counseling relationship with an ethnically different therapist? Ask clients to suggest words they associate with counseling relationships. Organize the most frequently occurring words into a paired-comparison format and ask individuals to rate them for similarity when thinking about counseling relationships. Run a three-way unconstrained MDS to uncover common constructs and also examine each individual’s fit to this common structure.
How does a single client think about his or her counseling relationship with an ethnically different therapist? Ask your client to list words he or she associates with the counseling relationship. Randomly combine all these words in a paired-comparison format. Ask client to rate the degree of similarity of each pair when thinking about his or her counseling relationship. Run an unconstrained two-way MDS analysis to uncover unique constructs of each client.
Confirmatory examples From the exploratory study above, 4 factors were identified. Now we want to confirm those factors in a new sample via application of confirmatory factor analysis. Approach
Analysis
Use the same instrument giving it to a new group. Ask group to rate the items on a Likert scale in terms of how the items relate to their experience of the counseling relationship. Run a confirmatory factor analysis on the measure specifying the number of factors as 4. Assess variance accounted for and item loadings.
Note. MDS ⫽ multidimensional scaling.
From study above, a 3-dimensional model emerged. Now present a new group with the same pairedcomparison instrument, use a constrained 3-way MDS to see whether the same model is found. Ask individuals to rate the same pairedcomparison instrument, that is, rating item similarity when thinking about their counseling relationship.
How does the same client think about his or her counseling relationship 1 month later? Is he or she using the same constructs to think about this multicultural relationship?
Run a constrained 3-way MDS (specifying the original dimensions) to examine whether the original group structure held. Also obtain each individual’s fit with this structure to identify how they uniquely use the group dimensions to organize their own thinking.
Run a constrained MDS analysis (specifying the original dimensions) on the paired-comparison data to assess whether the client is thinking in the same manner; assess fit to the original structure. Alternatively run an unconstrained analysis and examine the results in relation to the original structure.
Present client with his or her original paired-comparison instrument. Ask client to rate their degree of similarity again when thinking about his or her counseling relationship.
COMPLEMENTARY APPROACHES
which the common structure is used by each individual. Table 1 summarizes these distinctions. The three approaches are different in their (a) focus (Is the focus on group or individual information?), (b) construct level (Is the construct structure common across the entire group, or does it vary across individuals?), and (c) definition of individual differences (Do individual differences exist as relative score differences on a common construct, or as unique constructs?). Normative approaches focus on groups, using common constructs on which relative scores are examined. Idiographic approaches focus on individuals, revealing unique constructs of different types. Idiothetic approaches focus on both groups and individuals and the differential endorsement of common constructs (i.e., how well does the common construct fit each individual, and how do individuals uniquely use the construct dimensions). As outlined in Table 1, each approach has its own advantages and disadvantages. Clearly, the normative approach enables easy comparison across individuals; however, it may not adequately describe any specific individual. The idiographic approach will result in better description of the individual, but at the expense of difficulty comparing this uniqueness across individuals. The idiothetic approach has many of the benefits of both the normative and idiographic approaches (ability to compare on a common construct, information on the fit of individuals to this common construct, and how individuals use the common dimensions). However, in the idiothetic approach there is no accounting for unique construals of individuals. Each of these approaches can be applied in a purely exploratory manner or from a very strict confirmatory stance. To demonstrate this, in Table 1 we have provided simple examples of each of the three approaches, one as an exploratory study (assuming little a priori knowledge) and another as a confirmatory approach (a priori specification of constructs and hypotheses and explicit examination of these).
Assessment and Analytic Tools We suggest that idiographic and idiothetic approaches are underused in counseling psychology in general and multicultural research specifically because of unfamiliarity with the assessment and analytic tools necessary for such studies. In the following section, we focus on the use of paired-comparison approaches as a method of generating data and, although there are several analytic tools that can be used to characterize paired-comparison data (e.g., cluster analysis and network analysis), we focus on the use of MDS for simplicity and ease of presentation. Paired comparison and MDS can be used in investigations from which to draw both normative and idiographic conclusions.
Paired-Comparison Approaches Paired-comparison approaches have been used in psychological research for decades. This is particularly true in the cognitive realm (Bradley, 1984; Martignon & Hoffrage, 2002; Nishisato, 1994). A distinct advantage of paired-comparison approaches is that they have the potential to tap into the underlying dimensionality of psychological and sociological constructs that are salient to individuals and/or groups even though such dimensions may not be known to the respondent. An associated advantage is that, with paired-comparison tasks, biased responding is minimized because of the nature of the
141
task; that is, respondents are often unaware of what is being assessed. This feature is especially relevant when it comes to multicultural research in which the pressure to represent oneself in a socially desirable manner is often high (e.g., Devine, 1989). Additionally, paired-comparison tasks can obviate biasing factors associated with self-rating scales such as central tendency and the halo effect. Unlike the common assessment strategy of asking individuals to rate the presence of, frequency of, or agreement with a particular item, paired comparison focuses on having the individual rate the degree of similarity (or dissimilarity) between two different items or stems (e.g., knowledge of other cultures vs. knowledge of my own culture). In a paired-comparison task, the individual makes similarity ratings of all possible pairs of items or stems of interest. The specific items or stems can be either researcher provided or individual-generated. For example, in a study of prejudice, individuals may be asked to list the words they associate with this construct and then rate all possible pairs for their degree of similarity in relation to the construct prejudice. Thus, individuals have their own unique set of items. However, the researcher may provide words associated with prejudice and respondents are asked to rate all possible pairings for their degree of similarity, thus the items are similar across individuals. Respondents’ pairwise ratings can then be converted into numbers or “proximities” that represent the degree of perceived or actual similarity (or dissimilarity) between all pairs of items or stems within a set. The proximities can be arranged into data matrices. Thus, a researcher can have a set of matrices in which each matrix represents the similarity ratings of an individual member of a larger group. Alternatively, a single matrix may be produced in which the ratings of one person or a group of aggregated individuals are of interest (Kruskal & Wish, 1978; Treat et al., 2002).
MDS Analysis MDS techniques encompass a group of data analytic methods that can be used to uncover or represent the structure of proximity data (Buser, 1989; Kruskal & Wish, 1978; MacCallum, 1988; Treat et al., 2002). Unlike most analytic tools used to examine structure (e.g., factor analysis), MDS can be applied to a broad range of data from individual responses to group responses, allowing both idiographic and normative conclusions to be asserted. In the psychological context they can be used to uncover or represent ways of thinking or perceiving. MDS has been used to investigate how individuals think about vocational interests (Rounds & Zevon, 1983), control and influence in counseling (Tracey, 1991), emotions (Morgan & Heise, 1988), self-concept (DeSteno & Salovey, 1997), gender (Eckes, 1994), romantic involvement (Rusbult, Onizuka, & Lipkus, 1993), and love (Hassebrauck & Buhl, 1996). To date it has only been used in a few studies in the multicultural domain (e.g., Helms, 1990). MDS models generate a geometric representation or model of the proximity data that explains its organization by the least number of interpretable dimensions, usually in Euclidean space— dimensions are the MDS equivalent to factors in a factor analytic study (Young & Hammer, 1987). The geometric representation displays the individual data items arranged in reference to each other and relative to the main dimensions. In such representations, the larger the distance between points or items, the more dissimilar they are perceived to be. Two types of MDS models are of
142
DARCY, LEE, AND TRACEY
particular interest: two-way and three-way models (Arabie, Carroll, & DeSarbo, 1987; Kruskal & Wish, 1978). Two-way models work with a single proximity matrix that can be from an individual or an aggregated group (Arabie et al., 1987). Two-way models provide a number of indices with interpretive value: variance accounted for (VAF) by the model, VAF by the particular dimensions, and item–stem coordinates on the particular dimensions. Three-way analyses represent the data of more than a single input matrix and also have a number of indices of interest: VAF by the group model, VAF by the specific dimensions, item–stem coordinates on the particular dimensions for the group model, model fit indices indicating how well the model fits for each individual, and salience weights indicating how each individual uses the group dimensions. We chose to focus on only one analytic technique for idiographic and idiothetic studies, that of MDS. Given space limitations, we were only able to discuss its applications and logic briefly. Readers are referred to Kruskal and Wish (1978), Arabie et al. (1987), and Young and Hammer (1987) for more complete descriptions of the method. There are also several other data analytic approaches to idiographic and idiothetic analysis that we do not address here. As noted earlier, Kelly (1955) developed the grid method to assess idiographic models of individual conceptualization. Cluster analysis (Aldenderfer & Blashfield, 1984) and concept mapping (Schvaneveldt, Durso, & Dearholt, 1989) can each be applied to the similarity matrices generated in an idiographic study, and because each starts with different assumptions, each can provide different representations of idiographic structures. Only three-way techniques can be used to yield information at the idiothetic level. This is where MDS is most useful, although there are some three-way clustering models (e.g., INDCLUS; Arabie et al., 1987) that can be used in these contexts (e.g., Tracey & Rounds, 1993). Interested readers are directed to the sources noted for further information on these approaches. To demonstrate the utility of paired-comparison approaches and MDS techniques for multicultural research, we provide sample applications of these in two examples in the multicultural counseling competence content area: one idiographic and one idiothetic.
Multicultural Counseling Competence Sue et al. (1982) developed a tripartite model of multicultural counseling competence that highlighted three important elements for multiculturally competent counselors—attitudes and beliefs, knowledge, and skills. The original model was later updated by Sue, Arredondo, and McDavis (1992; see also Sue et al., 1998). To date, the majority of research surrounding the multicultural competencies has centered on the development of instruments to assess this construct (Fuertes et al., 2001). In the past decade, four such measures have been introduced into the literature. They include the Cross-Cultural Counseling Inventory—Revised (LaFromboise, Coleman, & Hernandez, 1991), Multicultural Awareness Knowledge and Skills Survey (D’Andrea, Daniels, & Heck, 1991), Multicultural Counseling Inventory (Sodowsky, Taffe, Gutkin, & Wise, 1994), and Multicultural Counseling Awareness Scale (Ponterotto, Sanchez, & Magids, 1990; Ponterotto et al., 2002). All four instruments have been based, to some extent, on the Sue et al. (1998) model. Although items for each of the instruments have
been derived from the tripartite (attitudes and beliefs, knowledge, and skills) model of multicultural competence, validation studies do not support this construction (LaFromboise et al., 1991; Ponterotto, Gretchen, Utsey, Rieger, & Austin, 2002; Ponterotto, Rieger, Barrett, & Sparks, 1994). Each instrument deviates to some extent from the tripartite model of competence upon which they are based. After reviewing these multicultural counseling competence instruments, Ponterotto and Alexander (1996) concluded that the multicultural counseling competence construct itself needs further explication and empirical support. They also called for additional research that would broaden the methodological base used to research this construct. Although many studies of multicultural counseling competence have used normative methodological approaches, few have incorporated idiographic or idiothetic approaches. We therefore chose this topic to underscore the benefits of incorporating idiographic and idiothetic approaches in multicultural research. The construct of social desirability has also proven to be problematic relative to the valid assessment of multicultural competence. Research has demonstrated that many of the multicultural self-rated instruments are highly related to social desirability and that the social desirability affects the validity of the scales (Constantine & Ladany, 2000; Ponterotto et al., 2002; Sodowsky, Kuo-Jackson, Richardson, & Corey, 1998; Worthington, Mobley, Franks, & Tan, 2000). Clearly, researchers in the area of stereotypes (a key component of multicultural competence) have long recognized the bias involved in self-rated assessments of perceptions of different cultural groups to such an extent that it is rare that self-ratings are used. A common approach now is to use a priming design, whereby a subliminal message is presented focusing on different cultures and then assessing reaction times in various tasks (Devine, 1989). Such an approach circumvents conscious processing and potential deception. An example of a priming design applied to therapists is a study conducted by Abreu (1999), in which he found support for the presence of unconscious stereotypes in client evaluations. In studies examining the multicultural counseling competence of graduate students, researchers have attempted to take into consideration the effects of social desirability either by providing items that check specifically for social desirability (Ponterotto et al., 2002) or by supplementing the self-report instruments with measures designed to capture socially desirable responding (Sodowsky et al., 1998). Given the presence of social desirability, the general biases inherent in self-rated assessments, as well as those associated with self-ratings of racial or cultural attitudes, self-rated assessments are somewhat suspect. We propose the paired-comparison methodology as another means of circumventing these biases associated with self-ratings of cultural attitudes. As described, pairedcomparison assessments with MDS analyses enable assessment of nonconscious processes (Nosofsky, 1992). Such methods seem especially appropriate to multicultural contexts in general and multicultural counseling competencies in particular.
Idiographic Example We adopted an exploratory, idiographic approach in this example. We were interested in discovering how individuals uniquely construe multicultural counseling competence. We approached this
COMPLEMENTARY APPROACHES
investigation from the bottom up, that is, using the experience and terminology of the individual participants. We wanted to make as few assumptions as possible and see how the resulting structures compared with one another as well as how they resembled our reigning models.
Method Three individuals at a southwestern university were invited to participate. Participant A (PA) was a 29-year-old White male who was in his third year in a counseling psychology doctoral program. Participant B (PB) was a 26-year-old Asian American female who was also midway through the same doctoral program. Finally, Participant C (PC) was a 49-year-old White male professor in the counseling psychology doctoral program at the same university. The 3 individuals were chosen because they had very different experiential and professional backgrounds. This allowed us to explicate the utility and strengths of an idiographic approach in that we expected to discover different and uniquely individualized construals of multicultural counseling competence. As stated earlier, idiographic approaches do not seek to generate representative, generalizable principles, rather the goal is to capture the individual in as much detail as possible, similar to the rich thick descriptions sought in a qualitative approach. Each participant was asked to list approximately 10 components he or she found important to effective multicultural counseling. Nine to 15 stems were generated by each respondent. The next stage involved constructing a paired-comparison instrument for each participant who randomly paired his or her unique stems with each other. This resulted in three unique paired-comparison measures, a different one for each participant. The participants were then asked to rate the similarity of his or her paired stems on a 7-point scale ranging from 1 (not very similar) to 7 (very similar).
Results The participant’s similarity ratings of the unique pairedcomparison items were put into a separate matrix for each participant. Thus PA’s similarity ratings for all combinations of his 9 items were constructed into a 9 ⫻ 9 matrix, PB’s ratings for her 14 items were constructed into a 14 ⫻14 matrix, and PC’s ratings for his 15 items were constructed into a 15 ⫻ 15 item matrix. These three matrices reflected the psychological distance or similarity among each participant’s unique items. Separate two-way nonmetric MDS analyses (Kruskal & Wish, 1978) were performed on each of the three matrices using the Sindscale program (Pruzansky, 1975). The purpose of using MDS in this example was to determine the underlying dimensions each individual used in conceptualizing multicultural counseling competence. From the analysis, a graphical representation or model of each participant’s thinking about his or her self-generated items was produced. Model generation. MDS models were produced for one-, two-, and three-dimensional solutions for each participant. Similar to the selection of factors in factor analysis, choosing the number of dimensions to represent the data optimally involves using the following criteria: the VAF, parsimony, and interpretability (Kruskal & Wish, 1978; MacCallum, 1988). The VAF for PA’s data for one-, two-, and three-dimensional models was .66, .78, and .68, respectively. The two-dimensional model accounted for only slightly more variance than the one- or three-dimensional models. The one-dimensional model was more easily interpretable than the two-dimensional model, and given the small increment of the two-dimensional model over the one-dimensional model, we chose the one-dimensional model. So, given the principles of
143
interpretability and parsimony, we selected the one-dimensional model as the best representation of PA’s data. The magnitude of the VAF (.66) indicates that PA’s data were well represented. For PB, the VAF for one-, two-, and three-dimensional models was .37, .48, and .42, respectively. The increase in fit to the data of two-dimensional models over the one- dimensional model was about the same as it was for PA’s data; however, examination of the results revealed that for PB’s data, the two-dimensional model was much more easily interpreted. We had difficulty making sense out of the one-dimensional model. Hence, we selected the twodimensional model as the best representation of PB’s data. The lower VAF value for PB’s data (.48) indicates that the model fits moderately. Last, the VAF for PC’s one-, two-, and three-dimensional models was .40, .70, .and 76, respectively. In terms of the competing criteria of fit to the data and parsimony, the two-dimensional model was selected as being the best representation of PC’s data. This too proved to be the most interpretable of the three possible representations. The VAF of .70 indicated that PC’s data were well described by the two-dimensional model. Figures 1, 2, and 3 display a graphical representation of each participant’s model. Items that were perceived to be more similar are found in closer proximity, whereas those that were perceived to be more different are found further apart. Model interpretation. Once the optimal dimensional configuration was determined, the next step involved interpreting, or attaching meaning to those dimensions. To do this, how each individual’s unique items were positioned in space was examined, juxtaposed to the dimensions (Kruskal & Wish, 1978) and then matched with information about the characteristics of the items themselves, in a manner similar to interpretation of factors in factor analysis. For example, items that most strongly represented PA’s one-dimensional model were G, A, and H at one end, and E and I at the other end. Given the nature of these items, we hypothesized two possible interpretations (hereafter referred to as interpretive vectors) for this unidimensional and possibly bipolar view of multicultural competence adopted by PA: (a) ability to understand the client’s culture and (b) creating an open and supportive environment. To test the validity of these hypotheses, a property vector fitting technique using multiple regression analysis was applied (Kruskal & Wish, 1978). To conduct the regression analysis, each unique item generated by PA was rated by him in terms of how closely it corresponded to the interpretive vectors (i.e., ability to understand the client’s culture, and creating an open and supportive environment). Ratings ranged from 1 (very unrepresentative) to 7 (very representative). These total ratings were then separately regressed onto the items’ coordinates on the dimension resulting from the MDS analysis of PA’s data. Ability to understand the client’s culture accounted for 59% of the variance of the items in PA’s model, F(1, 7) ⫽ 12.50, p ⬍ .05, whereas creating an open and supportive environment accounted for 16% of the variance of the items, F(1, 7) ⫽ 2.56, p ⬍ .05. Thus, the regression analysis revealed that PA conceived multicultural counseling competence in a bipolar manner, with greater emphasis on the ability to understand the client’s culture. PB’s model (see Figure 2) seemed more complex than PA’s, and more easily interpretable using two dimensions, although the twodimensional model accounted for only 48% of the variance. The
144
DARCY, LEE, AND TRACEY
Figure 1. Graphic depiction of Participant A’s construal of multicultural competence using two-way multidimensional scaling. Letters represent item symbols.
first dimension was differentiated by Items G and A on the one pole and Item C on the opposite pole. This dimension appeared to capture the therapist’s culture on one extreme and the client’s culture on the other. To conduct the regression analysis to aid interpretation of the model, each item generated by PB was rated by her in terms of how closely it corresponded to the interpretive vectors, that is, therapist’s culture and client’s culture. Ratings ranged from 1 (very unrepresentative) to 7 (very representative). These total ratings were then separately regressed onto the items’ coordinates for the two dimensions from the MDS results. Both interpretive vectors were found to be statistically significant for Dimension 1, with therapist’s culture accounting for 62% of the variance, F(2, 11) ⫽ 11.50, p ⬍ .05, and the client’s culture accounting for 54% of the variance of items for Dimension 1, F(2, 11) ⫽ 8.62, p ⬍ .05. Therefore, for PB, Dimension 1 appeared to be bipolar and seemed to reflect whose culture was the focus, with the client’s culture on one extreme and the therapist’s culture on
the other. These interpretive vectors are plotted on the MDS model in Figure 2 by using the regression weight for the particular dimensions to locate the line. The length of this line represents the strength of the relation, and the location represents similarity. The closeness of the therapist culture and the client culture vectors to the opposite poles of Dimension 1 in Figure 2 support its interpretation as a bipolar dimension representing therapist culture versus client culture. Items that were most representative of Dimension 2 for PB included on each extreme Stems I and M. Therefore, Dimension 2 was also thought to be bipolar, reflecting the emphasis on the interaction between client and therapist, with one extreme emphasizing therapist-directed interventions and the other representing therapist and client collaborative interactions. The regression analysis revealed that the emphasis on client–therapist interaction was statistically significant, F(2, 11) ⫽ 5.90, p ⬍ 05, and accounted for 21% of the variance of items on Dimension 2. Although the
COMPLEMENTARY APPROACHES
145
Figure 2. Graphic depiction of Participant B’s construal of multicultural competence using two-way multidimensional scaling. Item stems are used for selected items. Letters represent item symbols. A ⫽ Accurately empathizing with the client’s feelings; B ⫽ Recognizing the cultural bias of counseling; C ⫽ Recognizing how my own culture and cultural values will influence interaction; D ⫽ Recognizing how the client’s social status is impacting the client; E ⫽ Recognizing how my own social status is impacting me; F ⫽ Trying to view the world from the client’s perspective; G ⫽ Recognizing that the client’s presenting problems are culture bound; H ⫽ Making recommendations that are appropriate for the client’s situation and culture; I ⫽ Making recommendations that are empowering for the client; J ⫽ Affirming the client’s worldview, lifestyle, and culture; K ⫽ Being able to locate the client’s and my own cultural values and biases; L ⫽ Being able to state the differences between mine and the client’s cultural values and biases; M ⫽ Exploring with the client how our cultural differences will impact the working relationship; N ⫽ Recognizing how the interventions I make will be received by the client’s culture.
explanation for Dimension 2 is significant, it appears to be less descriptive than the explanation for Dimension 1. This suggests that there may be another construct that better explains Dimension 2. Overall, two factors appeared to be important in PB’s conceptualization of multicultural counseling competence: (a) focusing on the client’s and therapist’s cultures and (b) an emphasis on client–therapist interactions, although an alternate explanation for Dimension 2 may also apply. Similar to PB, PC’s items were also most easily interpreted using a two-dimensional model (see Figure 3), and this accounted for 70% of the variance. Dimension 1 appeared to reflect the degree to which the focus was on the client’s experience (Item B), or the therapist’s experience (Item F). A regression analysis to aid interpretation revealed that focus on the client’s experience accounted for 72% of the variance, F(2, 12) ⫽ 18.98, p ⬍ .05, whereas focus on the therapist’s experience accounted for 54% of the variance, F(2, 12) ⫽ 9.28, p ⬍ .05. Therefore, client’s experience and therapist’s experience represented the opposite poles of Dimension 1. These interpretive vectors are plotted on the MDS model in Figure 3, again with the length of the line representing strength of the relation, and location representing similarity. The items that appeared to strongly represent Dimension 2 in PC’s model included Item C on the one extreme and Item M on the
opposite extreme. This dimension appeared to reflect the degree to which the focus of counseling was on the minority or majority culture, or in PC’s conceptualization, the client’s culture and the therapist’s culture. Focus on the therapist’s culture accounted for 56% of the overall variance in items, F(2, 12) ⫽ 9.71, p ⬍ .05, and focus on the client’s culture explained 45% of the variance in items, F(2, 12) ⫽ 6.89, p ⬍ .05. Therefore, the model for multicultural counseling competence that PC uses includes two bipolar dimensions, a bipolar focus on both the therapist’s and client’s experiences on one dimension as well as the culture of the (minority) client and (member of the majority) therapist on the other dimension.
Discussion The above research is an example of an idiographic approach, and although interesting results were revealed, no generalizations beyond these individuals can be made. However, we believe the results of these idiographic examples could serve as a foundation for future research. The results indicated that the 3 individuals had unique and distinctive ways of construing multicultural counseling competence. The idiographic approach allowed individuals to state what they believe is important for effective multicultural counsel-
146
DARCY, LEE, AND TRACEY
Figure 3. Graphic depiction of Participant C’s construal of multicultural competence using two-way multidimensional scaling. Item stems are used for selected items. Letters represent item symbols. A ⫽ Understanding the client’s experience; B ⫽ Understanding the client’s experience with me; C ⫽ Understanding the client’s experience with others of the same culture; D ⫽ Understanding the client’s experience with others of the majority culture; E ⫽ Understanding the salience of culture to the client; F ⫽ Understanding my biases; G ⫽ Understanding how my biases affect the client; H ⫽ Understanding what the client will not reveal; I ⫽ Assisting the client to resolve issues; J ⫽ Assisting the client to make and implement decisions; K ⫽ Assisting the client to affect change in self; L ⫽ Assisting the client to affect change in relating to others of own culture; M ⫽ Assisting the client to affect change in relating to others of the majority culture; N ⫽ Being able to alter my behavior to fit with the client; O ⫽ Being able to alter my behavior to fit with the client’s culture.
ing, so that the results obtained were highly individualized. For PA, the unidimensional model reflects his belief that the important element in multicultural counseling competence is the counselor’s ability to understand the client’s culture and to provide a supportive environment in which to nurture a therapeutic relationship. These factors contrasted with a general empathic understanding. PB appeared to have a more complex conceptualization of multicultural counseling competence. She used two issues in construing this construct, namely, the ability to consider the client’s culture as opposite from considering the therapist’s culture and how these cultures intersect with the client– counselor interaction. PC had a similar conceptualization to PB, in terms of how the dimensions were defined, although his dimensions better fit the data accounting for 70% versus 48% for PB. PC believed that effective multicultural counseling involves prominent consideration of the client’s culture and the therapist’s culture, as well as the client’s and the therapist’s experiences of both cultures. There may be several reasons for the differences between these individuals’ constructions of multicultural counseling competence.
For instance, perhaps PA is at the early stage of his training in multicultural counseling. This may account for the relative simplicity of his conceptual configuration of multicultural counseling competence. PB and PC had more complex structures that were somewhat similar. Perhaps the complexity is related to amount of experience these individuals have had with multicultural issues. Another possibility, because PA and PB are both doctoral students who have had similar training experiences, PB’s configuration may be a result of her ethnic minority status. Perhaps this experience lends her another dimension for thinking about these cultural issues. Also, the second dimension for PC accounted for more variance, relative to the first dimension, than it did for PB. It could be that added experience may be related to greater salience given to a second dimension, thus resulting in increased complexity. PC used both dimensions equally (VAF Dimension 1 ⫽ 58%, and VAF Dimension 2 ⫽ 58%), whereas PB used Dimension 1 much more than Dimension 2 (VAF Dimension 1 ⫽ 47%, and VAF Dimension 2 ⫽ 37%). Given that the focus of this example is similar to that of many qualitative studies in its explication of the unique construals of its
COMPLEMENTARY APPROACHES
participants, one way to think about appropriate sample size from which to generate more generalizable conclusions is to use the qualitative concept of saturation or informational redundancy (e.g., Lincoln & Guba, 1985). Data should be collected from more individuals until the researchers are not obtaining any new or different information. In the case of this example, this would mean that more individuals’ data would be collected until no new patterns in responding were found. Thus, the researcher could be confident that the results revealed characterize individuals in general rather than one individual in particular.
Idiothetic Example In this example, we examined the construction of multicultural counseling competence at both the group and individual level. Our goal was to shed light on whether these individuals conceptualized multicultural counseling competence in the tripartite manner proposed by Sue et al. (1998). We adopted a mixed confirmatory– exploratory mode in that we used items from the tripartite model, but we let the data inform us as to the resulting structure. We were also interested in understanding how each individual fit the shared group structure. In particular, we were interested in examining whether length of time in the program and exposure to the multicultural counseling competence model were related to greater adherence to a normative structure for more advanced students, that is, larger fit indices for our sample. Last, we were interested in examining the range of structures that existed within the group. This study is included only as an example of the approach, not as a substantive test of these hypotheses.
147
As noted earlier, pairwise comparison methods and the use of MDS procedures are not only suitable for idiographic studies, but can also be applied in an idiothetic context. Weighted or three-way MDS programs are most appropriate for idiothetic studies. Such studies generally have a set of proximity matrices where each matrix corresponds to an individual who is a member of the larger group. Weighted MDS programs can display the salience of the extracted group dimensions for each separate matrix (Schiffman, Reynolds, & Young, 1981). Thus, in a three-way weighted MDS analysis there is group level information (i.e., the group structure) and individual level information, (i.e., extent of fit of each individual to the group structure, as well as salience weights for each dimension uncovered that show the extent to which the individual uses the group’s dimension).
Method Fifteen masters and doctoral level counseling students (10 masters, 5 doctoral) at a southwestern university were asked to participate in this study. Participation was anonymous and voluntary. It is important to stress that the data obtained from this sample are intended to serve only as an example of the merits of the idiothetic approach. Certainly no generalizations can be made based on a sample of this size. The Multicultural Competence Paired Questionnaire (MCPQ) was developed for this example by selecting 10 stems from the list of operationalized multicultural counseling competencies as delineated by Sue et al. (1992). At least three items representing each of the aspects of the tripartite model of multicultural competence (attitudes and beliefs: 3 items, knowledge: 3 items, and skills: 4 items) were selected. These 10 items (listed in Figure 4) were then combined in all possible pairs, randomly ordered, and
Figure 4. Graphic depiction of the group structure of multicultural competence using three-way multidimensional scaling. Italics represent the names ascribed to the dimensions. Letters represent item symbols. Abbreviations of rational categorization of items appear in parentheses. A/B ⫽ attitude/belief; K ⫽ knowledge; S ⫽ skills.
148
DARCY, LEE, AND TRACEY
the participants were asked to rate their degree of similarity using a 7-point scale ranging from very dissimilar to very similar.
Results Like the idiographic example, the participants’ similarity ratings were constructed into a proximity matrix. In this case, we constructed separate 10 ⫻ 10 proximity matrices for each participant; however, here each matrix had identical items, that is, those generated by the researchers. Instead of analyzing each matrix separately, or aggregating them into one 10 ⫻ 10 composite matrix, we created one three-way matrix (10 item ⫻ 10 item ⫻ 15 participants). Using this three-way data matrix, we applied a weighted, or three-way MDS procedure using Sindscale (Pruzansky, 1975). A three-way MDS approach yields a graphical representation or model of the data similar to that provided in two-way MDS; however, in the three-way situation, this structure is the best fit to the whole group, not just the individual. In addition to the information provided in a two-way MDS (i.e., VAF by the model, VAF by the particular dimensions, and item–stem coordinates on the particular dimensions), three-way MDS also provides several different individual fit indices (overall fit to the data and relative importance of each dimension) for each individual (Arabie et al., 1987; Schiffman et al., 1981; Young & Hammer, 1987). Thus, in the three-way context the researcher has both group level information and individual level information. Our analysis yielded a general structure that optimally fit the entire sample, as well as individual fit indices (how well each person fit the group structure), and dimensional salience weights (how important each dimension was to each individual). We examined one-, two-, and three-dimensional MDS models. The one-dimensional model yielded a VAF of 41%, the twodimensional model had a VAF of 61%, and the three-dimensional model revealed a VAF of 58%. The two-dimensional model was chosen as the most efficient explanation of the data, given fit, interpretability, and parsimony. This model is graphically represented in Figure 4. We used two approaches to interpret the configuration revealed by the MDS analysis, as in the previous idiographic example. First, we visually inspected the configuration, specifically examining the position of the items in relation to each other and the dimensions. Second, we applied a property vector fitting technique using multiple regression analyses. From our visual inspection, we hypothesized that Dimension 1 might represent a focus-on-self versus focus-on-other dimension. Items that most strongly represented this dimension included “Counselor values and respects cultural differences” and “Counselor seeks experiences that improve their work with culturally different clients” on one extreme and “Counselor demonstrates knowledge of the client’s culture” and “Counselor perceives the presenting problem within the context of the client’s culture” on the opposite end. For Dimension 2, we hypothesized that this might represent an active multicultural aspect versus more of a passive attitude adoption. Items that were most representative of this dimension included “Counselor seeks experiences to improve their work with culturally different clients” on one extreme and “Counselor respects and values cultural differences” on the opposite extreme. The main goal of this idiothetic example was to exemplify the assessment of the normative tripartite model for conceptualizing
multicultural counseling competence (Sue et al., 1998). To assess this, we asked an additional eight individuals (three counseling psychology professors with expertise in multicultural issues and five advanced doctoral students) to rate how closely each of the MCPQ items corresponded to the three dimensions underlying the tripartite model. Items were rated from 1 (very unrepresentative) to 7 (very representative). Using the mean ratings, we were then able to conduct a series of regression analyses on the MDS model for the original 15 participants. Results revealed that attitudes and beliefs accounted for 72% of the variance of the items, F(7, 2) ⫽ 9.16, p ⬍ .05, on the two-dimensional model. Knowledge accounted for 54% of the variance of the items, F(7, 2) ⫽ 4.12, p ⫽ .07. Skills was nonsignificant (R2 ⫽ 1%), F(7, 2) ⫽ 0.04, p ⫽ .96. The two significant interpretive vectors are plotted on the MDS model in Figure 4. The length of the lines represents the strength of the relation, and the location represents similarity. So, Dimension 1 appears to be bipolar, representing knowledge at one end and attitudes at the other, instead of the two separate dimensions posited by the tripartite model. This sample viewed attitudes and knowledge as opposites. Dimension 2, although not as important as Dimension 1, did not lend itself to easy interpretation given the tripartite model. To look more closely at each participant’s fit to this group model, we examined the correlation of the extent to which the structure of each participant fit the group configuration (rc ). The distribution of rc correlation values for the participants was fairly narrow, with a mean of .78 and a range of .69 –.84. This indicates that this set of participants was fairly well and equally described by the group structure. Given that there was little variance in the fit of the common structure across individuals, there was little point in examining things that might be related to differences in model fit. This lack of variation here could be attributable to the relative lack of variation in the sample; all were students in training. As noted earlier, three-way MDS also provides information on the importance of each dimension for each individual via the salience weights. These salience weights are indicative of the amount of variance along each dimension for each individual. Small salience weight values indicate that the dimension accounts for little variance and is thus not used by that individual. Large salience weight values indicate large variance and thus extensive usage by the participant. The ratio of these weights (here Dimension 1 and Dimension 2) provides information on the relative extent to which the individual uses each dimension. This ratio of salience weights is referred to as the salience ratio. For illustration purposes, we examined the salience ratio as it varied by ethnicity. The salience ratio had a mean of 1.29 (SD ⫽ 0.14) for the 10 White students, whereas the mean was 1.43 (SD ⫽ 0.01) for the 5 individuals of color. This difference was significant, t(12) ⫽ ⫺2.30, p ⬍ .05, indicating that the individuals of color favored Dimension 1 over Dimension 2 relatively more than did Whites.
Discussion In this idiothetic example, we examined whether individuals conceptualized multicultural counseling competence in the tripartite manner proposed by Sue et al. (1998). These limited results did not support the proposed tripartite model. In fact, only the attitude and beliefs and knowledge dimensions were found to have explanatory power, and these were found to be opposites on a single
COMPLEMENTARY APPROACHES
dimension, not two separate dimensions for the trainees in our limited sample. Ponterotto et al. (2002) found support for these two dimensions, but as separate factors not as opposite ends of a single dimension. Also, the skills component was not found in the perceptions of the small sample of students. Additionally, the second dimension was less clear and accounted for less of the variance. This result could mirror the results found in the idiographic study in Example 1. Greater complexity (as represented by using more dimensions) may be associated with a greater understanding of the multicultural counseling competence construct. Students not too far along in training may not use a second dimension in their thinking about working with clients from varied cultures. There was relatively little variance in the fit of the optimal model to the 15 individuals sampled, demonstrating some support for a normative structure here, but not one that was supportive of the tripartite model. Had there been differences found in the fit of the optimal model across individuals, it would have been appropriate to examine those individuals for whom the model fit and those for whom it did not. In a normative model, the general approach is to assume that the model equally fits all individuals. Only by adopting this idiothetic approach is it possible to determine whether the normative assumption of uniform structure is indeed valid, instead of just assuming it is so. Examining the salience ratio, we found that individuals of color in our sample used Dimension 1 more than Whites in terms of organizing their thinking; the bipolar knowledge–attitudes dimension had greater salience at least relative to Dimension 2. Clearly, caution is needed in terms of generalizing these findings beyond the present sample because of the small sample size and the limited number of items that were chosen to represent the tripartite model. The present example is provided solely to explicate the usefulness of an idiothetic approach in the examination of multicultural counseling competence.
Conclusions The three approaches—normative, idiographic, and idiothetic— have important contributions to make to counseling psychology research in general as well as research focusing on multicultural concerns in particular. Each approach generates information that can be combined to result in a richer, more complex understanding of questions of interest in the field. Normative approaches focus on differences in scores on common dimensions, idiographic approaches focus on unique dimensions for each individual, and idiothetic approaches focus on differences in adherence to common dimensions. With the availability of methods and tools to apply in each of these approaches, it is hoped that researchers will expand their methodological toolbox, thus allowing for a better match between research questions posed and associated designs. We suggest that using idiographic and idiothetic designs, in addition to more traditional normative designs, will aid in our understanding of how individuals think about constructs that are salient to counseling psychology broadly as well as to multicultural psychology research and practice. As can be seen in our idiographic and idiothetic examples of construals of multicultural counseling competence, our participants displayed very different conceptualizations of this construct, conceptualizations that deviated from the normative tripartite model as proposed by Sue et al.
149
(1998). We believe that by adopting an idiographic and idiothetic approach, a more thorough picture of individual variation is yielded. Furthermore, our idiographic example was completely exploratory in that our participants provided their own unique stimulus items. This allowed for the generation of a model specific to the individual. Our idiothetic example, however, drew on existing theory and research and sought to examine aspects of the tripartite model at the group level. This approach allowed us to assess the normative model, yet at the same time gain an understanding of individual levels of variation in terms of the group model. This is a unique benefit of the idiothetic approach that is not possible in more traditional normative designs. A particular advantage of using the paired-comparison rating approach adopted in these examples is that individuals require no conscious awareness of the dimensions that may emerge. This is particularly beneficial when one is seeking to ascertain subtle aspects of thinking that are associated with social desirability and the pressure to present oneself in certain manners such as multicultural counseling competence. Idiographic and idiothetic approaches have potential for exploring a variety of multicultural research questions. For instance, research questions pertaining to multicultural counseling competence may include: How do counselors’ conceptualizations change structurally as they are more exposed to multicultural issues and experiences? How do structural differences in the construal of multicultural counseling competence relate to therapeutic effectiveness? What, if any, are the differences between counselors from minority and majority backgrounds in terms of construal of this construct, and how is this related to the counselor’s role or function? How do clients’ and counselors’ conceptions of multicultural competence compare, and how does this relate to therapeutic effectiveness and client or counselor satisfaction? Applications of idiographic and idiothetic approaches can be used to gain a better understanding of how counselors-in-training perceive factors important to the overall counseling process, for example, expertise (Schvaneveldt et al., 1989), supervision, and presenting problems. Questions related to the development of conceptualization over time and experience can also be tracked using these approaches. An increasing sophistication and complexity would be reflected in differences in the choice and language of the items, as well as perhaps the number of dimensions revealed and how those dimensions are used. Idiographic and idiothetic approaches give rich, in-depth, and highly individualized descriptions; these aspects have a particular advantage for counseling research and practice in which the focus is often on assessing individual’s constructions of self and experience.
References Abreu, J. M. (1999). Conscious and nonconscious African American stereotypes: Impact on first impression and diagnostic ratings by therapists. Journal of Consulting and Clinical Psychology, 67, 387–393. Aldenderfer, M. S., & Blashfield, R. K. (1984). Cluster analysis. Beverly Hills, CA: Sage. Allport, G. W. (1937). Personality: A psychological interpretation. New York: Holt. Arabie, P., Carroll, J. D., & DeSarbo, W. S. (1987). Three-way scaling and clustering. Beverly Hills, CA: Sage. Bradley, R. A. (1984). Paired comparisons: Some basic procedures and
150
DARCY, LEE, AND TRACEY
examples. In P. R. Krishnaiah & P. K. Sen (Eds.), Handbook of statistics (Vol. 4, pp. 299 –326). New York: Elsevier. Buser, S. J. (1989). A counseling practitioner’s primer to the use of multidimensional scaling. Journal of Counseling & Development, 67, 420 – 423. Constantine, M. G., & Ladany, N. (2000). Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization ability. Journal of Counseling Psychology, 47, 155–164. D’Andrea, M., Daniels, J., & Heck, R. (1991). Evaluating the impact of multicultural counseling training. Journal of Counseling & Development, 70, 143–150. DeSteno, D. A., & Salovey, P. (1997). The effects of mood on the structure of the self-concept. Cognition and Emotion, 11, 351–372. Devine, P. G. (1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56, 5–18. Eckes, T. (1994). Explorations in gender cognition: Content and structure of female and male subtypes. Social Cognition, 12, 37– 60. Epstein, S. (1983). A research paradigm for the study of personality and emotions. In M. M. Page (Ed.), Personality: Current theory and research (pp. 91–154). Lincoln: University of Nebraska Press. Falk, J. L. (1956). Issues distinguishing idiographic from nomothetic approaches to personality theory. Psychological Review, 63, 53– 63. Fuertes, J. N., Bartolomew, M., & Nichols, C. M. (2001). Future research directions in the study of counselor multicultural competency. Journal of Multicultural Counseling and Development, 29, 3–12. Hassebrauck, M., & Buhl, T. (1996). Three-dimensional love. Journal of Social Psychology, 136, 121–122. Helms, J. E. (1990). Black and White racial identity: Theory, research, and practice. New York: Greenwood Press. Helms, J. E. (2002). A remedy for Black-White test-score disparity. American Psychologist, 57, 303–305. Kelly, G. A. (1955). The psychology of personal constructs (Vols. 1–2). New York: Norton. Klinger, E. (1995). Effects of motivation and emotion on thought flow and cognition: Assessment and findings. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory (pp. 257–270). Hillsdale, NJ: Erlbaum. Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage. LaFromboise, T. D., Coleman, H. L., & Hernandez, A. (1991). Development and factor structure of the Cross-Cultural Counseling Inventory— Revised. Professional Psychology: Research and Practice, 22, 380 –388. Lamiell, J. T. (1981). Toward an idiothetic psychology of personality. American Psychologist, 36, 276 –289. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage. MacCallum, R. (1988). Multidimensional scaling. In J. R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (2nd ed., pp. 421– 445). New York: Plenum Press. Martignon, L., & Hoffrage, U. (2002). Fast, frugal, and fit: Simple heuristics for paired comparison. Theory and Decision, 52, 29 –71. McFall, R. M., Treat, T. A., & Viken, R. J. (1997). Contributions of cognitive theory to new behavioral treatments. Psychological Science, 8, 174 –176. Morgan, R. L., & Heise, D. (1988). Structure of emotions. Social Psychology Quarterly, 51, 19 –31. Nishisato, S. (1994). Elements of dual scaling: An introduction to practical data analysis. Hillsdale, NJ: Erlbaum. Nosofsky, R. M. (1992). Similarity scaling and cognitive process models. Annual Review of Psychology, 43, 25–53. Ponterotto, J. G. (2002). Qualitative research methods: The fifth force in psychology. The Counseling Psychologist, 30, 394 – 406. Ponterotto, J. G., & Alexander, C. M. (1996). Assessing the multicultural competence of counselors and clinicians. In L. A. Suzuki, P. J. Meller,
& J. G. Ponterotto (Eds.), Handbook of multicultural assessment (pp. 651– 684). San Francisco: Jossey-Bass. Ponterotto, J. G., Gretchen, D., Utsey, S. O., Rieger, B. P., & Austin, R. (2002). A revision of the Multicultural Counseling Awareness Scale. Journal of Multicultural Counseling and Development, 30, 153–180. Ponterotto, J. G., Rieger, B. P., Barrett, A., & Sparks, R. (1994). Assessing multicultural counseling competence: A review of instrumentation. Journal of Counseling & Development, 72, 316 –322. Ponterotto, J. G., Sanchez, C. M., & Magids, D. M. (1990). The Multicultural Counseling Awareness Scale (MCAS): Form B—Revised selfassessment. New York: Fordham University. Pruzansky, S. (1975). How to use SINDSCAL: A computer program for individual differences in multidimensional scaling. Murray Hill, NJ: AT&T Bell Laboratories. Rounds, J. B., Jr., & Zevon, M. A. (1983). Multidimensional scaling research in vocational psychology. Applied Psychological Measurement, 7, 491–510. Rusbult, C. E., Onizuka, R. K., & Lipkus, I. (1993). What do we really want? Mental models of ideal romantic involvement through multidimensional scaling. Journal of Experimental Social Psychology, 29, 493–527. Schiffman, S. S., Reynolds, M. L., & Young, F. W. (1981). Handbook of multidimensional scaling. New York: Academic Press. Schvaneveldt, R. W., Durso, F. T., & Dearholt, D. W. (1989). Network structures in proximity data. In G. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 24, pp. 249 –284). New York: Academic Press. Sodowsky, G. R., Kuo-Jackson, P. Y., Richardson, M. F., & Corey, A. T. (1998). Correlates of self-reported multicultural competencies: Counselor multicultural social desirability, race, social inadequacy, locus of control, racial ideology, and multicultural training. Journal of Counseling Psychology, 45, 256 –264. Sodowsky, G. R., Taffe, R. C., Gutkin, T., & Wise, S. L. (1994). Development of the Multicultural Counseling Inventory: A self-report measure of multicultural competencies. Journal of Counseling Psychology, 41, 137–148. Sue, D. W., Arredondo, P., & McDavis, R. J. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Multicultural Counseling and Development, 20, 64 – 88. Sue, D. W., Bernier, J., Durran, A., Feinberg, L., Pedersen, P., Smith, E., & Vasquez-Nuttal, E. (1982). Position paper: Cross-cultural counseling competencies. The Counseling Psychologist, 10, 45–52. Sue, D. W., Carter, R. T., Casas, J. M., Fouad, N. A., Ivey, A. E., Jensen, J., et al. (1998). Multicultural counseling competencies: Individual and organizational development. Thousand Oaks, CA: Sage. Tracey, T. J. (1991). The structure of control and influence in counseling and psychotherapy: A comparison of several definitions and measures. Journal of Counseling Psychology, 38, 265–278. Tracey, T. J., & Rounds, J. (1993). Evaluating Holland’s and Gati’s vocational interest models: A structural meta-analysis. Psychological Bulletin, 113, 229 –246. Treat, T. A., McFall, R. M., Viken, R. J., Nosofsky, R. M., MacKay, D. B., & Kruschke, J. K. (2002). Assessing clinically relevant perceptual organization with multidimensional scaling techniques. Psychological Assessment, 14, 239 –252. Worthington, R. L., Mobley, M., Franks, R. P., & Tan, J. A. (2000). Multicultural counseling competencies: Verbal content, counselor attributions, and social desirability. Journal of Counseling Psychology, 47, 460 – 468. Young, F. W., & Hammer, R. M. (1987). Multidimensional scaling: History, theory, and applications. London: Erlbaum.
Received October 18, 2002 Revision received July 24, 2003 Accepted August 12, 2003 䡲
