Depression and Suicide Faces

1

23 Depression and Suicide Faces1 Michael Heller & Véronique Haynal

As far as psychiatry is concerned, suicide is the major cause of mortality. Un-fortunately suicide is difficult to prevent, as its prediction is unreliable. Indeed, neither the categories of “people at risk” as defined by studies ranging from sociology to neurophysiology nor the clinical questionnaires allow to predict satisfactorily an attempt. Therefore, clinicians remain somewhat at a loss when faced with a first attempt or even with a reattempt. Yet clinical practice has shown that some experienced therapists, probably following their intuition, are able to predict suicidal risks with a smaller error margin than their colleagues (Motto, 1991). What do these therapists rely upon? What semiology elements do they perceive even though they are unable to explain them? We propose the hypothesis that most of the cues that subtend this intuition are provided by the suicidal patients’ nonverbal behavior. To our knowledge, studies pertaining to the systematic analysis of suicidal patients’ nonverbal behavior are scarce, if nonexistent. In 1976, Ringel described a presuicidal syndrome that has since become well known. He observed in suicidal patients a cognitive and affective restriction that he termed “constriction effect”: patients tend to restrict their adaptive capacity until death appears more and more strongly as the only solution. Ringel also showed that the mode of communication in suicidal patients followed a similar evolution. More recently, Ekman (1985) presented the case study of “Mary.” Mary managed to obtain the approbation to leave the hospital by being quite convincing in her lying about her suicide intentions during the interview. This interview was filmed. Later on she admitted the deception. Ekman showed the interview to young psychiatrists and psychologists; most of them were deceived by Mary’s behavior—even many of the more experienced clinicians were deceived. In order to evidence the lie, Ekman scanned the film for hundreds of hours, going over it again and again, inspecting each gesture and expression in slow motion to uncover any possible clues to deceit. In a moment’s pause before replying to her doctor’s question about her plans for the future, we saw in slow motion a fleeting facial expression of despair, so quick that we had missed seeing it the first few times we examined the film. Once we had the idea that concealed feelings might be evident in these very brief micro expressions, we searched and found many more, typically covered in an instant by a smile. We also found a micro gesture. When telling the doctor how well she was handling her problems Mary sometimes showed a fragment of a shrug—not the whole thing, just a part of it. She would shrug with just one hand, rotating it a bit. Or her hands would be quiet but there would be a momentary lift of one shoulder. (p. 17) For more than 30 years Ekman and Friesen have studied facial behavior and the emotions expressed on the face. Inspired by Darwin and Duchenne, they have applied the methods of ethology to the study of facial mimics and their emotional signification, as well as on the means used to hide these (cf. where Mary dissimulates her true feelings with a smile, her real emotions only appeared as micro-expressions). Their data confirmed the observation made by Darwin that in order to express emotion, the face uses some elements that are common to all human beings and even to some animal species such as monkeys, cats, and dogs (Ekman, 1980; Grammer, Schiefenhövel, Schleidt, Lorenz, & Eibl-Eibesfeldt, 1988; Preuschoft, 1995). They show that the relationship between these elements and some emotions is relatively independent from behavioral variations owing to different cultural environments. For instance, joy is expressed by the same lengthening of the lips and lifting of the cheeks with a compression of the eyelids in populations as diversified as those in North Africa, New Guinea, or North America. To carry out their research, Ekman and Friesen have developed a system for the coding of facial action (Facial Action Coding System, or FACS; 1978), which has proved to be an invaluable tool since. The study of facial expressions appeared quite appropriate for our research and the FACS an adequate tool because of the reliability of the coding system (Ekman, & Friesen 1978, 1980; Ellgring, 1989; Krause, SteimerKrause, & Hufnagel, 1992) and its relative independence from cultural influences. Indeed, the population in Geneva (Switzerland), and therefore the patients we observed, is quite culturally varied.

1

Paul Ekman and Erika L. Rosenberg (Ed.) What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS), 179-180. 398-413. Oxford. Oxford University Press, 1998.

2

Studies Two studies are presented here. The first concerns suicide and the second depression. Both were carried out at the Laboratory of Affect & Communication (LAC).2

Study on Suicide The population was composed of 17 subjects: 10 women and 7 men aged from 19 to 62. They were treated in the psychiatric ward of the Geneva psychiatric institution. All suffered from severe depression as defined by the DSM III and presented almost no schizoid or psychotic symptoms. The patients had attempted to commit suicide within the past two weeks. Each patients was filmed during an interview with a psychiatrist. All patients gave their written consents for being filmed and only their faces were filmed. During the interview the psychiatrist asked 5 questions from the Present State Examination. We analyzed the patient’s facial behavior during the following question: “Do you still wish to attempt to end your life?” The sampling for the analysis was as follows: from the beginning of the silence that preceded the question, during the question, the following silence, then the first 10 seconds of the patient’s answer (mean length: 16.65 seconds). One year after the filmed interview, 5 patients had further reattempted. Thus we defined two groups:one composed of 5 depressive suicide with further reattempts (FR), the other 12 depressive with no further reattempts (NR).

Study on Depression The population was composed of 9 subjects: 5 women and 4 men aged from 29 to 59 years old. They were treated in the same institution as the patients of the previous study. All suffered from severe depression as defined by the DSM III and presented no schizoid or psychotic symptoms. The patients were filmed during 5 sessions of semistructured interviews with their psychiatrists, which occurred every two weeks. The interviews addressed the evocation of events that had aroused emotions in the patients (joy, sadness, anger, and neutral.) The sadness topic contained a question on suicide intentions. We analyzed in each interview, one sample per emotional topic (mean length: 19.33 seconds). After an interview, the patients filled out Beck’s Self-Evaluation Questionnaire for depression and the psychiatrists assessed the patients’ state using Hamilton’s Questionnaire for the evaluation of depression. A comparison of the scores obtained after the first and the last sessions shows that after 10 weeks the state of 3 of the nonsuicidal depressed patients had greatly improved, 1 patient showed moderate improvement, and 5 patients still presented major depression symptoms. In the sadness topic we coded the following question “Have you recently wished to die or to attempt suicide?” We analyzed the facial behavior of the patients from the end of the question until the beginning of the next question or during 20 seconds.

Coding and Analysis The patients’ facial behavior was analyzed using the Facial Action Coding System (FACS). The onset, apex, and offset of the action, as well as its laterality (S if it is symmetrical, L for Left, and R for Right) were coded sequentially. Thus an action of the lips may be coded S12D where S indicates its symmetry, 12 the pull of the lip corner, and D its intensity on a scale ranging from A (minimum) to E (maximum). After the data collection and analysis for both studies, we obtained a general repertoire of the emotional and facial behavior for the two populations observed. In order to test a common assumption that suicidal risk increases as a function of depression severity, it seemed interesting to compare the results of these two populations of severely depressed patients: suicidal depressives and nonsuicidal depressives. We isolated the results for the situations common to both studies—that is to say, the suicidal intentions as collected during the first interview.

Results Three main results are presented here: the first concerns the repertoire for both populations—that is, all the expressions used by each group; the second treats the asymmetrical expressions; and the third measures the duration of the movements in the upper face.

2

This work was directed by Michel Heller, with the participation of Pat Claus, Véronique Haynal, and Christine Lessko, within the Clinque de psychiatrie II, under the responsibility of Pr. André Haynal. Part of the data we published in France in Les Cahiers Psychiatriques Genevois, 16 (1994).

3

The Repertoire The repertoire of a given group is composed of all behaviors observed within this group. The 26 depressed patients (17 suicidal and 9 nonsuicidal) were analyzed over a period of approximately 20 seconds. They displayed 60 distinctly different facial expressions, as coded with an EMFACS procedure. Each patient used an average of 2.3 expressions never displayed by another patient. The behavior of depressed patients is finally quite diversified (see table 23.1). Only 10 expressions are used by at least one suicidal depressive patient and one nonsuicidal depressive patient. The precision of the coding (i.e., intensity) is not the only reason for the high diversity we observe: most expressions are really different when viewed on the tapes. Most of the facial expressions are used by only one person and only a few expressions are used by several patients of one or both populations. Many of the expressions displayed by suicidal patients were not observed in nonsuicidal patients and vice versa. TABLE 23.1. Facial repertoire Expression

Dep

Sui

Expression

Dep

Sui

L02B

0

2

S16B + S25B

0

1

L04B L14B R02B + L04B R10B R12A R14B R18B S01B + S02B + S15A S02B S04B + S07B S04B + S07B + S15A S06B S06B + S26B S07B S10A S10B S10B + S15B S10D S10E S12A + S25B S12B S12D S12E S14A S14B + S17D S14B + S17D + S24E S14B + S23B S15A S15A + S25B

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

1 2 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

S17B + S24B 0 S18B + S25B 0 S22B + S26B 0 S24A 0 S01B + S02B 6 S04B 1 S12A 1 S14B 1 S15B 1 S17C 2 S18B 1 S23B 3 S23B + S25B 1 S24B 3 L01B + L02B + S17B 1 S01B 1 S01B + S02B + S25B 1 S05C 1 S06B + S12C 1 S10A + S26B 1 S12A + S17C 1 S14B + S17B 1 S14B + S24D 1 S14B + S26B 1 S15C 1 S17A 1 S17B 1 S17D 1 S17E 1

1 1 1 2 3 1 1 4 2 1 1 1 1 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Expression: EMFACS facial combinations observed. Dep: number of nonsuicidal depressive patients using this combination (n = 17). Sui: number of suicidal depressive patients using this combination (n = 9).

Emotional Expressions Following the hypothesis of the Ekman and Friesen dictionary, the repertoire described in table 23.1 yields the emotional repertoire shown in table 23.2: Table 23.2 shows that a negative or contempt or disgust hypothesis is attributed to 6 suicidal depressive patients (and 3 out of 6 reattempters) and to only 1 nonsuicidal depressive. Within the repertoire of our patients, as shown in table 23.3, we coded 9 asymmetrical expressions: one composed of one symmetrical and two asymmetrical AUs, and 8 composed of only 1 or 2 asymmetrical AUs, without symmetrical AUs—for example, only the left eye brow is lifted in an otherwise impassive face or the left corner of the lips freezes into a small nearly permanent smile. The totality of these 8 asymmetrical expressions were present in 7 of the depressive suicidal patients. Among these, 4 reattempted to commit suicide. Only 1 of these asymmetrical expressions, more complex and mixed with

4

symmetrical AUs, was observed in the population of nonsuicidal depressive patients. Among these 8 expressions observed, 4 belong to the emotional expressions associated by Ekman and Friesen to “contempt.” They are displayed by 5 of the 17 suicidal depressive patients, 3 of these having reattempted suicide. No expression related to contempt, be it asymmetrical or not, was observed in our population of depressive nonsuicidal patients during this topic.

Upper Face Activity Suicidal depressive patients show a reduced activity in the upper face compared with nonsuicidal depressed patients. Figure 23.1 shows the duration (in percent) during which each upper face unit (around the eyes) was used.3 The patients are represented on the X-axis divided into two groups: the first 9 patients are the nonsuicidal depressives and ordered according to the severity of their depression after 10 weeks: the first 3 improved within 10 weeks (D1 to D3), the last 5 did not (D5 to D9). TABLE 23.2. Number of patients who used each emotional category ha-f ha-u sa an neg di co np neg + co + di Dep

1

1

2

4

0

1

0

7

1

Sui Snr Sr

0 0 0

4 2 2

0 0 0

3 2 1

1 1 0

2 2 0

5 2 3

15 10 5

6 3 3

ha-f: felt happiness, ha-u: unfelt happiness, sa: sad, an: anger, neg: unspecific negative feelings, di: disgust, co: contempt; np: no prediction, neg + co + di: either of these emotions. Dep: number of nonsuicidal depressive patients (n = 17). Sui: number of suicidal depressive patients (n = 9). Snr: non reattempter suicidal depressive patients (n = 12), Sr: reattempter suicidal depressive patients (n = 5). TABLE 23.3. Asymmetric expressions EMO AU Combination

TOT

Dep

Sui

No prediction

L01B + L02B + 17B

1

1

0

No prediction

L02B

2

0

2

No prediction

L04B

1

0

1

Contempt

L14B

2

0

2

No prediction

R02B + L04B

1

0

1

Contempt

R10B

1

0

1

Contempt

R12A

1

0

1

Contempt

R14B

1

0

1

No prediction

R18B

1

0

1

EMO: emotional hypothesis proposed by Ekman and Friesen’s dictionary. TOT: number of subjects using this combination. Dep: number of nonsuicidal depressive patients (n = 17). Sui: number of suicidal depressive patients (n = 9).

The next 17 patients are suicidal depressive patients: the first 12 patients (S10 to S21) did not reattempt to commit suicide within the following year, the next 4 patients (S1 to S4) reattempted to commit suicide and the last one (S5) succeeded in committing suicide four weeks after the interview. For instance, Patient D7 is a depressive nonsuicidal patient who is very depressed at the first interview (Hamilton score 36) and will not be better after 10 weeks (Hamilton 35); he moves his upper face during 87% of the sample duration. Patient D2 is depressed during the first interview (Hamilton score 29) but will feel much better after 10 weeks (Hamilton score 4); he activates his upper face during 23% of the coded sample. The Y-axis indicates the percentage of observation time during which each upperface Action Unit was activated (each unit was analyzed separately). Upperface Action Units are: lifting the eye brows (AUs 1 + 2), brow lowerer (AU 4), upper lid raiser (AU 5), cheek raiser and lid compressor (AU 6) (included only when not coded with AU12), lid tightener (AU 7). We did not consider lid droop (AU 41), slit (AU 42), eyes closed (AU 43), squint (AU 44), blink (AU 45), or wink (AU 46).

3 Means for Dep. and Dep-Sui.: 22.33; 2.88. SD for D.N.S. and D.S: 770.87; 32.26. Mann-Whitney W test: U = 158, p < = .01 (two-tailed).

5

Figure 23.1 Duration of upper face activity for nonsuicide and suicide depressive patients (%).

A more detailed analysis of the data evidences that it is mainly the duration of the mobilization of the brow raise (1 + 2) that is responsible for the difference, but this combination alone does not really yield a systematic difference. Looking at this graph, we can see that the two populations differ as far as the mean and standard deviation: (1) suicidal depressive patients show relatively low inter-individual variations with a small mean: their duration ratio ranges between 0 and 19%; and (2) nonsuicidal depressive patients display an important interindividual variation with a high mean as their duration ratio ranges from 0 to 87%. Unlike our expectations, our data show no relationship between the scores obtained on the Hamilton scales and the duration ratios for the nonsuicidal depressive patients. Nor was there a relationship for the suicidal patients between the fact that they belonged or not to the reattempt group. That is to say, the reduced activity observed in suicidal patients is not related to a more severe depression than for the nonsuicidal depressive patients.

Discussion Quantitatively, the repertoire of each population is quite diversified. Indeed, each patient uses at least one Action Unit displayed by no other patient. This diversity alone explains the little overlapping observed between the two repertoires. But behavioral imagination is not the sole variable that must be taken into account. It should be noted though that these results are rather complex because facial behavior also involves the duration and complexity of each movement. A sign typical of suicide seems difficult to evidence. Evidently, as noted by Frey, Hirsbrunner, Florin, Daw, and Crawford, (1983), the mere counting of the movements is not sufficient to measure activity. In our sample, the difference in activity between our two populations involves mobility, duration, and diversity of repertoire. For instance, if a face shows AU 4 (brow lowerer) only, during the whole time window, its meaning will be different from AU 4 appearing 3 short times or combined with AU 1. Our third result shows a clear difference in the activity in the upper face measured as a ratio of duration of the Action Units. The difference has two aspects: (1) there is a significant difference between the two averages; and (2) it is also a clear difference of variance between the two groups. This pattern is frequently observed in nonverbal communication as for most of the signs the nonactivation of the muscles is always possible. This has also been observed for postural behavior (Heller, 1991). The decreased activity in the upper face is mainly associated to a limited duration of eye brow mobilization in suicidal patients. How can we understand such phenomena? According to Ekman and Friesen’s “dictionary of emotions” the muscles surrounding the eyes are involved in the expression of several emotions (fear, surprise, joy, sadness), but the eyebrows are often mobilized with the purpose of capturing or maintaining the interlocutor’s attention. In other words, there are two meanings (at least) of the eye brow raise (AUs 1+2). One is the conversational sign, culturally expressed to sustain or get the partner’s attention or to encourage him to talk. The other, mostly when AU 1+2 is combined with other Action Units, may be an emotional expression (Grammer et al., 1988). Suicide patients filmed several days after their attempt showed a certain inhibition to communicate while they were talking about their suicide intentions. Some of these patients make little attempt to initiate or maintain contact with their psychiatrist. They seem to have constructed a blank mask. Nonsuicidal depressive patients do not display this

6

kind of inhibition. With them, the interaction looks easier. For us, then, both inhibitions are linked: the emotional as well as the conversational. As Krause and Lütolf (1988) remarked, it is interesting to note what is missing in the facial activity. Indeed, although variability of expressions is quite high among suicides it remains nevertheless restricted to certain areas of the face (lower face, the mouth . . . ). Also, the behavior of a given group may exclude some types of expressions. Besides movement occurrences, duration, and diversity, we also have more qualitative data. Our second result, asymmetric expressions being only observed among suicides, particularly in reattempter patients, may give us clues about meaning. These patients, who displayed few relevant emotions (no sadness nor disgust for reattempters) seem to show mostly contempt. It may be interpreted as contempt for the interview situation, but also probably as contempt of oneself and maybe of one’s own life. We were struck by this observation, as it corroborates the hypothesis found in clinical research such as in some psychoanalytical schools (Klein, 1933). Our results are in accordance with the observations reported by Ringel (1976) in which he defines a “suicidal syndrome” as a relatively differentiated and independent clinical entity. Shneidman (1993) also has developed this direction of analysis. The scope of this pilot study is, of course, restricted by the limited number of patients observed and by the relatively small duration of the samples analyzed. Yet we believe that our first observations are encouraged enough to be presented. We are now engaged in a larger investigation on nonverbal communication in this field, filming the physician as well as the patient. In studying the interaction we wish, among other things, to improve our data on a qualitative level.

Conclusion The results of these studies show that a population of nonsuicidal depressive patients behave differently from a population of depressive suicidal patients as far as the expression of emotions on the face is concerned. Analysis reveals that suicidal depressive patients do not suffer from more severe depression than the other patients, as the variable “depression severity” is not associated to the difference between the two populations. Therefore, we believe that suicidal risk is related to factors other than depression. We also believe that such factors may be evidenced in the patient’s nonverbal behavior. The inhibition revealed by the analysis of suicidal patients’ expressions suggests that the problem could be related to impulsiveness and to violence. In a larger research project that we are currently carrying out, analyzing the doctor’s and the patient’s facial behavior, we hope to shed further light on the processes involved in suicidal risk.

References American Psychiatric Association. (1980). Diagnostic and Statistical Manual of Mental Disorders, (3rd ed). Washington, DC: Author. Darwin, C. (1872/1965). The expression of the emotions in man and in animals. Chicago: University of Chicago Press. Ekman, P. (1980). L’expression des émotions. La Recherche 11, 1408–1415. Ekman, P. (1985). Telling lies. New York: Norton. Ekman, P. (1989). The argument and evidence about universals in facial expressions of emotions. In H. Wagner & A. Manstead (Eds.), Handbook of psychophysiology: The biological psychology of the emotions and social processes (pp. 143–164). New York: Wiley. Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3/4), 169–200. Ekman, P., & Friesen, W. V. (1969). Nonverbal leakage and clues to deception. Psychiatry, 32, 88–105. Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System. Palo Alto, CA: Consulting Psychologists Press. Ellgring, H. (1989). Nonverbal communication in depression. Cambridge: Cambridge University Press. Frey, S., Hirsbrunner, H.-P., Florin, A., Daw, W., & Crawford, R. (1983). A unified approach to the investigation of nonverbal and verbal behavior in communication research. In W. Doise & S. Moscovici (Eds.), Current issues in European social psychology (Vol. 1, pp. 143–198). Cambridge: Cambridge University Press. Fridlund, A. J. (1991). Sociality of solitary smiling: Potentiation by an implicit audience. Journal of Personality and Social Psychology, 60(2), 229–241. Grammer, K. Schiefenhövel, W., Schleidt, M., Lorenz, B., Eibl-Eibesfeldt, I. (1988). Patterns on the face: The eyebrow flash in crosscultural comparison. Ethology, 77, 279–299. Haynal, A. (1993). Psychoanalysis and the sciences: Epistemology and history. London: Karnac. Heller, M. (1991). Postural dynamics and social status. Doctoral dissertation, Psychology Faculty, University of Duisburg, Germany. Heller, M. (1992a). Corps et évaluation de psychothérapies: Un rêve à réaliser. Psychothérapies, 2, 111–121.

7

Heller, M. (1992b). Les émotions: Un parcours littéraire. In J. Besson (Ed.), Manuel d’enseignement de l’Ecole Française d’Analyse Psycho-Organique. Toulouse: Ecole Française d’Analyse Psycho-Organique. Heller, M. (1993). Unconscious communication. In B. Maul (Ed.), Bodypsychotherapy or the art of contact (pp. 155–179). Berlin: Verlag Bernhard Maul. Klein, M. (1940). Mourning and its relation to manic-depressive states. International Journal of Psychoanalysis, 21, 125–153. Klein, M. (1993). Contribution to the psychoanalysis of manic-depressive states. International Journal of Psychoanalysis, 16, 145–174. Krause, R., & Lütolf, P. (1988). Facial indicators of transference processes within psychoanalytic treatment. In Dahl, H. Kächele, & H. Thomä (Eds.), Psychoanalytic process research strategies, (pp. 257–273). Heidelberg: Springer. Krause, R., Steimer, E., Sänger-Alt, C., & Wagner, G. (1989). Facial expression of schizophrenic patients and their interaction partners. Psychiatry, 52, 1–11. Krause, R., Steimer-Krause, E., & Hufnagel, H. (1992). Expression and experience of affects in paranoid schizophrenia. European Review of Applied Psychology, 42, 131–138. Ladame, F. (1981). Les tentatives de suicide des adolescents. Paris: Masson. Ladame, F. (1993). Les paradoxes du suicide. Adolescence, 11, 125–136. Levenson, R. W. (1992). Autonomic nervous system differences among emotions. Psychological Science, 3, 23–27. Motto, J. A. (1991). An integrated approach to estimating suicide risk. Suicide & Life Threatening Behavior, 21, 74–89. Preuschoft, S. (1995). ‘Laughter’ and ‘smiling’ in macaques. Doctoral dissertation of the Faculteit Biologie Rijksuniversiteit Utrecht. Ringel, E. (1976). The presuicidal syndrome. Suicide and Life Threatening Behavior, 6, 131–149. Scherer, K. R., Summerfield, W. B., Wallbott, H. G. (1983). Cross-national research on antecedents and components of emotion: A progress report. Social Science Information, 22, 355–385. Shneidman, E. (1993). Suicide as psychache. Northvale, NJ: Jason Aronson.

8

AFTERWORD: Perspectives for Studies of Psychopathology and Psychotherapy Michael Heller & Véronique Haynal

Beyond the specific analysis described in the preceding article, we share a dream. We hope that fairly soon a psychotherapist may come to see us with a film of his interaction with a patient. After analyzing a sample of this film, we could provide our colleague with a reliable diagnosis and an evaluation of the therapeutic relationship, based on their nonverbal communication. This dream is based on our experience, which has convinced us that what a therapist may know of a patient cannot be acquired experimentally, and that what we can teach a therapist on how he interacts with a patient cannot be acquired through experience and clinical knowledge alone. By combining thus experimental and clinical approaches, we hope to gain new insights into human psychology and interpersonal interaction (Haynal, 1993; Heller, 1992). What we need to transform this dream into reality is not beyond the scope of what can be done today: (1) funds and support, of course; (2) setting up of a database that includes all the data colleagues and ourselves have collected through samples of behavior filmed in a given test situation; (3) a reliable coding procedure—several already exist, such as FACS for facial behavior and Siegfried Frey’s Time Bernese System for gestures; and (4) good software allowing us speedy and relevant comparisons between the newly coded data and our database, producing outputs we can show to a psychotherapist. Although several laboratories are working on such programs, this is where we still need to invest the biggest effort. We will focus the following discussion on some of the problems that need to be solved before such programs can be written.

How Much Information Is Processed While Communicating? When we analyze a film, we first code behavior. Just using FACS implies using a system that can distinguish on a face approximately 40 units with an average of 4 intensities: 0: non active 1: occurs slightly 2: occurs with a medium intensity 3: occurs at maximum intensity With such a procedure we must deal with 160 (4*40) possible items when we analyze a photographed face and 320 for two faces. This computation supposes that knowing that a unit is not active may be just as important as knowing what is active. Assuming that the scale found by movie makers is relevant, we suppose that precise coding requires the analysis of 25 frames per second. Thus, one minute of coded psychotherapy involves 480,000 items, and one hour 28,800,000 (Heller, 1993). The information we have then informs us on visible muscular facial behavior. It does not account for gaze or skin texture, which are also important sources of information. Furthermore, our own limits and our due respect for reliability prevents us from noting down some subtle facial expressions, which we nevertheless perceive as having a powerful impact. This rapid survey of our database provides us with a glimpse of how complex current human behavior really is, and of how much information we will eventually need to deal with. It also leads to two related questions: (1) How can research deal with such a mass of information collected on a single interaction? (2) How does a person’s brain manage such a huge amount of perceived information, sometimes in only a few seconds? Well, human beings manage, and research is slowly managing. Finding ways of dealing with such a huge amount of data may lead to interesting thoughts on which algorithms the brain may use. In our laboratory, as in many others, we write down the initial state of the face, and then only what changes and when it changes. We thus have much less than 28,800,000 items written on our hard disk for one hour of psychotherapy. We then have programs. Programs don’t take much space, but they can use the database to generate huge tables. These tables provide lists of the data that are implicitly coded: for example, the state of each Action Unit at any moment or the dynamics of facial configuration from the point of view of time (differentiating chronic smiles from fast ones, or smiles with a fast onset and offset from smiles with a slow offset, etc.). Other tables provide us with statistical analysis of our data. Such tables can be very large. For our nonsuicidal depression study, a listing of significant differences for a given variable (e.g., duration of activity) often occupied more than 20 megabytes on the hard

9

disk. When you try to find interesting results in these huge files really, you become aware of the enormous amount of computation the brain must generate before it can go beyond reflex reactions. The advantage of a data-programs-tables strategy is that the basic information does not require much space. Furthermore, programs can be reused with several databases. We only generate all the tables simultaneously when we are trying to understand our data. Later, we may erase all the large tables, and generate more focused tables that will take much less space. When the study ends the tables are erased from our hard disk with the security that they can be generated once again when needed—provided we remember how to use our programs. Managing all this and understanding what programs and films provide the researcher with still takes a few years, instead of a few seconds, like the brain. We mostly miss not having a theory capable of guiding the creation of such programs.

Emotional and Current Behavior As analyzing other people’s behavior is a daily task, we may have a tendency to think nonverbal communication should be fairly easy to understand. But every time we think we have found a tool allowing us to describe it better, we end up with a way of discovering a new land of complexity. Afirst key allowing us to penetrate the vast world of facial behavior was forged by Tomkins and Ekman, who used Darwin’s model to describe certain well-known emotional expressions. These expressions probably cover only a few percent of the total repertoire used by a person during his daily life and maybe are a bit too quickly described as “the” repertoire of emotional interaction (Heller, 1992b). It nevertheless is one of the few angles that allow us to grasp some behaviors in a coherent way. The scarcity of “basic emotional expressions” becomes evident as soon as we try to collect films in which these are numerous. To obtain such films researchers need to expose subjects to highly provocative stimuli (gun shots, horror or surgery films, etc.) (Fridlund, 1991; see also Grammer et al., 1988). The facial behaviors of subjects exposed to such events can reasonably be thought of as being emotional. It is interesting to note that in the realm of body psychotherapy, people who think in terms of a Darwinian emotional model also recommend extremely strong emotional stimuli to operationalize their methods. For example, Alexander Lowen will elicit basic emotions of the same type as those described by Ekman, by asking patients to pound, or shout, to use stressful posture or deep massage, to breath extremely fast and deeply, and so on. To study people suffering from depression, or people who come to see us just after a suicide attempt, we cannot confront them with such overwhelming stimulation. On the contrary, our interviews are designed to provide some support for our patient’s attempt to preserve a behavior as compatible as possible with his or her self-esteem. These patients are asked to sign a consent form, which states that the experimental situation may be useful to their own process as well as to a better understanding of similar patients. Research on psychopathology and nonverbal behavior is thus often carried out through films of semistructured interviews. It is in such a setting that our own studies have been carried out. These situations are fairly formal. We tend to make such interviews as short and unintrusive as possible. Patients nevertheless often express a certain gratitude in being able to talk about various aspects of their difficult life, and may show disappointment when they realize that we stop the interview after 10 to 20 minutes. When they are also included in our films, therapists tend to be the source of our difficulties. For the nonsuicidal depression study, the doctors of our clinic had shown some enthusiasm when we told them that we would like to study the interaction between psychiatrists and depressive patients. Nevertheless, patients suddenly received all sorts of diagnoses, but seldom that of major depression! Getting films with patients only was fairly easy; getting films with patients and therapists is a project in itself. Although some very emotional contents may be conveyed in such interviews, other mechanisms—related to social (cultural?) rules of interaction—are also explicitly activated. Our material thus confronts us with the variety of behaviors that is usually displayed in everyday life. There are two types of problems:

Control Groups One is the issue of how to manage control groups. Just putting “normal” people in the same situation does not necessarily generate an adequate control group because our interviews were designed to analyze a certain problem (suicide, depression), and would have a completely artificial relevance for a “normal” population. The data shown in this paper raise this issue in a particularly sharp way: one could surmise that part of the difference observed between suicidal depressive patients and nonsuicidal depressive patients may be due to the difference of relevance of our suicide topic for the two populations. For suicides, mentioning to a doctor that one would still want to commit suicide may be critical, as it is part of a psychiatrist’s duty to react in certain coercive ways to such a statement. On the other hand, nonsuicidal depressive patients may take the question as yet another moment to

10

communicate on deep personal questions. Krause tried to avoid this problem by choosing a situation relevant for any citizen, with or without psychopathology (Krause & Lütolf, 1988).

Classifying Any Type of Behavior Observed on Films As our situations do not induce a certain type of behavior, we do not know what sort of behavior our subjects produce. Some publications have proposed certain subgroups of behavior (emotional, emblematic, cognitive, cultural, etc.), but the relevance of these components, and knowing how to “classify” a coded behavior, still remains largely a question of personal choice, although some research is attempting to address this issue. Analyzing Current Behavior We shall illustrate our arguments by listing some concrete unanswered questions we have currently found in our attempts to analyze data.

Activity Measures We have tried to write programs that could perform a preliminary analysis of our data automatically. One obvious variable for such programs is that of facial activity, which seems to be related to 5 dimensions: (1) number of times events occur; (2) number of changes; (3) the length of time a unit is activated; (4) mean intensities of units; and (5) variety of repertoire. This last variable allows us to differentiate moving ten times the same unit, from moving ten times ten different units. These two types of activity are identical from the point of view of occurrences, but not from the point of view of repertoire or of the perceiver. Although such variables are fairly easy to define, and generally thought of as relevant, it is more difficult to find a way to combine them so as to produce a measure of activity. It is, for example, difficult to combine duration and occurrence in an activity measure. If a person activates a unit during a whole sample, the unit occurs once. A duration measure will thus suggest that the person has been very active, while an occurrence measure will suggest that the person has not been very active.

Time and Emotional Expressions: Signal/Regulation Several FACS studies have concentrated on photographs or on a short sample per patient. If in this context a “typical” emotional expression is coded, it can be described as the expression of an inner feeling. In our nonsuicidal depression study we saw each patient five times in ten weeks; and for each interview we coded 4 emotional topics. In other words we had 20 samples per patient. Each sample, being fairly short, yielded few expressions. By looking at one sample we may take certain facial configurations to be discrete emotional expressions. But what sometimes happened was that we observed the same “emotional signal” in 18 or 20 of the samples coded for a single patient. It became difficult to go on using the notion of emotional expression, if we define an expression as a specific semiological event closely related to a content and a given moment (a few minutes at the most). If these 18 similar expressions observed at 18 different moments are not an expression, what are they? Any decision on the subject really depends on the researcher’s imagination rather than on theory. One answer is that these expressions are like the tip of an iceberg— something like random recurrent expressions of a continuous mood. Such an explanation makes good sense for a study on depression. But other plausible explanations also exist, without even being contradictory. For example, what some Americans call a “pacifier smile” can be understood as part of a recurrent strategy to diminish another person’s potential aggressivity. To summarize, as soon as coded samples cover more than a minute, we are faced with a polarity within which we need to situate an expression: (1) at one extreme we have facial configurations that can reasonably be related to a psycho-physiological state (Levenson, 1992); (2) at the other extreme, we have a series of facial configurations that have to be treated as a series of events probably related to regulation mechanisms rather than expressive ones. Thus adding time to our experimental plan clearly adds dimensions in our data and raises new concrete questions.

Applying the Dictionary to Any Expressions? Recently, looking at a tennis match on television, we observed how often players displayed AU10 (upper lip raise). One could easily accept the idea that they were expressing some sort of disgust or contempt for either/or their opponent and themselves. But we could also associate this activity with something like pain and effort. Scherer et al. (1983) followed a similar trend of thought when he suggested that the elements of an emotional

11

expression may have independent functions that can be constructed around an affect to express it. For example, disgust or contempt could be used to convey how painful the other’s behavior may be to us.

Slips of the Face We use “slip of the face” in association with the Freudian slip of the tongue. This notion was suggested to us by Ekman and Friesen (1969), and Krause & Lütolf (1988). In one of our visits at Saarbrçken, we saw a patient displaying AUs 6 + 10 + 12 (cheek raise, upper lip raise, and lip corner pull). The perceiver, at first glance, mostly perceives a clear broad smile (AU 6 + AU 12). But coding the psychotherapist of this patient revealed that the psychotherapist did not react the same way when the smile contained or did not contain AU 10. AU 10 is often associated with a gorilla or a dog displaying the upper teeth—an obviously hostile expression. The interviews of this patient analyzed by Krause, as well as the personal reactions reported by the therapist, supported the idea that the therapist had reacted to an apparently hostile expression (AU 10) hidden by a smile (AU 6 + AU 12). In Krause’s example, the relevance of a slip of the face is supported by the protagonist’s reaction. For the studies discussed in this article, we filmed only the patient’s facial behavior. Attributing an emotional hypothesis to a very short facial dynamic was more tricky. In some cases they really did look like a “slip of the face” when looked at at normal speed. But others seem to convey an emotional content only when they are maintained artificially as a still image. Looking at them at normal speed, we mostly have the impression of a quasimeaningless transition between two mimics. Finding rules allowing us to associate meaning with a facial dynamic is a general preoccupation in the field (e.g., Ekman, 1992). But looking at current behavior in detail clearly shows that the algorithms regulating such associations, even when only considering affect regulation systems, are probably sufficiently complex to allow misunderstanding as well as understanding.

References Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6, 169–200. Ekman, P., & Friesen, W. V. (1969). Nonverbal leakage and clues to deception. Psychiatry, 32, 88–105. Fridlund, A. J. (1991). Sociality and solitary smiling: potentiation by an implicit audience. Journal of Personality and Social Psychology, 60, 229–241. Grammer, K., Schiefenhövel, W., Schleidt, M., Lorenz, B., & Eibl-Eibesfeldt, I. (1988). Patterns of the face: the eyebrow flash in cross-cultural comparison. Ethology, 77, 279–299. Haynal, A. (1993). Psychoanalysis and the sciences: Epistemology and history. London: Karnac. Heller, M. (1992a). Corps et évaluation de psychothérapies: un reve a réaliser. Psychothérapies, 2, 111–121. Heller, M. (1992b). Les émotions: un parcours littéraire. In J. Bresson (Ed.). Manuel d’enseignement de l’Ecole Française d’Analyse Psycho-Organique (Vol. 2, pp. 125–175). Toulouse: Ecole Française d’Analyse Psycho-Organique. Heller, M. (1993). Unconscious communication. In B. Maul (Ed.). Bodypsychotherapy or the art of contact (pp. 155–179). Berlin: Verlag. Krause, R., & Lütolf, P. (1988). Facial indicators of transference processes within psychoanalytic treatment. In H. Dahl, H. Kächele, & H. Thoma (Eds.). Psychoanalytic process research strategies (pp. 257–273). Heidelberg: Springer. Levenson, R. W. (1992). Autonomic nervous system differences among emotions. Psychological Science, 3, 23–27. Scherer, K. R., Summerfield, W. B., & Wallbott, H. G. (1983). Cross-national research on antecedent and components of emotion: A progress report. Social Science information, 22, 355–385.