Social science research and experimentation in Australian criminal proceedings: prejudicial pre-trial publicity and psychological research

Social Science Research and Experimentation in Australian Criminal Proceedings: Prejudicial Pre-trial Publicity and Psychological Research Ian Freckelton BA (Hons) LLB (Syd), PhD (Griff), Dip Th M (ANH) Barrister, Melbourne, Australia; Adjunct Professor o f Law and Psychological Medicine, Honorary Associate Professor of Forensic Medicine, Monash University; Adjunct Professor, School of Law and Legal Studies, La Trobe University, Melbourne, Australia and

Marilyn McMahon BA (Hons) LLB (Melb), GDLS MPsych (Monash), MAPS Lecturer, School of Law and Legal Studies, La Trobe University, Melbourne, Australia Correspondence to: [email protected] or [email protected]

Using the decision of Barr J in Attorney-General (NSW) v John Fairfax Publications [1999] NSWSC 318, the authors analyse the need for external validity and relevance in social science evidence adduced in the courts. They argue in favour of the rigour employed within Barr J’s judgment and contend that a constructive legacy of the decision should be a greater sensitisation on the part of researchers to the kinds of factors that can distance experimental scenarios from curial contexts to a point where generalisations from the former become strained and even spurious. A number of these considerations have the potential to be addressed to a significant extent by improved methodologies. However, the authors warn against an excess of purist fervour in demanding complete comparability of scenarios, lest the fruits of social science be denied to the courts and decision-making be adversely affected by the absence of expert insights falling short of complete replication of conditions between experiments and forensic reality.

Introduction At what point does research in a given area reach a level of scientific rigour, forensic relevance and validity that warrants weight being attributed to social science research supporting a conclusion related to a legal issue in criminal proceedings? This issue has previously arisen in Australia and other jurisdictions in relation to eyewitness testimony1 but

1

S Fulero, “The Role of Behavioral Research in the Free

February 2002

has received most detailed consideration in relation to the effects of pre-trial publicity. In a remarkable decision of the New South Wales Supreme Court in 1999, 2 Barr J was called upon to consider detailed and carefully constructed evidence based on social

Press/Fair Trial Controversy” (1987) 11 Law and Human Behavior 259. See, too, R v Ratten [1971] VR 87; Re Ratten [1974] VR 201. 2 Attorney-General (NSW) v John Fairfax Publications [1999] NSWSC 318.

347

FRECKELTON and McMAHON

science literature and an applied study conducted specifically for the purposes of the trial. Two North American psychologists, including an internationally renowned expert on juries and pretrial publicity (who also designed the applied study), provided expert testimony. The trial judge, Barr J, provided an incisive and sophisticated analysis of the psychological evidence and research. Although accepting much of the research as uncontroversial, Barr J found the applied study to be unpersuasive due to the many methodological limitations that he identified. Significantly, Barr J established a high benchmark that must be reached before experimental psychological evidence can be persuasively tendered to the courts . Exploring the legal issues considered by the New South Wales Supreme Court, the nature of the psychological evidence that was presented and the form and results of the applied research illuminates the manner in which contemporary Australian courts evaluate and utilise psychological research in particular and social science generally. The decision may well have significant repercussions also for the probative value attributed to epidemiology evidence. Matters that have long been the subject of comment and controversy within psychology – such as the utility to legal decision-makers of psychological research on pre-trial publicity, and the methodological limitations of this research – were transformed into judicial commentary in this case. Attorney-General (NSW) v John Fairfax Publications constitutes an intriguing, instructive case study in the social science-law area in Australia for it identifies two key matters of concern for those conducting forensic research. Barr J set a high threshold that must be reached before courts will attach probative value to research literature and, in particular, empirical psychological research said to be relevant to a legal issue; indeed, a pessimistic interpretation of his judgment suggests that, due to the unique setting of the courtroom, and the limitations of positivist scientific method, the threshold is unattainable. Additionally, and more prosaically, he drew attention to the significance of external validity when considering the utility of psychological research in legal proceedings. In the context of this contemporary Australian case, Barr J reiterated concerns that had been expressed by commentators more than a decade and a half previously when considering what use to make of psychological research on pre-trial publicity and the 348

extent to which empirical findings generalise to actual settings.3

Adverse pre -trial publicity: The legal framework The law in relation to the criminality of adverse pre-trial publicity has many features in common throughout Canada, England, New Zealand and Australia. In each of these countries, a key element in the attempt to protect accused persons’ rights to a fair trial historically has been the existence of the criminal offence of engaging in conduct which has a real and appreciable tendency to prejudice the prospects of a fair trial. 4 The remedies for the accused, though, in each of these countries are limited. By contrast with the situation in the United States,5 where the most popular method for determining pre-trial prejudice is conducting an extended voire dire with potential jurors,6 this option does not exist in these common law jurisdictions.7 What remain as options are judicial admonition, continuance, change of venue8 and, in principle – in an extreme case, where the prejudice is irremediable – staying the trial permanently on the basis of its being an abuse of process. Absent the option of conducting a detailed voire dire with those called for jury service but not yet empanelled, significant emphasis is directed in common law countries outside the United States to investigating

3

J S Carroll, N Kerr, J Alfini, F M Weaver, R MacCoun and V Feldman, “Free Press and Fair Trial: The Role of Behavioral Research” (1986) 10 (3) Law and Human Behavior 187; S S Diamond, “Illuminations and Shadows from Jury Simulations” (1997) 21 Law and Human Behavior 561. 4 Australian Law Reform Commission, Contempt, Report No 35 (AGPS, Canberra, 1985); M Chesterman, “Criminal Trial Juries in Australia: From Penal Colonies to a Federal Democracy” (1999) 62 Law and Contemporary Problems 69. 5 M Chesterman, “OJ and the Dingo: How Media Publicity Relating to Criminal Cases Tried by Jury is Dealt With in Australia and America” (1997) 45 American Journal of Comparative Law 109. 6 See L Wrightman, M T Nietzel and W H Fortune, Psychology and the Legal System (4th ed, Brooks/Cole Publishing Co, Pacific Grove, Ca, 1998). 7 See N Vidmar, “Generic Prejudice and the Presumption of Guilt in Sex Abuse Trials” (1997) 21 (1) Law and Human Behavior 5; R M Corbin, A K Gill and R S Jolliffe, Trial by Survey: Survey Evidence and the Law (Carswell, Ontario, 2000) for brief summaries of the process that operates in Canada. 8 See N Vidmar and J W T Judson, “The Use of Social Science Data in a Change of Venue Application: A Case Study” (1981) 59 Canadian Bar Review 76; Chesterman, op cit n 4.

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

the general effects of adverse pre-trial publicity on jurors both in the context of prosecutions for sub judice contempt of court and in support of applications for continuance, change of venue9 and permanent stays of criminal trials.10 In Australia’s most important case on the subject, Attorney-General (NSW) v John Fairfax Publications, the utility of an applied, empirical study, which specifically investigated the issues of prejudicial pre-trial publicity, was rejected. The quality of the research conducted and the carefully reasoned decision of the court raise fundamental and difficult issues for the forensic role of an area of social science, in particular applied psychology, which has been argued, at least by some psychologists,11 to be able to make a constructive contribution to courtroom analyses. In particular, the decision embodies many of the methodological criticisms that have been raised within psychology about research on pre-trial publicity and juries over the past two decades.12 The decision also demonstrates a curial antipathy towards research findings that are deemed to be neither realistic nor representative of actual cases, an orientation also demonstrated by some United States legal commentaries.13 The significance of the Australian decision is that it transmutes those methodological concerns into a legal yardstick for assessing the utility of applied research on pre-trial publicity. Insofar as the benchmark is set high, future applied research will have to address these concerns if such research is to be seriously considered by Australian and perhaps other courts. Immediately, then, the case poses confronting questions for social science disciplines, especially for forensic psychology. This article examines the utility of applied research in the context of juror recognition of prejudicial cues about accused persons who have

9

Vidmar and Judson, op cit n 8. See Corbin, Gill and Jolliffe, op cit n 7. 11 See, eg, Fulero, op cit n 1. 12 See, eg, W Weiten and S Diamond, “A Critical Review of the Jury Simulation Paradigm” (1979) 3 Law and Human Behavior 71; R M Bray and N L Kerr, “Use of the Simulation Method in the Study of Jury Behavior: So me Methodological Considerations” (1979) 3 Law and Human Behavior 107; Carroll et al, op cit n 3; G P Kramer, N L Kerr and J S Carroll, “Pretrial Publicity, Judicial Remedies and Jury Bias” (1990) 14 Law and Human Behavior 409. 13 See Diamond, op cit n 3. 10

February 2002

been the subject of adverse pre-trial publicity. It considers the threshold that must be reached before courts are likely to attach probative value to social science research. It explores how the criterion of generalisability, a standard by which social scientists have long evaluated their work, has become a key criterion by which the utility of such research, when it is adduced in the forensic context in Australia, is measured. It looks to the dimensions of external validity and probes their significance as a tool for evaluating psychological evidence offered to the courts. It analyses the context and ramifications of the decision in Attorney-General (NSW) v John Fairfax Publications. As previously noted, this case provides an important and, for Australia, a rare example of the use of psychological testimony in the context of evaluation of the impact of pre-trial publicity. While this case was by no means the first superior court decision in Australia to evaluate the utility of empirical research conducted specifically for a particular trial,14 most previous analyses of expert evidence concerning the interpretation of empirical research have been in the civil contexts of trademark and passing off cases or in applications for a permanent stay of proceedings in criminal trials or for a change of venue.15 By contrast, in Attorney-General (NSW) v John Fairfax Publications the expert evidence related to whether the publication of newspaper articles and photographs about a defendant had created a real likelihood that a fair trial would not be possible. We argue that, in relation to the threshold that must be reached before social scientists, such as psychologists, are justified in citing the general research literature and applied studies that have been conducted relevant to a particular legal issue, the approach of Barr J establishes a very high benchmark and risks excluding from the courts information which, while it has limitations, has the potential to assist decision-makers in evaluating the likelihood of publications causing prejudice in potential jurors.

14

See generally I Freckelton, and H Selby, Expert Evidence: Law, Practice, Procedure and Advocacy (Lawbook Co., Sydney, 2002). 15 Ibid.

349

FRECKELTON and McMAHON

The John Fairfax publications case The publisher of a major Sydney newspaper (John Fairfax & Sons) was tried for contempt in 1999. 16 The charge related to the publication of material in a daily newspaper on three separate pages of the same edition. The material was argued to be in contempt of court because it had a tendency to prejudice the prospects of a Vietnamese man, Duong Van Ia, obtaining a fair trial. At the time of the publication of the newspaper article, Duong had been charged with serious drug trafficking offences. The trial of Duong was set down to take place only a few months after the publicity and the fact of his upcoming trial was mentioned in the newspaper article, along with a variety of highly prejudicial information about Duong and his family. For a publication to be in contempt of court in Australia it must be found beyond reasonable doubt to have a real and practical tendency to interfere with the administration of justice.17 The tendency to embarrass or prejudice pending proceedings must be clear, or real and definite; there must be a substantial risk of serious interference.18 The statements in the articles in the Sydney Morning Herald of 27 October 1997 and the photographs of Duong were alleged by the New South Wales Attorney-General to be in contempt of court on four bases: • the articles were likely to influence jurors to believe that the accused had previously committed offences involving the supply of heroin, or had been knowingly involved in its supply, and that he continued to carry on business as a large-scale heroin distributor; • the articles were likely to influence jurors to believe that the accused was a person of bad character; • the articles were likely to influence jurors to determine the guilt of the accused otherwise than solely on the basis of the evidence admitted at trial; and

16

Attorney-General (NSW) v John Fairfax Publications [1999] NSWSC 318. 17 Australian Law Reform Commission, op cit n 4, p 138; Attorney-General v 2UE (unreported, NSWCA, 16 Oct 1997); John Fairfax & Sons v McRae (1955) 93 CLR 352. 18 Hinch v Attorney-General (Vic) (1987) 164 CLR 1; Victoria v Australian Building Construction Employees’ and Builders Labourers’ Federation (1982) 152 CLR 25; Attorney-General (NSW) v X [2000] NSWCA 199.

350

•

the publication of the photographs made it more likely that jurors who had read the articles would identify him at the trial as the subject of the articles. The publisher denied that it had prejudiced Duong’s prospects of obtaining a fair trial. The defence relied substantially upon the reports and evidence of two United States social psychologists, Dr Neil Williams, at the time resident at the University of New South Wales, and Professor Neil Vidmar, an academic in the School of Law and the Department of Psychology at Duke University in North Carolina.

The publicity the subject of the charge The articles that were the subject of the Attorney General’s criminal charges were so unusual that they may well have constituted a deliberate attempt by the publisher to provoke a test case on the law of sub judice contempt of court. They contained a series of components that made them, on the surface at least, unusually damaging for the accused. Three separate photographs of Duong were published. The first photograph was on the front page of the paper, was very large and was in colour. On a subsequent page the photograph was repeated, smaller and in black and white. Another picture was inserted on another page. The name of Duong was repeated on multiple occasions in the course of the articles. The colour photograph of Duong published on the front page of the newspaper was placed under the heading “The top heroin distributor”. He was personalised by the inclusion of biographical information, including details about his background in Vietnam, his gambling exploits and references to his wife and his lover (the latter being a nightclub singer and dancer who was referred to by her stage name, “Miss Kim”). “Miss Kim” was stated to be living with their child. In addition, Duong was referred to by the nickname of “Uncle Six”. His work “cover” (as a barbequer of pork in restaurants) was described. Duong’s extensive, highly suspicious wealth and flamboyant gambling habits (gambling more than $A20 million at a local casino in the previous two years) were repeatedly mentioned, including his having headed a list of suspected criminals whom the local Police Commissioner had banned from a prominent casino. The article referred to Duong’s involvement in the criminal importation of an illegal

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

drug and stated that he was to face trafficking charges in the District Court some months later in relation to the same drug. Finally, references to Duong were contextualised within broader consideration of a group of notorious criminal underworld figures and drug importers, as well as within his own family unit, members of which were identified as having been recently found guilty of drug offences.

The defence Defence counsel stressed that a prosecution for contempt had to be proved beyond reasonable doubt and contended that there had to be a reasonable doubt about whether any juror would recall the articles of 27 September 1997 by the time of Duong Van Ia’s trial some five months later and whether they would connect them with Duong in view of the facts that: • at the time of publication the trial was a considerable distance away; • the articles did not canvass the subject matter of the accused’s trial or assert that he was guilty of the charges. The fact that he was facing the charges was referred to only in the final paragraph of one of the articles; • the accused was not well known at the time of the publication; • the name of the accused was phonetically and visually unfamiliar to a non-Vietnamese reader; • the publicity was not repeated; and • it was reasonable to suppose that jurors would adhere to their oath and obey the directions of a trial judge to decide the case only on the evidence admitted at the trial. In support of the defence, psychological evidence was adduced which argued for a very low level of recognition and recall on the part of jurors about articles and photographs in a newspaper, when the articles and photographs related to a little-known person of non-Caucasian extraction, and when there was a significant gap in time between the potentially prejudicial publication and the occasion and context when it could work its prejudice.

the nature of the proceedings said to have been prejudiced (including whether they are proceedings before a jury), and the time which elapsed between publication and trial).19 It was clear that the fact that the charges against the accused were serious and that they were likely to be heard before a jury were relevant factors in determining the potential for the articles to prejudice the administration of justice. The court found that the five-month gap between publication and trial was also of consequence. So, too, was the kind of material which had been published, a particular concern having been identified in many contempt cases when a media outlet revealed not just bad character information about the accused but referred to matters amounting to prior and germane criminal conduct.20 The reasoning of the court on this matter is consistent with psychological research that has demonstrated that this type of information is particularly prejudicial if it is before jurors having to decide on the guilt or innocence of a person accused of comparable conduct.21 The test adopted by Barr J to decide what constitutes a real and practical tendency to interfere with the administration of justice (that is, to prejudice the prospects that an accused person will obtain a fair trial) was that, if any of the jurors remembered any of the statements about the accused, Duong would not receive a fair trial. He held that the damage would be irremediable – no direction to the jury by a trial judge could cure such prejudice. He found that multiple aspects of the articles and photographs were likely to make their way into evidence. While he accepted that psychological evidence indicated that readers tend only to scan newspaper articles and pay more attention to earlier rather than later parts of articles, he found the articles to be distinguishable from this pattern as they formed part of a series. Additionally, they were referred to again in a television program and in a Vietnamese newspaper, such further

19

[1999] NSWSC 318 at [5]. Attorney-General (NSW) v Willesee [1980] 2 NSWLR 143; Maxwell v DPP [1935] AC 309; Davis v Baillie [1946] VLR 486. 21 See I Freckelton, Prejudicial Publicity and Sub Judice Contempt, ALRC Contempt RP 4 (ALRC, Sydney, 1986); A Padawer-Singer, A Singer and R Singer, “Legal and SocialPsychological Research in the Effects of Pretrial Publicity on Juries, Numerical Makeup of Juries, Nonunaminous Verdict Requirements” (1997) 3 Law and Psychology Review 71. 20

The decision The Supreme Court of New South Wales had regard to a number of factors when determining whether a publication amounts to a contempt, including the nature and extent of the publication, February 2002

351

FRECKELTON and McMAHON

references being found by Barr J as “likely to increase the probability of memory”. 22 Barr J held that as a matter of practical reality it had been established beyond reasonable doubt on behalf of the Attorney-General that the articles of 27 October 1997 had a tendency to interfere with the due course of justice at Mr Duong’s trial, notwithstanding that it was not due to take place for a further five months.23 For technical reasons, however, Barr J found John Fairfax Publications not guilty of the charge of contempt. This was on the basis that he was satisfied that the prejudicial material was published merely in the course of a discussion about public affairs and the denunciation of public abuses as permitted by Jordan CJ in Ex Parte Bread Manufacturers; Re Truth and Sportsman Ltd.24 This has been the subject of criticism elsewhere 25 and Barr J’s decision was the subject of appeal to the Court of Appeal.26

The expert evidence adduced on behalf of the defendant The principal importance of the New South Wales Supreme Court decision in Attorney-General (NSW) v John Fairfax Publications lies in the analysis of the utility of the psychological evidence placed before the court. Much of the evidence relating to memory for names and faces was not contentious. It is significant that in AttorneyGeneral (NSW) v John Fairfax Publications no psychological evidence was adduced on behalf of the prosecution. In those jurisdictions in Australia where the defence is obliged to provide the prosecution with copies of expert reports upon which it proposes to rely at trial, it may well be in the future that the prosecution will itself call rebuttal social science evidence. However, in AttorneyGeneral (NSW) v John Fairfax Publications the attack mounted by the prosecution upon the expert

22

[1999] NSWSC 318 at [119]. Ibid at [121]. 24 (1937) 37 SR (NSW) 242 at 249. 25 I Freckelton, “Prejudicial Publicity and Sub Judice Contempt” (2000) 7 (1) Psychiatry, Psychology and Law 215; J Giddings, “Would Christopher Sk ase Receive a Fair Trial?” (2000) 24 Criminal Law Journal 281. 26 Attorney-General (NSW) v X [2000] NSWCA 199; see I Freckelton, “Evolving Attitudes Toward Sub Judice Contempt” in K Sen (ed), Essays in Honour of Michael Chesterman (Lawbook Co., Sydney, 2002, forthcoming). 23

352

evidence was conducted exclusively by crossexamination of the witnesses themselves. The evidence from the two psychologists that was crucial was of two types: 1. evidence based on general principles derived from accumulated psychological research on memory. Barr J accepted that cross-racial identification of faces was less accurate than intra-racial identification and that names in unfamiliar languages are remembered less frequently.27 Additionally, Barr J accepted much of the evidence that addressed the manner in which people read newspaper articles and the subsequent impact of the reading on memory (for example, scanning rather then detailed reading, earlier material generally having more impact than later material within an article). 2. information derived from an applied, quasiexperimental study specifically conducted for the trial that directly investigated the matters which were the subject of controversy and which underpinned the charges laid against the defendant. Barr J’s analysis of the study methodology and also of the reasoning of the psychologists for the defence in Attorney-General (NSW) v John Fairfax Publications was sophisticated. It highlights the difficulties confronting those who seek to introduce evidence of applied psychological research. His critique of the evidence of the two expert psychologists did not go to the admissibility of the evidence. It went to the weight that should be accorded to it. The crux of his decision is that, in the context of evaluating the likelihood of jurors recalling pre-trial publicity, empirical studies generally will not be helpful because of the differences between experimental conditions and the real-life circumstances of a criminal trial. The distinctions identified by Barr J between the research strategies of the psychologist and the lengthy and fluid process of the criminal trial are of real consequence for determining when media treatment of matters impacting upon a criminal trial is likely to be recalled by jurors.

General research on memory Although not the main focus of this article, brief reference will be made to the general evidence put

27

[1999] NSWSC 318 at [25].

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

forward by the two psychologists as it contextualises the subsequent consideration of the applied empirical study. The evidence of Dr Williams was based exclusively on literature relating to memory and the effects of adverse pre-trial publicity. The combined evidence of both experts sought to establish a number of matters, including the impact of the long delay between publication and trial, the selective and limited nature of the manner in which newspapers are read, and the impact of inter-racial differences in the recall of salient features (for example, names and faces). The essential conclusion reached by both experts was that it was highly unlikely that the articles in question, and the photographs within them, would be remembered by jurors approximately five months after publication or that they would be likely to influence jurors.28 Whilst accepting some of these matters, Barr J characterised this general evidence as being flawed in a number of respects: 1. There was an inappropriate assumption that all members of the jury trying Duong Van Ia would be non-Asian, a significant proportion of the Sydney population from which the juror pool would be drawn being Asian; 2. The evidence failed to take account of the many characteristics attributed to the accused in the articles that might have made the reader more likely to remember what he or she had read; 3. Some of the expert reasoning was especially problematic in that the approach adopted had been whether any person randomly selected from the community would remember enough of the articles to connect them with the accused at his trial. However, the court held that the real question was whether any of the jurors, interacting with other jury members, would have any relevant memory. Thus, appropriate empirical investigation should have acknowledged that jury members function as apart of a group and that memory may be synergistically activated within this group setting. 4. The social science literature on which the experts relied did not adequately account for the distinctive environment of the courtroom.

28

Citing A Marks, “How Quickly They Forget, If They Ever Knew” (1993) 72 Psychological Reports 1337.

February 2002

Trials may last for a considerable time, present jurors with repeated opportunities for recall and recognition of salient matters, and, as previously noted, involve jury decision-making that is a group (not individual) task.29 Accordingly, Barr J found that the generalised psychology literature had relatively little to contribute to evaluations of impact. Barr J was sufficiently troubled by the general evidence provided by one expert for the defence that he concluded that his “knowledge and expertise may not be entirely reliable for present purposes and … the literature on memory and forgetting, upon which his opinion was largely based, may not be apposite”.30 Having discounted the evidence on general principles, the attention of the court was then directed to a detailed evaluation of the expert testimony relating to the applied study that was conducted to investigate the precise matters that were being considered at the trial: the impact of adverse pre-trial publicity concerning Duong on the memory of potential jurors.

The applied study Professor Vidmar’s evidence, concerning the applied study which he had designed to investigate the ability of members of the public to remember the details of the publication that was the subject of the actual contempt prosecution, was intended to complement the more general evidence about the operation of memory.31 Although the trial judge repeatedly referred to the study as a “survey”, the applied study employed a quasi-experimental design with two stages of testing: an initial phase (during which participants were exposed to the prejudicial material); and subsequent critical testing conducted by telephone. Corbin, Gill and Jolliffe 32 have referred to the study as a “simulation experiment”, but this description is dubious due to the focused nature of the experiment

29

[1999] NSWSC 318 at [44]. Ibid at [33]. The approach adopted in this regard was similar to that followed by Gleeson CJ in F v The Queen (1995) 83 A Crim R 502 in devaluing the utility of counterintuitive or mythdispelling evidence. For a critique of this approach see Freckelton and Selby, op cit n 14. 31 See Table 1 below at 367 for a summary of the study. 32 Corbin, Gill and Jolliffe, op cit n 7. See, too, B Bornstein, “The Ecological Validity of Jury Simulations: Is the Jury Still Out?” (1999) 23 (1) Law and Human Behavior 75. 30

353

FRECKELTON and McMAHON

and the limited efforts to create a context that reconstructed key elements of either the original setting of exposure to the publicity or the recall environment of the courtroom. Lacking these features, it appears that the study more closely approximates a quasi-experimental design than a simulation experiment or survey. Initial telephone contact was made with 5,800 persons in the relevant geographic area; those who were identified as readers of the Sydney Morning Herald (the newspaper in which the original articles had been published) and who satisfied the stratification criteria of the study (age and gender) were invited to attend at the offices of a Sydney marketing research organisation. Potential participants were informed that they would be paid for their participation. An initial sample of 188 was obtained.33 Although no information was provided directly to the court about the participation rate, it was obviously very low. The substantial difference between the number of approaches and subsequent actual sample size raises concerns about the representativeness of the sample that was ultimately obtained. Additionally, the difficulty in obtaining research participants raises concern about the potential for a selection-specificity effect (that is, that effects validly demonstrated in the study only applied for the unique population from which the experimental and control groups were jointly selected). The 188 research participants were informed that they were assisting with a study on newspaper readership. They were divided into four groups (the allocation was not random: the absence of key materials in the early stage of the research dictated group allocation at this point). Participants were placed in a room (either individually, or in groups to a maximum of three members) and given materials to read, with the instruction that they should read them as they would ordinarily read a newspaper. Exactly how they did so, and the extent to which the exercise for some was a communal one is unclear. Group A participants were given a complete black and white photocopy edition of the Sydney Morning Herald newspaper of 27 October 1997.

Group B participants were given the same material as Group A participants plus copies of the editions of 28 and 29 October 1997. Group C participants were given a copy of the edition of 3 November 1997. Group D participants were given the same material as Group C participants plus copies of the editions of 4 and 5 November 1997. The additional material provided to participants in Groups B, C and D included articles referring to the local drug trade, money laundering and government policy in relation to drug importation, as well as the treatment of illegal drug users. It did not contain any references to Duong. Participants were then asked to complete a written questionnaire, were paid a sum of money and told that researchers might telephone them at a later date to ask additional questions. Responses to the questionnaire revealed that all participants were currently enrolled on the electoral roll and 93 per cent were eligible to serve as jurors.34 About 14 days later, researchers telephoned each participant early on a weeknight evening and asked them a series of questions designed to evaluate the extent to which each person recalled and/or recognised the names Duong Van Ia and “Uncle Six”, as well as any connection between the names and other names which had appeared in the articles they had been asked to read. Additionally, participants were asked a number of questions in relation to two bogus names (that is, names that did not appear in the relevant newspapers).35

Recall tasks Specifically, participants were asked: • a number of introductory questions about topics which had been dealt with in the particular newspaper editions; • whether they could recall reading or hearing any news stories during the past two years that involved the illegal drug trade, such as heroin or cocaine in Australia. A total of 97 per cent answered in the affirmative. • whether they could provide the names of persons associated with the stories and explain how they were associated with the stories. Eighteen

34 33

The initial sample size was described as 188 although data were only available for 187 participants.

354

Exhibit LE7 accompanying the affidavit of Lucille Economos. See Table 1, below at 367, for a summary of the main stages of the applied study. 35

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

per cent of participants in Group A said that they could do so, compared with 40 per cent, 32 per cent and 28 per cent respectively in Groups B, C and D. None of the participants mentioned the name of Duong Van Ia but two mentioned the name “Uncle Six” and one mentioned Duncan Lam (the other alleged drug boss featured in the photographs on p 1 of the 27 October edition and the articles on p 1). Other names that were recalled by participants included persons referred to in the course of articles on 27 October. • whether they could give the names of any persons identified as “drug bosses”. A total of 22 per cent, 30 per cent, 11 per cent and 6 per cent respectively of the four groups indicated that they could. One participant said “Uncle something”, another said “someone Lam” and several referred to “Asian names” but could not be more specific. A number of participants referred to the names of well-known Sydney criminal identities. Finally, participants were asked the real name of the man called “Uncle Six”. There was no significant increase in the rate of recognition of the three names over the rate that resulted from the earlier question about recognition of the names, only the one person identifying Duong Van Ia as Uncle Six.

Recognition tasks Participants were then asked to listen to 20 names that were read in random order. Some were well known. Two were bogus Vietnamese names and were included in order to identify prejudice that might exist among the participants toward anyone with a name that sounded Vietnamese. • Fifteen per cent of Group A participants claimed to recognise the name “Duong Van Ia”. Of the other groups, 10, 5 and 6 per cent purported respectively to recognise the name. However, the reasons for their doing so were disuniform, some thinking he was concerned with a murder but most identifying the name as connected with the drug trade. The rates of recognition of Duong Van Ia in the two groups that had actually read the 27 October issue were noteworthy: 15 per cent and 10 per cent. • Recognition of the name “Uncle Six” varied from 37 per cent of Group A to 20, 11 and 6 per

cent respectively in the other groups, most identifying him as being involved in drug trafficking. Again, there was a considerable difference between Groups A and B and Groups C and D. The significant false rates of recognition amongst groups C and D – who had read material that did not mention “Uncle Six” – suggested that some of the purported recognition rate in Groups A and B may have been erroneous. • Participants who purported to recognise the name “Duncan Lam” comprised 24, 10, 11 and 17 per cent respectively in the four groups, most saying he was a drug boss or involved in the illegal drug trade. • In relation to the bogus Vietnamese names employed in the study, “recognition” of the first bogus name varied from 21 per cent to 6 per cent in the four groups,36 most claiming that the name was connected with drug trafficking. The second bogus name was “recognised” by between 33 and 17 per cent of the members of the four groups. In the next phase of testing, participants were then asked whether during the previous two years they had read or heard anything about a Vietnamese immigrant called Lan Tran Cao (a fictitious name), a Vietnamese immigrant called Duong Van Ia or someone named Uncle Six; and whether there was any connection between Lan Tran Cao or Duong Van Ia or Uncle Six and any other person. Surprisingly little attention was directed to the interpretation and significance of information pertaining to the bogus names. Professor Vidmar argued that the recognition rate for these names suggested general beliefs existed about the involvement of Asian (especially Vietnamese) persons with the drug trade. Hence, he argued that the purported recognition of Duong Van Ia might be related to these generic beliefs rather than a simple reflection of memory for the newspaper material. However, no information was provided about the frequency of recognition of Duong Van Ia’s name or the name “Uncle Six” when participants who endorsed the bogus items were removed from the analysis, although this strategy has sometimes been

36

February 2002

[1990] NSWSC 318 at [67]. See also Table 1 below at 367.

355

FRECKELTON and McMAHON

used in other studies that have utilised bogus names.37 Attrition varied from 19 to 32 per cent among groups with Groups C and D having the highest rates (32 and 28 per cent respectively). The findings from the applied study led Professor Vidmar to express the opinion that recall and recognition of the accused were very low at what he called an absolute level and by comparison with names which had appeared more extensively in the news media. Professor Vidmar commented in his report: “[I]n comparison to other cases involving heinous or otherwise notorious crimes, the likelihood that there was a strong community response as a direct consequence of the two articles is low. This is because in comparison to other cases the articles were one-time events and were not substantially reinforced by other media sources and the story was not extended over a substantial time period.” 38 Moreover, he stated that this result was comparable with other research that he had conducted in other jurisdictions. Professor Vidmar explicitly addressed several matters pertaining to the validity of the applied study conducted for the trial. First, participants knew that they were participating in a study relating to their reading of a newspaper. Thus they may have been sensitised to give more attention to their reading than they normally would have done. In short, the artificiality of the experimental setting, in which participants were aware that they were participating in an experiment, itself may have generated “demand characteristics” for participants. These demand characteristics of the experimental situation may have influenced participants’ reactions insofar as during the interviews, the participants may have become sensitised to the focus on drug crimes and drug bosses, leading them to guess at the purpose of the study and to provide answers consistent with their understanding of the purpose of the study. Secondly, in the research study the period of time between reading the material and interview was, on average, 14 days whereas in the actual case the time

period between publication of the newspaper article and the trial was five months. Professor Vidmar argued that these matters did not undermine the utility of the study but actually strengthened it, for these two factors increased the likelihood of retention of information by participants. Hence results obtained in the study provided a maximal measure of the effects of the newspaper article on the memory, attitudes and beliefs of readers exposed to the article. Whilst acknowledging the worth of some aspects of the study, ultimately Barr J rejected its utility – largely on the basis that the study was not sufficiently generalisable to the situation of the courtroom, the trial and ht e circumstances of the defendant. It is interesting to note that little curial attention was directed to the differences between experimental groups, undoubtedly due to the more general problems identified in relation to the external validity of the study. This focus also ensured that other issues, such as the function and interpretation of the bogus names – a contentious matter in research on pre-trial publicity39 – were not explored. Analysis of the reasoning of Barr J on this issue suggests that he did not merely emphasise factors concerning the external validity of applied psychological research on the effect of adverse pretrial publicity but in fact set a benchmark so high that little current forensic psychological research could meet the standard. Indeed, it could be argued that he required not simply comparability of conditions but replication: an impossible standard.

Evaluating the survey: A failure to replicate? “The principal difficulty I have in giving effect to the survey results … lies in the difference between the conditions of the survey and the conditions that would prevail at trial.” 40 Barr J gave considerable attention to the design and methodology of the applied research that was conducted for the purposes of the litigation. In essence, much of Barr J’s analysis can be summarised as close consideration of the external validity of the empirical study (although he did not use that term): he found the study lacked

37

G Moran and B L Cutler, “Bogus Publicity Items and the Contingency Between Awareness and Media-induced Trial Prejudice” (1997) 21 Law and Human Behavior 339. 38 Annexure B to Vidmar affidavit, 16 February 1999 at [25].

356

39

Moran and Cutler, op cit n 37. Attorney-General (NSW) v John Fairfax Publications [1999] NSWSC 318 at [88] per Barr J. 40

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

persuasiveness because it could not readily be generalised to the situation of the actual trial. The framework of the judicial analysis in Attorney-General (NSW) v John Fairfax Publications is familiar to all scientists for it can be reduced to the central concept of validity. “Internal validity” basically refers to the degree to which the effects observed in an experiment can be attributed to an experimental treatment, commonly noted as the level of “control” that a researcher exerts over the research design. “External validity” refers to the generalisability of an experimental study – the degree to which the study can be generalised to other populations, settings, treatments and measurement variables,41 informally referred to as the “realism” of the study. External validity is frequently demonstrated by identifying similarities between the original and experimental individuals, contexts and behaviours.42 Two features of external validity are salient here. First, it is multifaceted – hence there are different, discrete aspects of external validity. Accordingly, there is no single standard against which to judge the realism of an experiment in applied psychological research.43 Secondly, external validity may be assessed on a continuum. Hence, it is appropriate to refer to experiments that are high or low in external validity rather than to describe external validity as an “all or nothing” phenomenon. Concern with the external validity of research in the psychology-law area, especially in the area of jury research, has existed for three decades.44 It is a truism in the social sciences that good research is characterised by high internal and external validity.45 It is also a truism that control and realism are frequently in conflict, and that increasing

41

See D T Campbell and J C Stanley, Experimental and Quasiexperimental Designs for Research (Rand McNally, Chicago, 1966). 42 See Bray and Kerr, op cit n 12. Note that the authors have chosen not to conceptualise external validity in terms of “ecological validity”, due to the particular meaning of this term within the Brunswikian research approach, disagreement about the meaning of this term and accordingly, claims of misunderstanding and misuse in the literature: L Petrinovich, “Representative Design and the Quality of Generalisation” in L W Poon, D C Rubin and B A Wilson (eds), Everyday Cognition in Adulthood and Late Life (Cambridge Uni Press, New York, 1996). See, too, in this context Bornstein, op cit n 32. 43 See Bray and Kerr, op cit n 12; Petrinovich, op cit n 42. 44 See Bray and Kerr, op cit n 12. 45 For example, see Campbell and Stanley, op cit n 41, p 5.

February 2002

one form of validity necessarily adversely impacts upon the other. However, not all social scientists accept this position. In a series of provocative articles, for instance, Mook46 argued against conventional emphases on the need for high external validity, arguing that some research can be low on generalisability but still be significant and nomothetic because of the conceptual or theoretical implications of the results obtained. From this perspective, direct generalisation of original findings from an experimental study is frequently immaterial; hence obsessive attention to external validity is misguided.47 Mook’s arguments have by no means been unchallenged.48 Notably, however – and critical for the current analysis – Mook49 exempted applied psychological research from his argument, accepting that in this sphere external validity would always be important. Mook50 distinguished applied psychological research on the basis of its raison d’être which necessarily requires generalisation to another specific setting, group or whatever. While issues of internal and external validity underpin all scientific research, problems of external validity particularly have been noted in the broad psychology-law area, due to the applied nature of much of the research carried out. As will be seen subsequently, problems of external validity were emphasised by Barr J in Attorney-General (NSW) v John Fairfax Publications and can be broadly summarised as: 1. concerns regarding the effect of the delay between the pre-trial publicity and trial; 2. the significance of findings in relation to the bogus names; 3. the comparability of the emp irical study with an actual trial; 4. the motivation of research participants compared with jurors; and 5. the limitations of the context of testing. Additionally, Barr J did not accept the interpretation of the results of the empirical study offered by the expert witness.

46

D Mook, “In Defence of External Invalidity” (1983) 38 American Psychologist 379; D Mook, “The Myth of External Validity” in Poon et al, op cit n 42. 47 Compare Weiten and Diamond, op cit n 12. 48 For example, see Petronovitch, op cit n 42. 49 Op cit n 46. 50 Op cit nn 42, 46.

357

FRECKELTON and McMAHON

Delay Barr J acknowledged that only one-third of potential jurors within the relevant area read the Sydney Morning Herald, this meaning that the recognition rates reported in the applied study needed to be significantly discounted in the assessment of the real likelihood of the publicity causing a significant prejudice to the fair trial of Duong Van Ia. Moreover, the testing had been carried out only two weeks after the participants read the newspaper that they were given, whereas the trial of the accused was not due to take place until five months after the adverse pre-trial publicity in the Sydney Morning Herald. Professor Vidmar’s view was that memory of other activities would occur during the interim, contributing to further decline in me mory of the articles. Barr J accepted the validity of the point but commented: “[I]t is difficult to know how great it is in view of the evidence of both experts that memory of the detail of an event rapidly diminishes unless there exist any of the features which may combine to produce long term memory.” 51 He acknowledged that unless an event “is of such interest or importance or of such a striking nature as to produce long term memory, little if any of it will remain in the memory after two weeks”. However, he further stated: “On the other hand, ... whilst it seems reasonable to suppose that that memory will fade over a long time, it is difficult to say whether there will be any significant loss of memory between a time two weeks after the event and a time five months after the event.”52 He accepted that the concentration of the questions on drugs and drug bosses would be likely to prompt guesses consistent with the anticipated purposes of the study, this being another of the factors tending to make the results of the survey conservative.

Bogus names Barr J found that the sequencing of the questions about the bogus names “must have led the minds of the participants to an expectation that they would be asked about a Vietnamese person who had some

connection with the illegal trade”. 53 Secondly, the question asked in respect of one of the fictitious men would have been true of the accused, meaning, so Barr J held, that the false identification should be given little weight. Finally, and most importantly, the names chosen included the name “Tran”, a very common Vietnamese name, and the name of a number of persons who had been the subject of significant publicity in respect of violent and drugrelated crime. Barr J stated that he did not accept that the study results demonstrated the existence in the community of any general assumption that Vietnamese persons are involved in drug or other crimes: “I therefore reject the conclusion of Professor Vidmar that the survey results show that the ‘drug bosses’ and ‘Uncle Six’ articles resulted in relatively little recognition and recall memories of Mr Duong as a drug boss that could be separated from general assumptions about persons with Vietnamese names being involved in drugs and other crimes.” 54 It should be recalled that the experimental group (Group A) reported high recognition rates of the two bogus Vietnamese names (33 per cent and 19 per cent). While, on the basis of the previously mentioned reasoning Barr J discounted the significance of these findings, it is important to note that critics in the United States (such as prosecutors confronting the use of pre-trial publicity research by defendants seeking a change of venue) have also reported false recognition rates as high as one-third of persons surveyed.55 The issue warrants further attention, as does the general function and interpretation of bogus names in pre-trial publicity research.

Trial versus applied study Just as Barr J dismissed the general evidence of research on pre-trial publicity because it was not sufficiently similar to the situation of the courtroom, he did not view the results of the applied study as assisting him in any significant way with the task of evaluating the effect of the relevant pre-trial material in the Sydney Morning Herald. He

53

Ibid at [75]. Ibid at [81]. 55 C Haney, Declaration in People v Hunt, Thompson & Lainer (Superior Court of California, Yolo County, California, 1991, Nos 11694, 11971), cited in Moran and Cutler, op cit n 37. 54

51 52

[1999] NSWSC 318 at [84]. Ibid.

358

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

identified a series of factors that distinguished the experimental conditions in the applied study from those that would prevail at a trial of the accused. He found these to stand in the way of giving effect to the survey results. First, he noted that in the applied study the critical testing of recall and recognition was carried out without warning, save the indication two weeks previously that participants might be contacted. This feature of the study ensured that there was little possibility for stimulation of memories in the interim period. This, he held, contrasted markedly with the trial process. Secondly, he was troubled by the fact that in the study, by contrast with a trial, the only possible means of memory stimulation for participants was the questions that they were asked. Barr J here identified limits in the measurement of effects at the relevant (phase 2) stage of testing. The study, relying on single-event questioning of participants by telephone at the critical phase, only employed verbal recall and auditory recognition (of relevant names) and disregarded the possibility that continued exposure to visual and other information (as would occur in the actual trial) could facilitate the recall and/or recognition of salient information. Thus, the possible specificity of effects (that is, the amount of recall and recognition) among participants may be specific to the particular testing strategies employed in the study. Accordingly, this foils the generalisation of the results to other situations, including the actual trial. Thirdly, the telephone testing was short, with the questions only being asked once. This again contrasted with the trial of the accused that could reasonably be expected to last (as it did in fact) for several days and to involve the asking of many questions in diverse ways. Again, what Barr J identified here are limitations in the measuring of effects employed in the applied study. Fourthly, the photographs of the accused were a striking (located on the front page and in colour) and important feature of the article but the participants in the study were only shown reducedsize black-and-white versions of them. Thus, the effects reported in the study may be specific to the particular stimuli utilised and may not generalise to the materials that actually constituted the adverse pre-trial publicity in this case. This aspect of Barr J’s criticism of the research was not novel: many prior researchers have commented that the February 2002

amount and type of pre-trial publicity presented in empirical research often does not match the level of exposure or the time period over which exposure occurs in actual cases.56 In the applied study, participants were exposed to the stimuli once and were tested for their memory of this material just once. However, at trial the opportunities for recall and recognition are manifold for the jurors would see the accused every day. The general differences between the environment of the courtroom and the context of the applied study were also noted by Barr J, suggesting the possibility of an interaction of the experimental stimuli with the setting. Accordingly, further doubts were raised about the generalisability of results obtained to the actual courtroom. Although Barr J accepted that identification of an actual person from a photograph may be less reliable than from a face, he held that jurors having some memory of the article and the photographs would have a significantly greater chance of memory stimulation by seeing Mr Duong and remembering the photographs than would participants in the study. Again, the particular limitations that are being addressed here are the testing materials (the photographs) and the method of measuring effects (single–event questioning by telephone in the study rather than continuous exposure to the defendant, as would occur in an actual trial). The name of the accused and the subject of drugs were never brought together in the study in the way that they would have been at trial. Barr J decided that the results of the applied study and research upon which the two experts for the defence based their opinions derived from circumstances so different from those of a jury trial that the court should be “cautious about accepting any opinion based upon them”.57

Motivation of participants versus jurors A further matter concerned Barr J. He noted that there was no way of assessing the degree of commitment of participants in the applied study and whether they had been prepared to make a genuine

56

See C Studebaker, J K Robbennolt, M K Pathak-Sharma and S D Penrod, “Assessing Pretrial Publicity Effects: Integrating Content Analytic Results” (2000) 24 Law and Human Behavior 317. 57 [1999] NSWSC 318 at [45].

359

FRECKELTON and McMAHON

effort to answer fully and honestly. He held that account needed to be taken of the fact that those asking the questions over the telephone had little control or knowledge of how people were answering the questions, and that since the participants were telephoned without warning it was likely that they were being distracted by other things on their mind and going on around them. Additionally, Barr J emphasised that the overall attrition rate of the study (22 per cent) “demonstrates that not every participant was wholeheartedly committed to the success of the survey. [By contrast] there are well established methods of ensuring that the attention of jurors at trial is not distracted.”58 Researchers in social science have long been familiar with these sources of concern.59 Thus, Hans and Vidmar, 60 commenting on survey research on the effects of prejudicial pre -trial publicity, noted that respondents differed significantly from actual jury members insofar as the former were anonymous, not under oath and asked questions in conditions that were considerably different to the formal atmosphere of the courtroom. They cautioned that “judges’ concerns about the correspondence between answers given in a survey and in a courtroom are by no means completely unfounded”.61 This combination of factors led Barr J to a view that has application well beyond the factual matrix of the particular case. He reflected that it was impossible to replicate trial conditions in an empirical study and commented on the “consequent difficulty about accepting survey results as a reliable indicator of what might happen at a trial”. 62

Limitations of the context of testing Barr J found that a range of characteristics which had been described by the newspaper articles could be raised at trial, giving jurors a further means of recognising the accused as the man the subject of the Sydney Morning Herald articles. He acknowledged that the articles on 27 October 1997,

58

Ibid at [96]-[97]. See Fulero, op cit n 1; Weiten and Diamond, op cit n 12. V Hans and N Vidmar, “Jury Selection” in N L Kerr and R M Bray (eds), The Psychology of the Courtroom (Academic Press, New York, 1982). 61 Ibid, p 51. 62 [1999] NSWSC 318 at [95]. 59

when read by the participants in the study, were not current news, suggesting that they were less likely to be memorable to participants than they would have been to potential jurors when reading them at the time of publication. In addition, the articles as published at the time were part of a series and so potentially retained by readers and even re-read between the date of publication and the date of the trial, especially within the first few days after publication.

The interpretation of the results Barr J went even further, finding that a number of the study results showed the opposite of the position contended for by the publisher. He noted that 11 members of Group A reported that they recognised the name “Duong Van Ia”, seven of them in connection with drugs, and 25 participants in this group recognised the nickname “Uncle Six”, 22 of them in connection with drugs.63 He accepted submissions from the Attorney-Genera l that it followed from the results that 27 of the 79 members of Group A identified either Duong or Uncle Six as a person connected with drugs. He classified this as a “high rate of recognition”. The different interpretations of the results offered by the defence expert and Barr J may be related to the interpretation of the legal standard employed. At issue here is the criterion used in deciding what constitutes a real and practical tendency to interfere with the administration of justice: while the expert witnesses referred to very low probabilities that the pre-trial prejudicial publicity would be recalled five months later by actual jury members, Barr J utilised a test that was satisfied if any of the jurors remembered any of the prejudicial material. Barr J concluded that the limitations of the study were so great and the differences between the conditions of the study and those which would apply at a trial were so marked that “the survey results cannot form the basis for any reasonable conclusion that there was a small likelihood as a matter of practical reality that the material complained of had significant adverse effects at the trial of Mr Duong on the beliefs and attitudes of persons exposed to them”. 64 In addition, he found that Professor Vidmar had not

60

360

63 64

Ibid at [103]. Ibid at [102].

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

demonstrated any basis independent of his study results that reasonably supported his opinion about the unlikelihood of the adverse impact of the Sydney Morning Herald publicity about the accused. In respect of Professor Vidmar, Barr J was also troubled by what he classified as the “theoretical inclination” of the witness. By this, he appeared to refer to the reluctance of Professor Vidmar to extrapolate from other empirical studies and relevant theory to the particular circumstances of the Duong case. He noted that Professor Vidmar was disinclined to speculate when his opinion could not be based upon empirical research. This is a curious, significant and problematic aspect of the culture clash between psychology and law manifested in the case. The exp ert witnesses endeavored to protect themselves against assertions that they were biased or were engaged in speculative comments by confining themselves rigorously to available scientific data. Interestingly, this position conformed to the expectations of the Chief Justice of the High Court of Australia who previously castigated a psychologist for giving evidence which exceeded his professional area of expertise and involved speculation, surmise and conjecture.65 However, Barr J clearly found the psychological evidence adduced before him to be theoretical and unhelpfully confined to scenarios other than that in the case which he had to decide. What he wanted of the witnesses was a preparedness to extrapolate from the general knowledge about jury influences to the case before him in a way that the witnesses did not regard as scientifically justifiable. Thus the witnesses failed to provide what would have been most helpful for the forensic fact-finding process, doing so both as a matter of what they perceived to be scientific propriety and also so as to avoid breaching the speculation preclusion. Future appearances by experts in Australian courts may involve walking a “tightrope”, with concern to avoid speculation and scientific impropriety (“going beyond the data”) balanced against the desire of some judges to have experts freely translate theoretical findings to current cases.

65

HG v The Queen (1999) 197 CLR 414; I Freckelton and T Henning, “Speculation, Uncorroborated Opinions and Forensic Expertise in Child Sex Prosecutions” (1999) 6 (1) Psychiatry, Psychology and Law 105; Freckelton and Selby, op cit n 14.

February 2002

The ramifications of the Fairfax case: External validity, pre -trial publicity and the elusive “gold standard” of research design The reasoning of Barr J in Attorney-General (NSW) v John Fairfax Publications was arguably only obiter dicta as the basis for his dismissal of the charge against the Sydney Morning Herald was that the public interest defence was available for the defendant. However, another, and better, view is that the ruling about the pre-trial publicity was an integral part of his Honour’s reasoning, the result of which was that the question of the public interest defence arose for consideration. This means that not just the decision in respect of the tendency of the articles, but the way in which the decision about the tendency was arrived at by Barr J, are of real significance in terms of the precedent that they have set. On any analysis, the Sydney Morning Herald case involves the most significant use of applied psychological research in criminal proceedings in Australia, both because of the efforts made by the publisher to mount a defence by the commissioning of the applied study and by the extensive use of evidence from the two United States social psychologists (one of whom is an acknowledged expert of international standing in the area), and because of the extent to which the reasoning of the experts was subjected to critical scrutiny by Barr J. The decision has considerable relevance for social scientists as it addresses many of the concerns about research design and methodology that have been prominent in the literature over the past three decades. The decision by Barr J moves these concerns, at least in Australia and other common law countries outside the United States, from the realm of academic controversy to the forefront of forensic practice. Subsequent applied psychological research must successfully address the issues raised in Barr J’s decision to be deemed persuasive, at least by Australian judges. Corbin, Gill and Jolliffe 66 have suggested that the experimental evidence presented to Barr J was “next best” evidence. However, Barr J. dismissed the utility of this evidence. Implicit in his analysis was a “gold standard” for future research in this area that

66

Op cit n 7, p 128.

361

FRECKELTON and McMAHON

will meet criteria deemed persuasive in the forensic context. This standard suggests that such research should be quasi-experimental in nature, utilise actual pre-trial publicity materials, employ participants who are similar to the actual jury pool, include multiple items of information in the pre-trial publicity and incorporate mock juries – all factors that have been identified by previous commentators who have reviewed research in this area. 67 This suggested research strategy has not yet been widely utilised in relation to investigations of the effect of pre-trial publicity68 but has been persuasively endorsed by leading researchers.69 Another two factors that are foundational for future research – one pertinent only, perhaps, to common law countries outside the United States, the other of general import – are that such research should focus strongly on the memorial (rather than attitudinal or evaluative) impact of pre-trial publicity and that explorations of the impact of pre-trial publicity must occur in the context of jury (rather than individual juror) functioning. The former point follows from the test adopted in Australia for the likelihood that pre-trial publicity will prejudice the prospect of a fair trial and suggests that much of the research conducted in the United States (such as that which investigates the biasing impact of pre-trial publicity on juror judgments)70 may not be directly relevant to the Australian legal context. The latter point follows from the possible synergistic effect on memory of participation in a jury.

The problem of inductive research: External validity and “impossibilism” Inherent in the criticisms that Barr J directed at the research conducted by the expert witnesses for the defence in Attorney-General (NSW) v John Fairfax Publications are profound issues about the use of empirical evidence in the law, acceptable forms of empirical research and – at bedrock – questions relating to scientific method. Thus far, the analysis has explicated Barr J’s critique through the explanatory framework of “external validity”. As

previously noted, the issue of external validity in relation to scientific research is an essential and recurrent issue in social scientific research. However, discussions of external validity also raise more fundamental questions about scientific method. The problem of external validity (the generalisability of research) can be traced to the 18th century Scottish philosopher David Hume whose epistemological scepticism led him to argue that induction (and, accordingly, any scientific generalisations based on it) is never fully justified logically. Despite the logical problem associated with induction, Hume acknowledged that individuals use inductive reasoning in everyday and scientific reasoning. Underpinning much common reasoning as well as scientific theorising is the assumption that the closer two events are in time, space and measured value on any or all dimensions, the more they tend to follow the same laws. In relation to modern research in the psychologylaw area, Bray and Kerr 71 acknowledge the general problems of inductionism and note that some critics argue that it is impossible to generalise from simulation research to actual trial. They note: “In a strict sense they are correct since … a study’s results are always conditional on the particular choice of actors, contexts and behaviors towards objects.”72 However, it has also been claimed that “Most contemporary philosophers, as well as most social scientists, regard the logical problem of induction as a technicality.”73 In practice, concern has focused on major threats to inductive generalisations, typically identified as problems of generalising across experimenters, populations, treatments and research settings. The logical problem of induction has been reconceptualised as the problem of external validity which, in turn, has been regarded as ultimately empirical in nature. 74 The reasoning of Barr J – the insistence that the research design replicate the actual experience of

71

Op cit n 12. Ibid, p 309. 73 J Neale and R Liebert, Science and Behavior: An Introduction to Methods of Research (3rd ed, Prentice-Hall, New Jersey, 1994), p 255. 74 See, eg, G P Kramer and N Kerr, “Laboratory Simulation and Bias in the Study of Juror Behavior” (1989) 13 (1) Law and Human Behavior 89. 72

67

For instance, see Studebaker et al, op cit n 56. See Carrol et al, op cit n 3. For instance, see Studebaker et al, op cit n 56. 70 See, eg, A L Otto, S D Penrod and H D Dexter, “The Biasing Effect of Pretrial Publicity on Juror Judgments” (1994) 18 Law and Human Behavior 453. 68 69

362

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

potential jurors and, most critically, his reference to the “impossibility of replicating trial conditions in a survey”75 – suggests a contemporary attack on inductionism and, ultimately, an anti-scientific orientation (that is, suggesting that some phenomena, such as trials and courtroom processes, simply cannot validly be investigated experimentally). Indeed Barr J was curiously inconsistent on this point, for he also stated that the evidence of the expert witness, Professor Vidmar, “suffered because of his unwillingness or inability to answer certain questions by reference to his own ‘experience of life’”. 76 Thus, a rejection of the specific empirical research conducted for the case on the basis essentially of a lack of external invalidity was combined with a desire to have more detailed general information about theoretical principles derived from memory research and research on pretrial publicity plus information derived from the expert witness’s own (non-scientific) “experience of life”. However, “experience of life” is often a euphemism for anecdote, bias and untested opinion. Additionally, if Barr J had used the same rigorous criteria that he employed when evaluating the specific research study presented to the court in the in Attorney-General (NSW) v John Fairfax Publications to much of the relevant psychological research that could have been cited or relied upon by an expert more prepared to engage in conjecture, it is doubtful whether he would have found this research any more acceptable than that which was sought to be tendered to the court. Indeed, this point is established by his critical comments on much of the research relied upon by Dr Williams which, Barr J reported, “made no attempt to replicate a criminal trial and … [was] a poor substitute for a trial”. 77 How is the social scientist who appears before the courts to address the concerns noted by Barr J in Attorney-General (NSW) v John Fairfax Publications? Although Barr J’s criticisms of the empirical research in the case before him suggest a return to an implacable attack on inductionism, addressing the factors that are identified as influencing the external validity of research conducted within the general, positivist framework of contemporary psychology may provide some manner of

facilitating the future acceptance of empirical psychological research into Australian courts. In turn, this requires close consideration of Barr J’s criticisms as well as a familiarisation with the extant literature on juries and the “psychology of the courtroom”, for nearly all the difficulties identified with the applied research in Attorney-General (NSW) v John Fairfax Publications have been previously considered in this literature. Thus, more than two decades ago an American commentator noted: “Studies concerning the jury’s performance in situations involving pre-trial publicity all have their weaknesses. The major difficulties are first, that the subjects were not real jurors but were either students, or a cross section of adults in the community, or persons whose names were drawn from voter registration lists; second, that the stimuli did not adequately characterise real trials because the subjects were exposed to audio tapes, written statements, summaries of trials, or some combination thereof; and, finally, that the setting in which the research was conducted was not a courtroom but a university or someone’s home.” 78 The decision of Barr J revives many of the methodological criticisms noted in the psychological literature and has a number of implications for social scientists proposing to conduct applied research for forensic purposes. The call for greater external validity in applied social psychology research such as that pertaining to the effects of pre-trial publicity on potential jurors can be responded to in two principal ways. First, new research strategies need to be developed to overcome some of the deficiencies already referred to. Thus, Neale and Liebert 79 have argued that greater external validity can be achieved by conducting field research, utilising broad samples and conducting meta-analyses of relevant studies on the phenomenon of interest. Secondly, the quality of research done within the experimental and quasiexperimental paradigm needs to be improved. This is the dominant research strategy in psychological studies on the effects of pre-trial publicity. In this

78 75

[1999] NSWSC 318 at [48] (emphasis added). 76 Ibid. 77 Ibid at [42].

February 2002

R Simon, “Does the Court’s Decision in Nebraska Press Association Fit the Research Evidence on the Impact on Jurors of News Coverage?” (1977) 29 Stanford Law Review 515 at 520. 79 Op cit n 73.

363

FRECKELTON and McMAHON

framework, improving external validity requires the maximum similarity of an experiment to the conditions of application that is compatible with internal validity.80 It is clear that attention needs to be given to defining the target situation adequately. Considerations of generalisability and representativeness – the key concepts in external validity – require consideration of “representative of what”? Perhaps the most important task in maximising the external validity of an experimental design is a rigorous and comprehensive definition of the target situation. Obviously, in research investigating the effects of pre-trial publicity on memory for names and faces of individuals who subsequently are defendants in criminal trials, the broad target situation involves investigating the effects of such publicity on the memory of jurors. But more specific features of the target situation demand consideration.

Sample Utilising a sample that is representative of the target population of jurors is essential. 81 This consideration has both direct and indirect significance, as sample selection may interact with testing and measurement effects to limit external validity.82 Although Neale and Liebert83 argued for the utilisation of broad samples as a means of increasing external validity, in the particular area of research being considered (the effects of pre-trial publicity on jurors’ memories) it is likely that, given the circumscribed pool of persons who may serve as jurors, this strategy will be less effective than sampling of the relevant population or use of actual venires. Selection of actual venires, although desirable, may present the researcher with substantial challenges, as Australian juris dictions are more stringent in restricting disclosure by jury members than is common in the United States of America. 84 Additionally, it is important to report the

80

See Campbell and Stanley, op cit n 41, p 18. See Carroll et al, op cit n 3; N M Steblay, J B Besirevic, S M Fulero and B Jimenez-Lorente, “The Effects of Pretrial Publicity on Juror Verdicts: A Meta-analytic Review” (1999) 23 Law and Human Behavior 219. 82 See R Tourangeau, L J Rips and K Rasinski, The Psychology of Survey Response (Cambridge University Press, Cambridge, 2000); Weiten and Diamond, op cit n 12. 83 Op cit n 73. 84 See, eg, Juries Act 2000 (Vic), ss 77, 78. 81

364

actual participation rate in empirical studies. As well as describing participants, research studies should report information on the number and type of individuals who have been approached but declined to participate in the study. From this information it is possible to estimate potential selective biases in the sample, including a possible interaction between selection and the experimental treatment.

Setting Utilising a setting that is representative of the target setting is also critical. Rather than adopt the a priori belief that it is impossible to replicate an actual trial, attention should be directed to ascertaining the features of a trial that are essential to preserve the essence of the phenomena under investigation:85 the experimental setting should preserve the “processes of interest” in the target setting.86 In the psychology-law area this source of invalidity is most significant, given the status, authority and symbolism associated with the particular setting of courtrooms.

Pre-trial publicity materials It is important to use materials, including photographs, that are representative of the target manipulation/key event. Thus, use of actual (rather than fictional) pre -trial publicity materials is essential.87 Although this strategy was employed by the researchers called by the defence in AttorneyGeneral (NSW) v Fairfax, it is also essential in applied studies that the materials be identical, not a reproduction that may lack the full impact of the original. Future research should also recognise that the effect of pre-trial publicity is influenced by the type of crime described; prior research has already identified strong effects when crimes related to drugs are specified.88 Additionally, some adverse pre-trial publicity may be admissible at an accused’s criminal trial and this should be acknowledged in the design of research. This situation contrasts with much of the available research, which has investigated the effects of prejudicial publicity that was not admissible at the subsequent trial of an accused person.89 Finally, materials should be

85

Weiten and Diamond, op cit n 12. Mook, op cit n 46. 87 See Studebaker et al, op cit n 56. 88 See Steblay et al, op cit n 81. 89 For instance, Kramer and Kerr, op cit n 74; Kramer et al, op cit 86

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

examined for emotionally and factually biasing content for, although this distinction is not unproblematic, it has been productively utilised in relation to evaluative impact90 and may also be useful when assessing the memorial impact of pretrial publicity materials.

Testing Memory testing in relation to measuring the effects of pre-trial publicity has two essential characteristics that have, thus far, been relatively disregarded in the literature. First, such testing must be direct and comprehensive, investigating at least visual and auditory memory for pre-trial publicity. Thus, it follows that “mere” telephone contact with interviewees will be insufficient. Direct contact between interviewer and subject enables gauging of the attitude and non-verbal responses of the interviewees, as well as permitting recall of visual material. 91 Secondly, the testing must include some group testing of recall. The judgment of Barr J relocates research on the effects of pre-trial publicity clearly within the general domain of research on juries, suggesting that the appropriate unit for analysis in subsequent research in this area must include juries as well as individual jurors.

Measures Concern has long been expressed about the use of inappropriate dependent variables in research on juries.92 Recently, Steblay et al93 noted that “The impact of PTP [pre-trial publicity] on a juror’s decision has been the focus of PTP research”. However, a different focus – and therefore different dependent variables – may be required in Australia and similar jurisdictions. The Australian legal test, in contrast to the position in the United States, suggests that the prejudicial effects of pre-trial publicity are established if any jurors remember any of the adverse pre-trial publicity about an accused person.

n 12. 90 For example, Kramer et al, op cit n 12; Studebaker et al, op cit n 56; cf J Wilson and B H Bornstein, “Methodological Considerations in Pretrial Publicity Research: Is the Medium the Message?” (1998) 22 Law and Human Behavior 585. 91 Testing which simulates the circumstances of an actual trial will necessarily involve repeated presentations of both auditory and visual stimuli. 92 Weiten and Diamond, op cit n 12. 93 Op cit n 81, at 230.

February 2002

Hence, psychological research in countries embracing such a test needs to focus on investigating the memorial (rather than attitudinal or evaluative effects) of pre-trial publicity. Much of the research conducted in the United States on the effects of pretrial publicity on sympathy for a defendant, likeability of a defendant, or assessment of a defendant’s guilt 94 will not be directly relevant to the situation in Australia. Addressing these matters of external validity does not eliminate the “problem of induction” which underlies much psychological reasoning and theorising and which is the core of the critique offered by Barr J in Attorney-General (NSW) v John Fairfax Publications. However, it is the practical manner in which most contemporary social scientists address this problem and provides a pragmatic – and probative – strategy for conducting research on legal phenomena.

Conclusion More than two decades ago Weiten and Diamond 95 warned: “As the legal establishment devotes greater attention to empirical research … the external validity of that research becomes a critical issue.” The challenge confronting social researchers who may seek to adduce evidence in future cases will be to persuade courts that empirical data, whether from experimental studies or otherwise, are sufficiently applicable to the actual circumstances of the case before the court to be accorded probative value. Barr J was dissatisfied with evidence extrapolated from a study in which many of the influences germane to the particular case did not exist. To a degree the problems identified by Barr J constitute another example of disagreement between social scientists (especially psychologists) and lawyers concerning the threshold that must be reached before significant value can be attached in the forensic domain to the fruits of their research. This issue has arisen in other jurisdictions and in relation to other forensic issues. For example, similar controversy has arisen in the United States

94

See, for instance, Moran and Cutler, op cit n 37; Otto, Penrod and Dexter, op cit n 70; Steblay et al, op cit n 81; Wilson and Bornstein, op cit n 90. 95 Op cit n 12, at 72.

365

FRECKELTON and McMAHON

concerning the utility of research on eyewitness testimony in criminal proceedings.96 Although Fulero advocated that “[p]sychologists should not be deterred in the presentation of empirical data to courts by those who urge overly high standards of proof prior to participation in the legal process”,97 social scientists working in Australian jurisdictions will have to come to terms with the high benchmark established by Barr J. What we have argued is that part of the “coming to terms” with this benchmark lies in addressing some of the methodological deficiencies highlighted by Barr J. There is a fundamental problem in simulating trial conditions. In its entirety it is not feasible, just as it is not viable to bridge the inductionist problem. The best that can be done experimentally is a snapshot of something akin to the trial process, capturing a degree of the complexities, inter-relationships and qualities of the criminal trial. But some attempt toward this end (better than was done by the researchers in AttorneyGeneral (NSW) v John Fairfax Publications) needs to be made in order to afford comparability of conditions. Barr J’s judgment draws important attention to the gulf between, for instance, a telephone poll context and what takes place within a criminal court over an extended period of time and with the repetition of cues on multiple occasions. The constructive legacy of Barr J’s decision should be a greater sensitisation on the part of researchers to the kinds of factors that have the potential to distance experimental scenarios from curial contexts to a point where generalisations from the former become strained and even spurious. A number of these considerations have the potential to

be addressed to a significant extent by improved methodologies which focus upon the kinds of conditions likely to enhance recall and recognition memory in the forensic context. It is important, though, that sight not be lost in a purist scientific fervour of the fundamental proposition that courts can usefully be assisted, as Barr J in fact was, by the provision of information outside the ken of non-psychologists about the processes of memory, recall and recognition, as well as by a host of other social science matters. At least the provision of such background information enriches the fact-finding processes and enhances it by factoring out potential sources of uninformed error. Additionally, despite the onerous demands of conducting acceptable research in this area, the need for such research is clearly demonstrated by the inefficacy of traditional safeguards (such as continuance, change of venue and judicial instructions) that can be utilised in Australia and similar jurisdictions to address the effects of adverse pre-trial publicity. For instance, it is important that the forensic backlash against psychological evidence that may be prompted by Barr J’s decision not extend to the reception and valuing of useful counterintuitive information which many psychologists can provide uncontroversially to the courts.98 A positive consequence of the New South Wales Supreme Court decision in Attorney-General (NSW) v John Fairfax Publications should be considered reflection by researchers about how best they can bridge the potentially problematic research/curial divide, a microcosm of the more general divide between social science and the law.

98

96 97

See Fulero, op cit n 1. Ibid at 263.

366

See R W Davis, “Pretrial Publicity, the Timing of the Trial, and Mock Jurors’ Decision Processes” (1986) 16 Journal of Applied Social Psychology 590; Steblay et al, op cit n 81; Kramer et al, op cit n 12.

JOURNAL OF LAW AND MEDICINE – Volume 9

Social Science Research and Experimentation in Australian Criminal Proceedings

Table 1. Recall and Recognition of Newspaper Information Group A Stage 1 (n) Total: 188 Task: read newspaper editions

98

27 Oct

Group B 37

27, 28, 29 Oct

Group C

Group D

28

25

3 Nov

3, 4, 5 Nov

Stage 2 (n) Total: 146

79

30

19

18

Attrition rate (%) Overall: 22%

19

19

32

28

Recognition items

%

%

%

%

Names: Duong Van Ia Uncle Six Duncan Lam Bogus name #1 Bogus name #2

15 37 24 19 33

10 20 10 20 20

5 11 11 21 26

6 6 17 6 17

Assisted recall Read/heard about: Duong Van Ia Yes No Don’t know Total

14 75 11 100

0 93 7 100

11 74 16 100*

6 89 6 100*

Uncle Six Yes No Don’t know Total

32 63 5 100

23 67 10 100

0 100 0 100

0 94 6 100

* Due to rounding of percentages, the total of these response categories exceeds 100. In addition, some minor discrepancies exist between this Table, which is based upon the figures provided to the court, converted into percentage form, and those figures referred to by Barr J.

February 2002

367