Evidence-based Practice: A Critical Appraisal

EDITORIAL

Evidence-based Practice: A Critical Appraisal Published online 8 February 2012 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/oti.1322

Welcome to this special issue on evidence-based practice and occupational therapy. It has been a privilege to be the guest editor and to learn about evidence-based practice in the Netherlands (Ammeraal and Coppers, 2012; Döpp et al., 2012), Norway (Aas et al., 2012), the Republic of Ireland (Stronge and Cahill, 2012) and the United States (Winkle et al., 2012). In this editorial, I will draw out some themes from the five papers and offer a personal perspective on evidence-based occupational therapy. Evidence-based practice is framed as an evolving, contested concept with many achievements and some notable challenges ahead. Some theoretical frameworks, references and online resources are signposted to help the reader explore the nuances of evidence-based practice. In 2007, evidence-based medicine (EBM) was voted as one of the 15 greatest medical advances since 1840 (BMJ 2007). The success is unsurprising given that evidence-based practice is about clinical efficacy. This focus is stated in two simple questions: Does it work? And is it cost effective? These questions are pertinent to everyone with a stake in health care throughout the world. This includes policy makers responsible for funding healthcare systems; organisations that provide health care; and multi-disciplinary teams and individual practitioners incorporating research results into clinical practice and, most importantly, to patients who want ethical, efficacious treatment. The term EBM first appeared in the medical literature in the early 1990s when it was presented as a fundamentally new approach to the teaching and practice of medicine (Dawes et al., 2005). This was because EBM promotes the examination of evidence from clinical research and requires new skills, especially critical appraisal. Such critique challenges custom and practice, making it legitimate to question authority or eminence-based practice (Isaacs and Fitzgerald, 1999). A critical attitude towards one’s own practice and to the evidence is, in my opinion, one of the greatest achievements of the evidence-based movement. Critical

Occup. Ther. Int. 19 (2012) 1–6 © 2012 John Wiley & Sons, Ltd.

thinking can, and perhaps should, be integrated with reflexive practice (Bannigan and Moores, 2009). There are many tools to support critical thinking, appraisal and action. The evidence-based practice movement coincided with technological advances, widening access to repositories of knowledge, via the Internet and electronic databases. The authors in this special issue refer to several resources including the Cochrane Library, http://www.thecochranelibrary. com/view/0/index.htmland, and PubMed, http://www. ncbi.nlm.nih.gov/pubmed/. An additional resource is OTseeker, which is a database of systematic reviews and critically appraised randomized controlled trials. It was established in 2003 and now contains over 7,400 studies relevant to occupational therapy. This is a rare, open access occupational therapy resource, http://www.otseeker.com/. Another resource, the HINARI programme, is beginning to address the particular challenges to evidence-based practice in resource-constrained countries with limited infrastructure and little relevant evidence (Buchanan, 2011). The HINARI programme was set up by the World Health Organisation in 2002 with major publishers to allow developing countries to gain access to biomedical and health literature. There are more than 8,000 information resources available to health institutions in 105 countries, http://www.who.int/hinari/en/. Importantly, the HINARI resources are in 30 different languages, overcoming one of the biggest barriers to equal access to occupational therapy knowledge internationally – the English language (Ilott et al., 2006). This was an obstacle for Dutch occupational therapists (Döpp et al., 2012). Evidence-based medicine and practice have evolved over the last 20 years (Dawes et al., 2005; Montori and Guyatt, 2008). Dawes et al. (2005) in the Sicily statement on evidence-based practice (EBP) define EBP as requiring “decisions about health care are based on the best available, current, valid and relevant evidence. These decisions should be made by those receiving care, informed by the tacit and explicit

1

Ilott

knowledge of those providing care, within the context of available resources” (Summary, para. 2). This definition retains the triad identified by the pioneers, namely research, patient choice and practitioner wisdom (Sackett et al., 1996; Bury and Mead, 1998; Haynes et al., 2002). It also recognizes the criticality of the context, especially competing priorities, resource constraints and the myriad differences between settings, populations, countries and people (Lin et al., 2010; Estabrooks et al., 2011). A feature that has remained constant is the emphasis on putting the five-step evidence-based practice process into curricula, to inculcate appropriate knowledge, skills and attitudes (Dawes et al., 2005). The steps, taken by individual practitioners, can be summarized in five As. These are assessing the evidence-base, asking an answerable question, acquiring relevant evidence, appraising the relevance and scientific rigour and applying in practice. The World Federation of Occupational Therapists expects occupational therapy students to become competent in the EBP process (WFOT, 2006). According to Stronge and Cahill (2012), this expectation is being met by senior students in the Republic of Ireland. Two papers use the steps as the framework for integrating evidence with clinical reasoning (Aas et al., 2012; Döpp et al., 2012). Both give insights into the theoretical and practical difficulties associated with each step of the EBP process. The call for papers focused on two aspects of evidence-based practice: systematic literature reviews and implementation. High quality, systematic literature reviews, which use a transparent and replicable methodology, offer a short cut around the EBP process by providing pre-appraised evidence. They are a way of coping with the exponential growth in research – the volume of medical papers doubles every 10–15 years (Dawes et al., 2005). Although systematic reviews take a variety of forms (Grant and Booth, 2009), they all present synthesized summaries of what is known about a topic. This is a vital starting point for research and development. The number of systematic reviews and randomized controlled trials in OTseeker has almost doubled since June 2006 when there were 4,340 entries (Bennett et al., 2007). This is a positive sign, if interpreted as a barometer of the shift from being a research emergent to a research active profession. Such a shift could be attributed to the increasing demand for evidence of clinical and cost effectiveness. This special issue has one systematic review. It is about an emerging 2

Editorial

area of practice – using service dogs to facilitate independent living (Winkle et al., 2012). Applying research to decision making in everyday practice is the most problematical step in the EBP process. This is why the call for papers focused on research implementation. Research can be used in different ways, including conceptually, to inform thinking; instrumentally, to influence behaviour; and symbolically, to persuade others (Graham et al., 2010). In EBP, research knowledge is supposed to be used instrumentally, through decision making and behaviour, by putting research into practice to benefit patients. However, this is a slow, haphazard and unpredictable process (Eccles et al., 2009). It is known as the research–practice gap (Squires et al., 2011) or the second gap in knowledge translation (Woolf, 2008). Unsurprisingly, four of the five papers address this gap, albeit with little reference to theory. The implementation challenge is receiving considerable attention from theorists and researchers, including occupational therapists, under the topics of research utilization (Brown and Rodger, 1999; Cameron et al., 2005), knowledge translation (Metzler and Metz, 2010; Colquhoun et al., 2010) and implementation science, which has an open access journal (http:// www.implementationscience.com/). These developments extend the scope of evidence-based practice into quality improvement, as another way of trying to close the gap between health service research and everyday clinical practice (Shojania and Grimshaw, 2005; Glasziou et al., 2011). The individual model of evidence-based practice, which depends on research-based practitioners, has been supplemented by other approaches that involve national systems and local organisations (Walter et al., 2004). The research-based practitioner model, where the individual is responsible for applying the EBP process, declined in the UK following a systematic review that highlighted the complexity of changing professional behaviour (NHS CRD, 1999). A more systemic, embedded research model came to the fore in the 2000s, with a quality framework for the National Health Service in England (Ilott, 2003). In this model, research use is achieved by embedding research findings into national systems and processes, through the use of clinical guidelines and audit, and in funding and regulatory regimes, so it is managers and policy makers who play a key role. A profile of healthcare systems in 14 countries (Thomson et al., 2011) reports Occup. Ther. Int. 19 (2012) 1–6 © 2012 John Wiley & Sons, Ltd.

Ilott

that evidence-based practice is encouraged through national and regional systems in Australia, Denmark, England, Germany, New Zealand, Norway and Sweden. The embedded research model is also being challenged. Rycroft-Malone et al. (2011) suggested that the “rational–logical notion that producing research, packaging it in the form of guidelines and assuming it will automatically be used is now outdated. There is a substantial body of evidence showing that using research involves significant and planned change involving individuals, teams, organisations and systems” (Background section, para. 3). A third model, the organisational excellence model, has been successful in Australia (Caldwell et al., 2011) and is being tested in England (Rycroft-Malone et al., 2011). This model promotes collaborative, knowledge exchange between local healthcare services and universities, through learning within teams and organisations, which includes adapting research findings to fit the local context. The case study by Ammeraal and Coppers (2012) illustrates aspects of the organisational model. They customized and combined knowledge from research and practitioners, with the priorities of people with mental health problems, to improve the effectiveness of life skills interventions. Although evidence-based practice has flourished over the last 20 years, it is a contested, controversial concept. There are “enthusiastic supporters and vociferous detractors. Many clinicians view EBP as only one tool amongst many for improving quality and others believe it impedes professionalism and the ability to provide individualised, holistic patient care” (Davies et al., 2007, p. 26). Perhaps some of the most powerful critiques of EBP were raised in a special issue of the British Medical Journal, when Straus and Jones (2004) acknowledged the lack of impact from teaching EBM on improving patient outcomes. A similar finding has been reported in nursing (Shaneyfelt et al., 2006). Some of the most disputed aspects relate to epistemology, or what counts as knowledge, and the hierarchy of evidence (Tomlin and Borgetto, 2011). Clinical research is expected to be relevant, uses rigorous methods such as randomized controlled trials, with a low risk of researcher bias, so the results can be trusted and transferred to comparable settings, populations or patients (Herbert et al., 2005). This means that clinical research is ranked according to the research design – the hierarchy of evidence – with higher value assigned to designs that show cause and effect or “what works” Occup. Ther. Int. 19 (2012) 1–6 © 2012 John Wiley & Sons, Ltd.

Editorial

and moderate the effect of chance, bias or other confounding factors. However, different types of questions require different types of evidence (Tonelli, 2010), and evidence can be defined more expansively. Some definitions give equal weight to evidence from patients as experts through their experience, the clinical wisdom of expert practitioners and the organisational knowledge gained from regulations and codes of practice (Pawson et al., 2003) and from theory, such as occupational science (Kinsella and Whiteford, 2009) and neuroscience (Hubbard et al., 2009). Others emphasize that knowledge is embedded in shared decision making with patients (Coulter and Collins, 2011), social relationships and sociocultural contexts. Gabbay and Le May (2011) have studied individual and collective mindlines, which refer to the tacit, internalized knowledge derived from experience, which is reinforced by interactions with colleagues and takes account of the context. Both Döpp et al. (2012) and Stronge and Cahill (2012) refer to learning from colleagues. Such learning could be reframed as collective, occupational therapy mindlines rather than low ranked evidence. It seems appropriate to conclude this editorial with three future challenges to evidence informed occupational therapy: global disability, personalized medicine and equity for therapists working in low and middle income countries. According to the World Report on Disability (WHO and World Bank, 2011), more than one billion people live with some form of disability. We know there are links between disability, poverty, stigma and social exclusion in developing and developed countries (Van Kampen et al., 2008). This evidence is juxtaposed with the growth of molecular medicine to prevent and treat diseases (Miles et al., 2008; Samani et al., 2010). It is impossible to foresee what impact personalized medicine will have on health care in the next 20–50 years. However, it is easy to envisage the benefits of making knowledge accessible to, and applicable by, occupational therapists working in resource-constrained contexts. Buchanan (2011) laments the scant attention paid to evidence-based occupational therapy in these contexts. Perhaps we need to continue to be critical, to ask “does it work” and “does it offer value for money”, but add a third question, “how can it apply to all members of our global community”. Irene Ilott Sheffield Teaching Hospitals NHS Foundation Trust, 11 Broomfield Road, Sheffield S10 2SE, UK 3

Ilott

REFERENCES Aas RW, Alexanderson K (2012). Challenging evidencebased decision making: a hypothetical case study about return to work. Occupational Therapy International, 19(1): DOI: 10.1002/oti.326 Ammeraal MA, Coppers J (2012). Understanding living skills: first steps to evidence-based practice. Lessons learned from a practice based journey in the Netherlands. Occupational Therapy International 19(1): DOI: 10.1002/ oti.1323. Bannigan K, Moores A (2009). A model of professional thinking: integrating reflective practice and evidence based practice. Canadian Journal of Occupational Therapy 76(5): 342–350. Bennett S, McKenna K, McCluskey A, Tooth L, Hoffmann T, Strong J (2007). Evidence for occupational therapy interventions: effectiveness research indexed in the OTseeker database. British Journal of Occupational Therapy 70(10): 426–430. BMJ News Readers choose the “sanitary revolution” as greatest medical advance since 1840 (2007). British Medical Journal, 334: 111. 2. DOI: 10.1136/bmj.39097.611806.DB Brown GT, Rodger S (1999). Research utilization models: frameworks for implementing evidence-based occupational therapy practice. Occupational Therapy International, 6(1): 1–23. DOI: 10.1002/oti.85 Buchanan H (2011). The uptake of evidence-based practice by occupational therapists in South Africa. WFOT Bulletin 64: 29–38. Bury T, Mead J (1998). Evidence-based Healthcare. A Practical Guide for Therapists. Oxford: Butterworth Heinemann. Caldwell E, Fleming J, Purcell A, Whitehead M, Cox R (2011). Knowledge translation in cancer services: implementing the Research and Evidence in Practice model. British Journal of Occupational Therapy 74(11): 535–539. Cameron KAV, Ballantyne S, Kulbitsky A, Margolis-Gal M, Daugherty T, Ludwig F (2005). Utilization of evidence-based practice by registered occupational therapists. Occupational Therapy International 12(3): 123–136. DOI: 10.1002/oti.1 Colquhoun HL, Letts LJ, Law MC, MacDermid JC, Missiuna C (2010). A scoping review of the use of theory in studies of knowledge translation. Canadian Journal of Occupational Therapy 77(5): 270–279. DOI: 10.2182/cjot.2010.77.5.3 PubMed Coulter A, Collins A (2011). Making Shared Decisionmaking a Reality. No Decision about Me, without Me. London: The King’s Fund. (Available at http://www. kingsfund.org.uk/publications/nhs_decisionmaking. html) (Accessed 10 December 2011). Davies H, Powell A, Rushmer R (2007). Health Professionals’ Views on Clinician Engagement in Quality 4

Editorial

Improvement. A Literature Review. London: The Health Foundation. (Available at http://www.health.org.uk/public/ cms/75/76/313/534/Healthcare%20professionals% 20views%20on%20clinician%20engagement%20in% 20quality%20improvement.pdf?realName=W5EHsp. pdf) (Accessed 10 December 2011). Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, Porzsolt F, et al. (2005). Sicily statement on evidence-based practice. BMC Medical Education, 5:1. (Available at http://www.biomedcentral. com/1472-6920/5/1) (Accessed 10 December 2011). Döpp CME, Steultjens EMJ, Radel J (2012). A survey of evidence-based practice among Dutch occupational therapists. Occupational Therapy International, 19(1): DOI: 10.1002/oti.324 Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, Glasziou P, Ilott I, Kinmonth AL, Leng G, Logan S, Marteau T, Michie S, Rogers H, Rycroft-Malone J, Sibbald B (2009). An implementation research agenda. Implementation Science 4: 18. DOI: org/10.1186/17485908-4-18 Estabrooks CA, Squires JE, Strandberg E, Nilsson-Kajermo K, Scott SD, Profetto-McGrath J, Harley D, et al. (2011). Towards better measures of research utilization: a collaborative study in Canada and Sweden. Journal of Advanced Nursing 67(8): 1705–1718. DOI: 10.1111/ j.1365-2648.2011.05610.x Gabbay J, Le May, A (2011). Practice-based Evidence for Healthcare: Clinical Mindlines. Abingdon: Routledge. Glasziou P, Ogrinc G, Goodman S (2011). Can evidence-based medicine and clinical quality improvement learn from each other? BMJ Quality Safety 20(Suppl_1): i13–i17. (Available at http://qualitysafety.bmj.com/content/ 20/Suppl_1/i13.full) (Accessed 10 December 2011). Graham ID, Bick D, Tetroe J, Straus SE, Harrison MB (2010). Measuring outcomes of evidence-based practice: distinguishing between knowledge use and its impact. In: Bick D, Graham ID (eds). Evaluating the Impact of Implementing Evidence-based Practice. Oxford: Wiley-Blackwell. Grant MJ, Booth A (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2): 91–108. (Available at http://onlinelibrary.wiley.com/ DOI/10.1111/j.1471-1842.2009.00848.x/full) (Accessed 10 December 2011). Haynes RB, Devereaux PJ, Guyatt GH (2002). Physicians’ and patients’ choices in evidence based practice. British Medical Journal 324:1350. (Available at http://www.bmj.com/ content/324/7350/1350.full) (Accessed 10 December 2011). Herbert R, Jamtvedt G, Mead J, Hagen KB (2005). Practical Evidence-based Physiotherapy. Edinburgh: Elsevier Butterworth Heinemann. Occup. Ther. Int. 19 (2012) 1–6 © 2012 John Wiley & Sons, Ltd.

Ilott

Hubbard IJ, Parsons MW, Neilson C, Carey LM (2009). Task-specific training: evidence for and translation to clinical practice. Occupational Therapy International 16:3–4: 175–189. Ilott I (2003). Challenging the rhetoric and reality: only an individual and systemic approach will work for evidence-based occupational therapy. American Journal of Occupational Therapy 58(3): 347–352. (Available at http://www.otevidence.info/images/Ilott.pdf) (Accessed 10 December 2011). Ilott I, Taylor MC, Bolanos C (2006). Evidence-based occupational therapy: it’s time to take a global approach. British Journal of Occupational Therapy 69(1): 38–41. (Available at http://www.otevidence.info/images/ IllotMay10.pdf) (Accessed 10 December 2011). Isaacs D, Fitzgerald D (1999). Seven alternatives to evidence-based medicine. British Medical Journal 319: 1618. (Available at http://www.bmj.com/content/319/ 7225/1618.full) (Accessed 10 December 2011). Kinsella EA, Whiteford GE (2009). Knowledge generation and utilisation in occupational therapy: towards epistemic reflexivity. Australian Occupational Therapy Journal 56: 4, 249–58. DOI: 10.1111/j.1440-1630.2007.00726.x Lin SH, Murphy SL, Robinson JC (2010). Facilitating evidence-based practice: process, strategies, and resources. American Journal of Occupational Therapy 64(1): 164–171. (Available at http://ajot.aotapress.net/content/ 64/1/164.full.pdf) (Accessed 10 December 2011). Metzler MJ, Metz GA (2010). Translating knowledge to practice: an occupational therapy perspective. Australian Occupational Therapy Journal 57(6): 373–9. DOI: 10.1111/j.1440-1630.2010.00873.x Miles A, Loughlin M, Polychronis A (2008). Evidencebased healthcare, clinical knowledge and the rise of personalised medicine. Journal of Evaluation in Clinical Practice 14(5): 621–49. DOI: 10.1111/j.13652753.2008.01094.x Montori VM, Guyatt GH (2008). Progress in evidencebased medicine. Journal of the American Medical Association 300(15): 1814–1816. NHS CRD (1999). Getting Evidence into Practice. Effective Health Care. York: National Health Service Centre for Reviews and Dissemination. (Available at http:// www.york.ac.uk/inst/crd/EHC/ehc51.pdf) (Accessed 10 December 2011). Pawson R, Boaz A, Grayson L, Long A, Barnes C (2003). Types and Quality of Knowledge in Social Care. London: SCIE. (Available at http://www.scie.org.uk/ publications/knowledgereviews/kr03.pdf) (Accessed 10 December 2011). Rycroft-Malone J, Wilkinson JE, Burton CR, Andrews G, Ariss S, Baker R, Dopson S, et al. (2011). Implementing health research through academic and clinical Occup. Ther. Int. 19 (2012) 1–6 © 2012 John Wiley & Sons, Ltd.

Editorial

partnerships: a realistic evaluation of the Collaborations for Leadership in Applied Health Research and Care (CLAHRC). Implementation Science 6:1. DOI: 10.1186/1748-5908-6-74 Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS (1996). Evidence based medicine: what it is and what it isn’t. BMJ Clinical Research Ed. (Available at http://www.bmj.com/content/312/7023/71.full) (Accessed 10 December 2011). Samani NJ, Tomaszewski M, Schunkert H (2010). The personal genome–the future of personalised medicine? Lancet 375(9725): 1497–1498. Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, Whelan C (2006). Instruments for evaluating education in evidence-based practice. Journal of the American Medical Association 296:9, 1116–1127. DOI: 10.1001/jama.296.9.1116 Shojania KG, Grimshaw JM (2005). Evidence-based quality improvement: the state of the science. Health Affairs 24(1): 138–150. (Available at http://content.healthaffairs.org/content/24/1/138.full.pdf+html) (Accessed 10 December 2011). Squires JE, Hutchinson AM, Boström AM, O’Rourke HM, Cobban SJ, Estabrooks CA (2011). To what extent do nurses use research in clinical practice? A systematic review. Implementation Science 6: 1. DOI: 10.1186/17485908-6-21 Straus SE, Jones G (2004). What has evidence based medicine done for us? British Medical Journal 329(7473): 987–8. DOI: 10.1136/bmj.329.7473.987 Stronge M, Cahill M (2012). Self-reported knowledge, attitudes and behaviour toward evidence-based practice of occupational therapy students in Ireland. Occupational Therapy International, 19(1): DOI: 10.1002/oti.328 Thomson S, Osborn R, Squires DRSJ eds. (2011). International Profiles of Health Care Systems. London: Commonwealth Fund. (Available at http://www.commonwealthfund.org/~/media/Files/Publications/Fund% 20Report/2010/Jun/1417_Squires_Intl_Profiles_622.pdf) (Accessed 10 December 2011). Tomlin G, Borgetto B (2011). Research Pyramid: a new evidence-based practice model for occupational therapy. American Journal of Occupational Therapy 65(2): 189–196. DOI: 10.5014/ajot.2011.000828 Tonelli MR (2010). The challenge of evidence in clinical medicine. Journal of Evaluation in Clinical Practice 16(2): 384–389. DOI: 10.1111/j.1365-2753.2010.01405.x Van Kampen M, van Zijverden IM, Emmett T (2008). Reflections on poverty and disability: a review of the literature. Asia Pacific Disability Rehabilitation Journal 19(1): 19–37. (Available at http://www.aifo.it/english/resources/ online/apdrj/apdrj108/poverty_disability.pdf) (Accessed 10 December 2011). 5

Ilott

WHO, World Bank (2011). World Report on Disability. Geneva. (Available at http://whqlibdoc.who.int/publications/ 2011/9789240685215_eng.pdf) (Accessed 10 December 2011). Walter I, Nutley SM, Percy-Smith J, McNeish D, Frost S (2004). Improving the Use of Research in Social Care Practice. Knowledge Review. London: Social Care Institute for Excellence. (Available at http://www.scie.org.uk/publications/ knowledgereviews/kr07.pdf) (Accessed 10 December 2011). Winkle M, Crowe TK, Hendrix I (2012). Service dogs and people with physical disabilities partnerships: a

6

Editorial

systematic review. Occupational Therapy International, 19(1): DOI: 10.1002/oti.323 Woolf SH (2008). The meaning of translational research and why it matters. Journal of the American Medical Association, 299(2): 211–213. DOI: 10.1001/ jama.2007.26 World Federation of Occupational Therapists (2006). Evidence-based practice competency standards for occupational therapists: proposed outline for WFOT. (Available to WFOT members at http://www.wfot.org/ ResourceCentre.aspx)

Occup. Ther. Int. 19 (2012) 1–6 © 2012 John Wiley & Sons, Ltd.