Blackwell Publishing Ltd.Oxford, UK and Malden, USAIJTDInternational Journal of Training and Development1360-3736Blackwell Publishing Ltd. 2005200594256270ArticlesInstructor quality affecting EMT preparedness
International Journal of Training and Development 9:4 ISSN 1360-3736
Instructor quality affecting emergency medical technician (EMT) preparedness: a LEADS project* Darlene F. Russ-Eft, Philip D. Dickison and Roger Levine This represents one of a series of studies of the Longitudinal Emergency Medical Technician Attributes and Demographics Study (LEADS) being undertaken by the National Registry of Emergency Medical Technicians and the National Highway Traffic Safety Administration (NHTSA). This secondary analysis of the LEADS database, which provides a representative sampling of EMTs throughout the United States, examines the effects of instructor quality on the level of preparedness of emergency medical technicians (EMTs). Results showed significant differences, based on instructor quality, in the ratings on ten dimensions of EMT preparedness for both EMT Basics and EMT Paramedics. Implications for HRD practitioners, adult educators and researchers are discussed.
Emergency medical technicians (EMTs) comprise a critical segment of the medical care delivery system in countries throughout the world (e.g., Carney, 1999; Chapleau, 2001;
* An earlier version of this paper was presented at the Fourth Conference on HRD Research and Practice across Europe 2003, Toulouse, France. r Darlene F. Russ-Eft, Oregon State University, School of Education, 411 Education Hall, Corvallis, OR 97034, USA. Email:
[email protected]. Philip D. Dickison, National Registry of EMTs, Rocco V. Morando Building, 6610 Busch Blvd, P.O. Box 29233, Columbus, OH 43229, USA. Email:
[email protected]. Roger Levine, American Institutes for Research, 1070 Arastradero Road, Suite 200, Palo Alto, CA 94304-1334, USA. Email:
[email protected] © 2005 The Authors. Journal Compilation © 2005 Blackwell Publishing Ltd., 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main St., Malden, MA 02148, USA.
256 International Journal of Training and Development
Hay, 2000; Suserud et al., 1998; Weller, 2000). In the United States in particular, there are at least 17,000 Emergency Medical Services (EMS) agencies, approximately 815,000 EMTs, and more than 114 million annual emergency room visits (American College of Emergency Physicians, 2005), and these numbers have continued to grow since 1997. Cady and Scott (1995) reported that in 1995 approximately 28 million patients were treated and transported by EMTs. In most accidents and sudden illnesses, these EMTs are the first medical professionals providing medical treatment, and the quality of the care given at this initial stage will have an impact on the patient’s ultimate medical outcome (Cydulka et al., 1989). EMTs comprise a large segment of the US workforce, and successful performance of their health care duties requires adequate training, motivation and supervision. Accordingly, factors related to the training of these EMTs (for initial licensure, relicensure, certification, and re-certification) yield a direct impact on the quality of critical pre-hospital medical care received by millions of Americans. As a result, various aspects of EMT training have been examined. For example, Cannon et al. (1998) investigated the relationship between training hours for paramedics and their performance on the National Registry of Emergency Medical Technician’s Paramedic (NREMT-P) examination. Janing (2001) studied the link between teaching approaches and learning styles. More recently, studies focused on the use of simulations, games and multimedia (Stolk et al., 2001), the effects of video case studies (Janing & Sime, 2001), and computer-assisted learning (Jones & Cookson, 2002). Nevertheless, few studies have examined factors associated with the quality of this training or the quality of the instructors and the resulting preparedness of the EMTs to perform their work. Theoretical and empirical research focused on trainee populations has pointed to a variety of factors affecting such transfer of training (e.g., Baldwin & Ford, 1988; Holton, et al., 1997, 2000; Kupritz, 2002; Russ-Eft, 2002). Many of these factors focused on the environment surrounding training, such as the organizational climate and supervisory support, or on issues related to training design. Indeed, Rouillier and Goldstein’s study (1993) resulted in much of the recent emphasis on organizational issues related to the transfer climate. However, as stated by Stiehl and Bessey (1993, p. 33), ‘training and managing high performance is a learner-centered function. At a time when organizations are more customer driven, employees must be perceived as internal customers whose learning needs must be met by trainers and managers.’ Thus, the quality of the instructor affects the quality of the instruction and the quality of the materials being used, which in turn, can affect both the learning and the performance. An examination of the theoretical and empirical literature on trainee reactions to the instructor and the training shows mixed results. Kirkpatrick (1959a,b; 1960a,b; 1994) suggested that positive reactions lead to positive results in learning and behavior. Clement (1982) found a positive relationship between reactions and learning. In contrast, Bretz and Thompsett (1992), Dixon (1990), and Warr and Bunce (1995) reported that such relationships did not hold. A meta-analytic study by Alliger et al. (1997) found a positive relationship between utility reactions and performance but no significant relationships between affective reactions and learning or performance. More recently, however, Tan et al. (2003) found that negative reactions to technical training (i.e., brakes training to automotive technicians) resulted in higher learning and performance. This study contributes to the needed literature on the role of training and instructional quality, particularly as perceived by trainees, on EMT preparedness. Furthermore, it adds to the literature on factors affecting training transfer by examining the role of instructional quality on such transfer, defined as EMT preparedness.
Research problem The National Registry of Emergency Medical Technicians (NREMT) and the US Department of Transportation, National Highway Traffic Safety Administration (USDOT/NHTSA) have been concerned about a number of issues related to the work of EMTs, including the quality of the education and training being received. The Instructor © 2005 The Authors. Journal Compilation © 2005 Blackwell Publishing Ltd.
quality affecting EMT preparedness 257
NHTSA has been identified as an agency within the federal government to work with the State EMS lead agencies to create national standardized education programs for EMTs. These two agencies have collaborated on issues related to EMT education and training (e.g., Brown et al., 1999). Together the two agencies identified the need to research the EMS workforce as an aid to better understand the impact of future decisions from a regulatory and educational perspective. The present study focused on issues related to the licensure, training, and certification received by EMTs. The major questions revolved around issues related to the quality and costs of the instruction. We examined whether the quality of the instructors affected the quality of the materials and the level of EMT preparedness. H1 H2 H3 H4
There will be no differences in the quality of the materials (or at least the perception of that quality by the trainees) based on the rated quality of the instructors. There will be no differences in the level of preparedness of the trainees based on the rated quality of the instructors. There will be no differences in quality of the instructor based on whether or not the respondent paid a fee for the instruction. There will be no differences in the level of preparedness of the trainees based on whether or not the respondent paid a fee for the instruction.
Methods Background This Longitudinal EMT Attributes and Demographics Study (LEADS) is a joint venture between NREMT and NHTSA. The project is led by a team of researchers who have experience as State EMS Directors, State EMS Training Coordinators, EMS System Managers, Emergency Physicians, EMS Educators, and survey researchers. It also includes staff of the NREMT and NHTSA. More about the LEADS effort can be found in Brown et al. (2002). The current study on education and training represents a secondary analysis of this larger study. Sample The sample for the LEADS studies, including the present study, comes from the NREMT database. This database consists of nearly 185,000 EMTs who are nationally registered in the US. The sampling frame for the study is EMT-Basics and EMTParamedics who were currently registered at these levels by the NREMT, as of September 15, 1999. Separate frames were established for EMT-Basics and EMTParamedics. These frames were stratified by duration of continuous registration at each level (‘new’, referring to those registered at the level for less than one year, versus ‘old’, referring to those registered for one year or longer) and by race (‘white’, referring to those who self-identified as white, other, or did not self-identify, versus ‘minority’, referring to those who self-identified as Asian, Black, Hispanic, or Native American). Sample sizes were intended to maximize the efficiency of the sample for comparing different types of EMTs as well as for producing estimates of population parameters. The sizes of the strata and return rates appear in Table 1. Case weights were calculated for respondents in each stratum, reflecting the individual’s probability of selection. These case weights were adjusted, within strata, for non-response. Instrument development Since this is a longitudinal study, the survey instrument was designed to consist of two sections. The first section, the ‘core,’ consisted of 78 items and will continue to be readministered in each annual survey. The second section, the ‘snapshot,’ has a different focus in each mailing. The first snapshot survey consisted of 41 items, focusing on EMS education and training. This first snapshot will comprise the basis for much of 258 International Journal of Training and Development
© 2005 The Authors. Journal Compilation © 2005 Blackwell Publishing Ltd.
Table 1: Frames and sample characteristics Frame size
Sample size
Number of respondents
Response rate (%)
EMT-Basics White, New White, Old Minority, New Minority, Old Total
23,432 45,804 2,093 2,799 74,128
1,050 1,050 551 551 3,202
354 296 129 101 880
33.7 28.2 23.4 18.3 27.5
EMT-Paramedics White, New White, Old Minority, New Minority, Old Total
6,376 27,828 646 1,075 35,925
840 840 442 440 2,562
302 341 137 130 910
36.0 40.6 31.0 29.5 35.5
the current study. Copies of the survey are available on-line (http://www.nremt.org/ about/lead_survey.asp). Draft survey instruments were reviewed and modified, leading to the production of a pilot test instrument. This version was pilot tested on 42 EMT-Basics and EMTParamedics at eight different locations. After respondents completed the instrument, a debriefing protocol was administered. The protocol elicited extensive feedback about specific survey items as well as general feedback about the instrument and the proposed cooperation elicitation procedures. As a result of the pilot test, additional response categories were developed for several items, several items underwent minor revisions, and several items were deleted to reduce respondent burden. Analysis items and categorizations The items examined in the current study focused on the quality of the instructors, the quality of the materials, and the EMT’s self-assessed level of preparedness for performing various EMT tasks. Items on instructor quality asked about (1) technical knowledge, (2) practical knowledge, (3) teaching ability, (4) enthusiasm, (5) availability outside of class, and (6) professionalism. EMTs were asked to respond on a scale of ‘excellent’ = 4, ‘good’ = 3, ‘fair’ = 2, and ‘poor’ = 1. ‘High-quality instructors’ were defined as those receiving no ‘fair’ or ‘poor’ ratings, while ‘Low-quality instructors’ were defined as those receiving at least one ‘fair’ or ‘poor’ rating. In addition, the mean scores on each of the instructor quality items were combined to create a variable called ‘total instructor quality.’ Items on the quality of the materials asked about (1) the textbook, (2) audiovisual materials (videos, slides), and (3) course equipment (mannequins, splints, etc.). These three items used the same rating scale as that used for rating instructor quality. As for preparedness, EMTs were asked to rate the following: (1) trauma patient assessment, (2) medical patient assessment, (3) cardiac arrest management, (4) airway management, (5) spinal immobilization, (6) fracture management, (7) hemorrhage control, (8) childbirth, (9) pediatric patient management, and (10) patient transport. EMTs rated their preparedness as ‘very well prepared’ = 3, ‘adequately prepared’ = 2, or ‘poorly prepared’ = 1. The mean ratings on the ten items were combined to create a variable called ‘EMT preparedness.’ Data collection procedures Surveys were mailed on September 17, 1999 to the selected EMT-Basics and EMTParamedics. Included with the survey were a postage-paid return envelope and a letter Instructor © 2005 The Authors. Journal Compilation © 2005 Blackwell Publishing Ltd.
quality affecting EMT preparedness 259
that outlined the goals of the project and provided assurance of confidentiality. To motivate response, two round trip airline tickets to any location within the continental United States were offered to one randomly selected participant. After surveys were returned, they were scanned by an optical reader and used to create an analytic data file. Non-respondent survey Procedures. An abbreviated version of the survey, containing demographic, attitudinal, income, and educational items from the regular survey, was mailed to 500 EMT-Basics and 500 EMT-Paramedics survey non-respondents in March 2000. Surveys were received from 154 EMT-Basics (30.8 per cent) and 207 EMT-Paramedics (41.4 per cent), optically scanned, and used to create analytic data files. This enabled comparisons of respondents and non-respondents. Results. The responses of EMT-Basics and EMT-Paramedics who completed and returned the non-respondent survey were compared with the responses of demographically similar regular survey respondents. Discriminant analysis indicated differences in the response patterns of three types of EMT-Paramedic respondents and non-respondents (minority, new; minority old; white, new). These three types of EMTParamedics comprised less than a quarter (22.5 per cent) of the total number of EMTParamedics. Differences were associated with responses to an item about EMS income in the past 12 months: non-respondents reported significantly higher incomes than respondents. When this earnings item was excluded, there was little evidence that non-respondents differed from respondents on demographic, attitudinal, or education items (Brown et al., 2002).
Results – demographic data Brown et al. (2002) provided an initial reporting of some of the demographic characteristics of the sample. This section will present a brief overview of some of the results. Note that these results present the weighted data, reflecting the population. Age The average age of both the NREMT-Basics and the NREMT-Paramedics was 35 years. Race/ethnicity The racial/ethnic composition of the EMT-Basic and EMT-Paramedic workforces was similar. The vast majority of both EMT-Basics (90 per cent) and EMT-Paramedics (92 per cent) reported themselves to be White. Gender EMTs are predominantly males. Approximately 71 per cent of the EMT-Basics were male, and 69 per cent of the EMT-Paramedics were male. Level of education The Associate’s Degree was the highest level of education attained by 22 per cent of the NREMT-Basics and 30 per cent of the NREMT-Paramedics. About 21 per cent of both EMT-Basics and EMT-Paramedics reported their highest level of education attainment was a Bachelor’s degree. About 5 per cent of EMT-Basics and 6 per cent of EMTParamedics reported having received a graduate degree. 260 International Journal of Training and Development
© 2005 The Authors. Journal Compilation © 2005 Blackwell Publishing Ltd.
Level of practice The majority of EMT-Basics (81 per cent) and EMT-Paramedics (96 per cent) reported practicing at their level of national registration. Income Basic EMT’s reported a median income from all sources of $23,350 compared to EMTParamedics reporting $37,282. When asked to report income from EMS-related jobs, the EMT-Basic median income was $3,607 compared to the EMT-Paramedic reporting $32,460. This difference reflects the fact that a substantial proportion of EMT-Basics are volunteers, serving as EMTs for only a few hours per week. EMT and other work experience Randomly selected EMT-Basics from the general population of the NREMT database reported working a median of about two years as an EMT compared to about nine years reported by the EMT-Paramedics. Similar proportions of EMT-Basics (14 per cent) and EMT-Paramedics (15 per cent) reported having served as an EMS provider in more than one state during their careers. Communities served EMT-Basics (85 per cent) and EMT-Paramedics (67 per cent) do most of their EMS work in communities with populations under 150,000. EMT-Paramedics are more likely than EMT-Basics to be doing most of their work in larger communities. The percentage of EMT-Paramedics (11 per cent) who reported doing most of their work in cities with populations of 500,000 or more was nearly twice as large as the percentage of EMT-Basics (6 per cent). The percentage of EMT-Paramedics (12.5 per cent) who reported doing most of their work in mid-sized cities was twice as large as the percentage of EMT-Basics (5.5 per cent). Workload The median number of calls NREMT-Basics responds to in a typical week was reported to be 4, while the EMT-Paramedic reported responding to 18 calls during a typical week. The median number of hours NREMT-Basics perform the duties of an EMT during a typical week was reported to be 9, while the EMT-Paramedic reported performing the duties of an EMT for 45 hours during a typical week.
Results – EMT certification training data The maintenance of EMT licensure or certification typically requires continuing education or training. The following sections will provide some background information on the certification training courses being taken by the EMT respondents. Note that these data are weighted to represent the population. Level of training Among the NREMT-Basics, 93.2 per cent completed an EMT Basic course, 4.2 per cent completed an EMT Intermediate course, and 2.5 per cent completed an EMT Paramedic course. Among the NREMT-Paramedics, 99.5 per cent completed an EMT Paramedic course, with the remainder completing either a Basic course (0.3 per cent) or an Intermediate course (0.2 per cent). Instructor © 2005 The Authors. Journal Compilation © 2005 Blackwell Publishing Ltd.
quality affecting EMT preparedness 261
Frequency of meetings The courses taken by the NREMT-Basics tended to meet twice a week (61 per cent), three times a week (16 per cent), or once a week (12 per cent), with the remainder meeting four or more times a week (12 per cent). The courses for the NREMTParamedics tended to meet somewhat more frequently – twice a week (45 per cent), three times a week (29 per cent), five times a week (13 per cent), once a week (8 per cent), four times a week (4 per cent), or six or more times a week (1 per cent). Length of class The typical class length reported by the NREMT-Basics was three hours (32 per cent), four hours (44 per cent), or eight hours (12 per cent) with the remainder reporting shorter or longer class lengths. Similarly, the typical class length reported by the NREMT-Paramedics was three hours (18 per cent), four hours (31 per cent), or eight hours (30 per cent). Course length The NREMT-Basics reported that the course took the following amounts of time to complete: three months (21 per cent), four months (27 per cent), or five to six months (26 per cent). The remaining respondents reported shorter lengths (17 per cent) or longer lengths (10 per cent). The NREMT-Paramedics reported that the course took five to six months (12 per cent), seven to 12 months (52 per cent), 13 to 24 months (26 per cent), with 8 per cent reporting shorter periods and 1 per cent reporting longer periods. Course location For the NREMT-Basics, the most common certification training course location was at a community college (32 per cent), followed by fire station (18 per cent), other location (16 per cent), in hospital (12 per cent), at an EMS station (9 per cent), at a university (6 per cent), at an EMS training academy (4 per cent), at a community hall or a church (4 per cent). For the NREMT-Paramedics, the most common certification training course location was at a community college (38 per cent) or in a hospital (24 per cent), followed by a four-year college or university (11 per cent), other location (11 per cent), an EMS training academy (6 per cent), an EMS station (5 per cent), a fire station (3 per cent) and a community hall or church (1 per cent). Lead instructor certification The focus of the present study is on the quality of the instructor. It seems appropriate, therefore, to report on the instructor’s level of certification. Table 2 presents the reported level of certification of the lead instructor or the coordinator of the course. Costs Another variable of interest in this study involves the costs of the course. Table 3 presents the reported non-reimbursed, out-of-pocket costs paid by the respondents for their certification training course. Relationships between perceived instructor quality, perceived materials quality and EMT preparedness The results were examined separately for EMT-Basics and EMT-Paramedics, and the frequencies reported here represent the unweighted data. Among the EMT-Basics, 616 rated all areas of instructor quality as ‘good’ or ‘excellent’ while 180 rated some aspect 262 International Journal of Training and Development
© 2005 The Authors. Journal Compilation © 2005 Blackwell Publishing Ltd.
Table 2: Level of certification of the lead instructor or coordinator Lead instructor certification level
Respondent level EMT-Basic
EMT-Paramedic
67% 48% 70% 20% 2% 2% 14% 1%
43% 28% 91% 50% 3% 7% 11% 0.4%
EMT-Basic EMT-Intermediate EMT-Paramedic Nurse Physician Assistant Physician Other None
Note that multiple certifications can be indicated. Therefore, the totals will add to more than 100%.
Table 3: Costs of the course Reported course fee
Respondent level EMT-Basic
EMT-Paramedic
55% 7% 28% 8% 2% 0.4% 0.2%
34% 3% 7% 10% 21% 16% 9%
No fee $1 to $100 $101 to $500 $501 to $1,000 $1,001 to $2,500 $2,501 to $5,000 More than $5,000
Table 4: t-tests on quality of materials as rated by EMT-Basics based on instructor quality Material quality Textbook Audiovisual materials Course equipment
High quality instructors
Low quality instructors
Difference
t value
Degrees of freedom
Probability
3.42 3.13
3.24 2.73
0.183 0.403
3.15 4.98
785 784
