Do Web Sites Catch Residency Applicants?

968

BRIEF REPORTS

References 1. Saum CA, Inciardi JA. Rohypnol misuse in the United States. Subst Use Misuse. 1997; 32:723–32. 2. ElSohly MA, Feng S, Salamone SJ, Wu R. A sensitive GC-MS procedure for the analysis of flunitrazepam and its metabolites in urine. J Anal Toxicol. 1997; 21:335–40. 3. Salamone SJ, Honosoge S, Brenner C, et al. Flunitrazepam excretion patterns using the Abuscreen OnTrak and OnLine immunoassays: comparison with GCMS. J Anal Toxicol. 1997; 21:341–5. 4. Henretig F, Vassalluzo C, Osterhoudt K, et al. ‘‘Rave by Net’’: gamma-hydroxybutyrate (GHB) toxicity from kits sold to minors via the Internet [abstract]. J Toxicol Clin Toxicol. 1998; 36:503. 5. Anglin D, Spears KL, Hutson HR. Flunitrazepam and its involvement in

BRIEF REPORTS

date or acquaintance rape. Acad Emerg Med. 1997; 4:323–6. 6. Hallifax J. FDLE: roofies harder to find; after crackdown, crime labs getting fewer samples to test. Associated Press, 1997; Nov 19. 7. Seifert S. Substance use and sexual assault [abstract]. J Toxicol Clin Toxicol. 1997; 35:554. 8. Beck O, Lafolie P, Hjemdahl P, et al. Detection of benzodiazepine intake in therapeutic doses by immunoanalysis of urine: two techniques evaluated and modified for improved performance. Clin Chem. 1992; 38:271–5. 9. Browne R, Sloan D, Fahy S, et al. Urine screening for drugs. Br J Psychiatry. 1996; 169:669–70. 10. Calhoun SR, Wesson DR, Galloway GP, Smith DE. Abuse of flunitrazepam (Rohypnol) and other benzodiazepines in Austin and south Texas. J Psychoactive Drugs. 1996; 28:183–9.

Do Web Sites Catch Residency Applicants? Emergency medicine (EM) residency web sites have increasingly become a tool for applicants to obtain information about individual EM residency programs. Residency web sites generally serve as interactive brochures that provide information about curriculum, faculty and resident activities, salary and benefits, program location, department statistics, and specifics about each program’s unique opportunities. The Society for Academic Emergency Medicine web site contains the Catalog of Emergency Medicine Residencies, which provides information about the 118 EM programs with links to 61 program web sites.1 This paper evaluates the usefulness of EM residency web sites to residency applicants through the use of a survey distributed to those interviewing for a residency position in the UCSF–Fresno Emergency Medicine Residency Program. We examine the attitude of applicants toward web sites in relationship to ranking preference, need for printed residency brochures, and quality of information presented. Finally, suggestions are provided for improving and optimizing new and existing residency web sites for use by future applicants.

METHODS Study Design. A survey was dis-

tributed to EM residency applicants regarding their experience with EM residency program web sites. (Table 1 lists the survey questions.) Because of the survey method and voluntary participation, this study was considered exempt from informed consent.

Survey Population. The study population consisted of applicants interviewing for a resident position in EM at the UCSF–Fresno Emergency Medicine Residency Program. Applicants who participated in EM medical student electives at University Medical Center of Fresno were excluded.

Survey Administration. A survey was handed to each applicant at the end of the interview day during the November 12, 1997, to February 6, 1998, interview season. The survey responses were kept anonymous and collected by the department secretaries. No data were analyzed until after the residency ranking list was submitted.

Data Analysis. Data from completed forms were entered into a spreadsheet (Excel 97, Microsoft, Redmond, WA) and responses were tabulated and reported as percentages.

RESULTS The UCSF–Fresno Emergency Medicine Residency Program interviewed 79 applicants for six PGY2– 4 EM resident positions. Fifteen of the 79 applicants interviewed participated in EM medical student electives at University Medical Center and were excluded from the study. Four of the remaining 64 applicants interviewed prior to onset of the study. Survey forms were distributed to 60 interviewed applicants eligible for the study. A total of 60 completed surveys were returned. The majority of applicants (71%) believe that applicants use EM web sites in deciding where to apply (Table 1). Fourteen percent stated that they would rank a program with a web site in the match higher than a program without a web site. Half of the applicants believed that if a residency had what they nonspecifically defined as a ‘‘complete’’ web site, an additional brochure would not be needed. Forty-four percent of the applicants visiting EM residency web sites were able to access the sites on every attempt, while 30% were not able to access the sites better than 80% of the time. More than half of the applicants (55%) believed that 20% or more of the EM residency web sites loaded too slowly on their computer to be helpful. Two-thirds of the applicants primarily viewed the web sites from a home computer, while the remaining one-third used a school computer. Three-fourths of the applicants used the Windows Operating System and one-fourth used Macintosh. Twentyfour percent of all applicants used modems of 33.3 Kbps or less, with 8% of all applicants using 14.4 Kbps (Table 2). The majority (89%) of the applicants used Netscape as their web browser, with only 11% using Internet Explorer. Of the applicants who knew which version of the browser that they used (48%), approximately half (48%) used the most current version. Eighty-two percent of the applicants had visited our web site (www.ucsfresno.edu/em). Ninetyeight percent agreed that it was helpful in teaching them about the program, with the majority (67%) strongly supporting this statement.

ACADEMIC EMERGENCY MEDICINE • September 1999, Volume 6, Number 9

TABLE 1. Survey Questions Have you seen the UCSF–Fresno Emergency Medicine Residency Web Site (www.ucsfresno.edu/em)? (n = 60; 100% of total) Yes 49 (82%) No 11 (18%) Was our web site helpful in your learning about the UCSF–Fresno Emergency Medicine Residency? (n = 49; 82% of total) Strongly agree 33 (67%) Agree 15 (31%) Disagree 1 (2%) Strongly disagree 0 Do applicants use emergency medicine web sites in deciding where to apply? (n = 58; 97% of total) Strongly agree 9 (16%) Agree 32 (55%) Disagree 15 (26%) Strongly disagree 2 (3%) If an emergency medicine residency has a web site, would you rank that program in the match higher than a program without a web site? (n = 60, 100% of total) Strong agree 1 (1%) Agree 13 (13%) Disagree 36 (36%) Strongly disagree 10 (10%) If a residency has a ‘‘complete’’ web site, do you feel that an additional printed brochure is helpful? (n = 60; 100% of total) Yes 30 (50%) No 30 (50%) What percentage of the time were you not able to access other emergency medicine residency web sites? (n = 50; 83% of total) Always accessible 22 (44%) 1 to 10% 16 (32%) 20% or more 11 (22%) 50% or more 4 (8%)

969 Barbosa of the University of Illinois at Chicago (UIC) cited that it cost $7.33 to print and $1.47 to mail each individual 20-page residency brochure.3 The projected cost for sending 1,000 brochures was approximately $9,000. UIC subsequently decided to refer residency candidates to their web site and provide printed brochures only to those applicants without Internet access. Half of the applicants questioned in our survey stated that no additional printed brochure was needed if a residency had what the applicant nonspecifically defined as a ‘‘complete’’ web site. This leads one to wonder why the other half prefer an additional printed brochure. There are likely a number of reasons. First, the use of the Internet by applicants as an information providing tool is still in its infancy. However, it is possible that as students become increasingly familiar with the Internet, it will become their informationgathering tool of choice. Second, a printed brochure is tangible and portable. Several EM web sites are now providing applicants the ability to download a brochure off of their web sites, which they can print into a tangible format using their own printers. Finally, at the time of the study only half of EM programs had web sites, and those that did varied tremendously in the amount of program information that they provided. Currently, UCSF–Fresno re-

What percent of all of the emergency medicine web sites loaded too slowly on your computer to be efficiently helpful? (n = 53; 88% of total) None loaded too slowly 21 (40%) 1% to 10% 9 (17%) 20% or more 21 (38%) TABLE 2. What Software and 50% or more 9 (17%) Computer Hardware Do Applicants Use Most to Access Emergency Medicine commercial provider for approxi- Residency Web Sites? DISCUSSION

Medical students are increasingly using the Internet as a tool for research and education. The numbers of individuals with personal computers and Internet access has been steadily increasing as the affordability and availability of these tools increase. The production and maintenance of an EM residency web site likewise are becoming easier and less expensive. Residencies increasingly see the benefits of developing a departmental web site to inform and educate residency candidates about their programs. The computer on which a web site resides is known as a server. Server space may be rented from a

mately $20 per month. Many programs have server space provided by their university at no direct cost. A knowledge of coding web pages in HyperText Markup Language (HTML) is no longer required as new web page editors using WYSIWYG (‘‘Wissy Wig’’ = What You See Is What You Get) make the creation of a web site as simple as using a word processor for a cost of approximately $100. Yamamoto provides an excellent description of the anatomy of a web site.2 Residency programs are beginning to compare the cost of producing a residency brochure with the cost of maintaining a web site. In a recent letter to the editor, Hart and

Access mostly using: (n = 55; 92% of total) School computer 20 (35%) Home computer 35 (64%) Modem speed used from home computer (n = 29) 14.4 (Kbps 8 (28%) 28.8 Kbps 8 (28%) 33.3 Kbps 8 (28%) 56 Kbps 5 (17%) Browser used (n = 54; 90% of total) Netscape 48 (89%) Internet Explorer 6 (11%) Computer operating system (n = 55; 92% of total) Windows 42 (76%) Macintosh 13 (24%)

970

BRIEF REPORTS

TABLE 3. Total Page* Views June 1, 1997, to March 1, 1998 Web Page Home Page Fresno Application Curriculum Residents Links Faculty Hospital Medical Students

Page Views 4,093 695 933 609 560 450 406 376 316

*http://www.ucsfresno.edu/em/

fers applicants to our web site for information and distributes printed brochures only to those who specifically request it and to those who are interviewed. By using this strategy we decreased our yearly use of brochures from approximately 800 to 100. The UCSF–Fresno Emergency Medicine Residency Home Page received 4,093 nonstratified page views between June 1, 1997, and March 1, 1998 (Table 3). Most of our interviewed applicants had viewed our web site prior to the interview day. Candidates are referred to the site in numerous ways. The address is given by phone to those who call, and it is listed on many departmental mailings. There are links to the site from the SAEM Online Catalog of Emergency Medicine Residency Programs, through the ACEP home page, or through the use of web directories such as Yahoo (www.yahoo. com), or search engines such as Excite (www.excite.com) or Hotbot (www.hotbot.com). Each of the online search engines provides users with the ability to find a web site address, also known as the Uniform Resource Locator (URL). These online search engines use computer programs known as spiders that ‘‘crawl’’ the web going from link to link categorizing and ranking web sites. There is no easy way to consistently increase a web site’s ranking, thus making it visible on the first page of search results. However, web pages can contain statements known as ‘‘meta tags’’ which can provide descriptions and key words for search engine spiders to use for classification and ranking. Residency program web sites should insert meta tags into their web

BRIEF REPORTS

pages by following their web page editing program’s instructions or by learning this simple process from Search Engine Watch (www. searchenginewatch.com). Other ways to increase the visibility of program’s web site include placing the address on all departmental stationary, business cards, and e-mail. In addition, scientific publications, book chapters, and lecture handouts may include the program’s web site address. Providing the web site address of the university instead of the specific program site may force the reader to follow a series of links in search of the desired site. A casual review of web sites in the SAEM Emergency Medicine Residency Catalog revealed four different programs that had provided the university web address as the location of their home page. One site could be located only after navigating the correct combination of five separate university web pages. Programs should strive to provide web sites that download rapidly and are visible on the majority of applicants’ computers. The candidates we interviewed found that a large number of residency web sites loaded too slowly to be helpful. Many factors affect the speed at which a web page loads. After files are transferred from the residency program’s server computer to the candidate’s computer, they are reassembled. The active memory (RAM) and processor speed of the applicant’s computer influence the rate of reassembly. A slow computer with less RAM will take longer to view a web page than a fast computer with a large amount of RAM. The modem speed affects the speed at which data are transferred. One-third of the medical students we interviewed used university computers to connect to our web site. The majority of university computers connect to web sites using a T1 line, which allows the rapid transfer of data at speeds of 1544 Kbps. However, one-fourth of the candidates connected to the web sites from home at much slower speeds of 33 Kbps or less. One very attractive and creative ‘‘high-tech’’ EM web site with multiple spinning logos and morphing pictures took 30 seconds to completely load on our departmental computer using a speedy

T1 line, 1 minute and 21 seconds using a 56-Kbps modem, and 3 minutes and 41 seconds to load at 19.2 Kbps. It is likely that applicants using a 14.4-Kbps modem would find it difficult to use such a web site to attain information. Residency web sites should avoid using the newest features of the most recent browsers. A web site programmed using Dynamic HTML, which at the time of the interview season was supported only by the most recent version of Microsoft’s Internet Explorer, could prevent applicants using other browsers from correctly seeing the web site. Nine out of ten of those we interviewed used Netscape Communicator, with the majority using version 3 rather than the most recent version 4. In addition, the 24% of our applicants using a Macintosh would have difficulty viewing the moving images created using the programming language Java unless they had installed the most recent upgrade of the Macintosh operating system. Residency programs should test their web sites on slower computers, similar to that which some applicants may use. The web site should not alienate the applicant who attempts to view the web site using an older Macintosh with an earlier version of Netscape, connecting from home using a 33.3-Kbps modem. We addressed this issue by providing a ‘‘high-tech’’ and ‘‘low-tech’’ option for viewing. The ‘‘low-tech’’ version provides the same information as the ‘‘high-tech’’ version, without the advanced features that require newer, faster computers with the most current software to run efficiently. The information on a web site should be formatted differently than in the traditional printed brochure. A recent study by Morkes and Nielsen found that people rarely read web pages word by word, but instead they scan the page, picking out individual words and sentences.4 Web sites should incorporate the use of highlighted key words that allow the reader to easily find relevant information. A short question-and-answer format allows the applicant to quickly scan for questions that are pertinent to him or her. Bulleted lists and tables highlight desired information. A general rule is that a web page should contain half of the

971

ACADEMIC EMERGENCY MEDICINE • September 1999, Volume 6, Number 9

word count of conventional writing. Applicants made numerous individual suggestions as to what they would like to see on EM residency web sites (Table 4). A casual review of all current EM residency web sites shows that frequently many of these elements are absent. A survey and comparison of informational brochures distributed by 78 EM residency programs performed in 1992 shows an interesting parallel to our findings.5 It was found that half of the printed brochures were either deficient in information regarding salary and employment benefits, the local emergency medical services system, or introduction to the surrounding community. More than 80% of the brochures in that survey also found deficiencies in providing a map to the hospital, failing to mention the program’s ‘‘moonlighting’’ policy, and failing to give a starting date. The items that were recommended at that time for inclusion in the printed brochures were echoed by the interviewed applicants for inclusion on residency web sites. A majority of the interviewed applicants used web sites in deciding where to apply. One of seven applicants would rank a program with a web site higher than a program without a web site. Therefore, it is possible that EM residency programs without web sites are losing qualified candidates to programs with web sites. This would support the presence of a user-friendly informative web site as an inexpensive but valuable recruiting tool for EM residency programs.

LIMITATIONS AND FUTURE QUESTIONS The primary limitations of this survey are due to selection bias. UCSF– Fresno received 402 applications from a total applicant pool of 2,049, from which 79 were interviewed and 60 were surveyed. The results obtained from this survey for applicants interviewed at a single program may not correlate with a survey including the entire pool of applicants. The fact that we have a web site may have skewed our applicant population to those with more favorable opinions of web sites. The candidates were generally aware that the program had a web

TABLE 4. What Would Applicants Like to See More of on Emergency Medicine Residency Web Sites? Biographies and e-mail addresses of faculty, including research and personal interests (12) Biographies and e-mail addresses of residents, including research and personal interests (12) Curriculum information (4) Explanation of salary and benefits (4) Information and links about local town (4) All sites linked to SAEM’s residency page (4) Links to ‘‘ helpful’’ sites (3) Case presentations (3) Call-schedules for off-service rotations (2) Housing information (2) Travel directions to hospital (2) Alumni information (2) Patient population demographics (2) Statistics on applicants (2) Specialty board/inservice pass rates Statistics on procedures performed by individual residents Percentage of graduating residents pursuing academic/private career List of current publications Bulletin news about residency Bulletin news about important issues in emergency medicine Comments by current residents ‘‘Moonlighting’’ policy

site and could possibly surmise from the survey that the program believed the web site to be important. Though the survey was given anonymously, handed back to the survey collector anonymously with a stack of other papers, and was not a requirement in the interview process, it is possible that the applicants felt pressure to respond to the survey in favorable terms. In addition, applicants have geographic preferences in applying for residency positions, and the percentage of west coast programs with web sites is higher than east coast and midwest programs listed in the SAEM Emergency Medicine Residency Catalog. Therefore, applicants interviewing at west coast programs such as UCSF–Fresno may view web sites as being more important than those applicants interviewing elsewhere in the country. The applicants were not isolated from fellow applicants while completing the survey. While the survey was anonymous and answered independent of faculty and residents, the effects of comments of individual applicants during the completion of the survey may have influenced their answers. Finally, one of the authors (RCW) of this paper created and maintained the UCSF–Fresno Emergency Medicine Web Site at the

time of this study. This likely produced bias that skewed the interpretation of the data to favor the usefulness of residency web sites. Future studies should distribute the survey to a blinded sampling of all EM residency candidates independent of the specific programs to which they applied and should be done at a neutral time separate from the interview day. In addition, it would be interesting to quantify the total number of program web sites each applicant visited in relationship to the total number of programs he or she applied to and/or interviewed at.

CONCLUSIONS Emergency medicine residency web sites can provide easily accessible and useful information to potential residency candidates at a low cost. Web sites should load quickly and be written in way that allows individuals to scan for desired information using a variety of computer hardware and software. Residency candidates use web sites to learn about, apply to, and rank individual programs. — RICHARD C. WINTERS, MD, GREGORY W. HENDEY, MD, Department of Emergency Medicine, University of California, San Francisco at Fresno, Fresno, CA

972

BRIEF REPORTS

Key words. computer communication networks; internship and residency; education; Internet; web sites.

References 1. Society for Academic Emergency Medicine. Catalog of Emergency Medicine Residencies. Lansing, MI: SAEM, 1998 (http://www.saem.org/rescat/contents. htm). 2. Yamamoto LG. Creating a home page

CORRESPONDENCE

on the World Wide Web: an inexpensive means to promote medical education and physician recruitment. Am J Emerg Med. 1997; 15:393–9. 3. Hart RG, Barbosa J. Emergency medicine residency brochure on the Internet [letter]. Acad Emerg Med. 1996; 3:386. 4. Morkes J, Nielsen J. Concise, scannable, and objective: how to write for the web. 1998 (http://www.useit.com/papers/ webwriting/writing.html). 5. Holliman CJ. Survey and comparison of the informational brochures and application forms for US emergency medicine residency programs [letter]. Am J Emerg Med. 1992; 10:167–9.

CORRESPONDENCE International Introspection We applaud the insight of Taylor and Jelinek in their article comparing the Australasian and American emergency medicine (EM) training programs.1 Having just returned from a one-month elective in New South Wales, Australia, we, too, believe that looking to systems other than our own can contribute to the development of EM training programs. Such an analysis also enhances personal, professional, and societal development. Allow us to illustrate. One afternoon, a lone gunman stalked King’s Cross, Sydney, and shot an innocent victim. Despite that this is a city with a total population similar to that of Chicago, the significance of one gunman was insurmountable. In our memory, this violent firearm-related crime was the only one to occur in March. Restrictive firearm policies and an accepting society make such penetrating trauma an anomaly. Ironically, with a mere 700 firearm deaths per year in a country of 17 million, Australians still aggressively pursue gun control regulation.2 Interestingly, their medical societies share in this responsibility. Can we learn something from a country with its origin derived from a British penal colony? A 75-year-old man on coumadin, with an INR of 4, fell 12 hours ago. He denied loss of consciousness. An examination showed visible head contusions, no neurologic deficit, and a normal radiographic cervical spine series. An extensive debate ensued arguing for and against obtaining a head CT scan; the majority

was opposed. When cordially asked for an American opinion, our reflexive answer was ‘‘just do it’’! In retrospect, was our patented response appropriate or just a conditioned response to our medicolegal climate and availability of resources? Even as we incorporate evidence into medical decision making3 as in this case, will we ever truly practice the art of medicine without the art of law? A senior resident discharged a 23-year-old female after evaluating minor complaints related to a motor vehicle crash. An attending never saw the patient. While almost unheard of in the United States, this practice is commonplace in Australia, and may have several benefits. The patient is able to be treated and released earlier, the attending is relieved of an overwhelming workload, and the resident gains a sense of accomplishment and clinical responsibility. The role of an academic attending shifts appropriately from one of chart hoarder to one of teacher and supervisor. Should we reevaluate our belief that an attending’s signature on a chart ensures quality of care for all? Walking through waiting rooms in both inner-city and tertiary care EDs, we found them relatively quiet. The numbers of nonurgent and semiurgent patients were low. Why might this occur? The Australian medical system allows free and easy access to family physicians. For each citizen, a minimum level of basic medical care is a right and not a privilege. As a result, general physicians attend to

many of the ambulatory patients that would otherwise fill the ED waiting room. A second tier in the health care system also exists. Additional insurance may be purchased providing for more expedient elective procedures and care at private hospitals. This system operates at a lower cost per capita than the U.S. system.4 Furthermore, Australians have lower infant mortality rates and longer life expectancies.4 Currently, one in every six Americans has no health insurance, and even a larger number are inadequately insured.5 Can we deduce that the Australian health care delivery system better serves the public? Taylor and Jelinek have thoroughly outlined the Australasian and American EM training programs and raised interesting insights on improving each. As part of our training, we have actively sought experience in another medical system. On return to the United States, we believe the preceding actual scenarios best illustrate some of our perceived training and societal differences. We strongly advocate using elective time abroad to stimulate such introspection. We can now continue to reflect, and ponder pertinent solutions. We, as physicians, must continue to make an impact on these dilemmas. — SCOTT K. C. KING, MD, AND DEBORAH R. NATALE, MD, Department of Emergency Medicine, Residency Training Program, Rhode Island Hospital, Brown University School of Medicine, Providence, RI Key words. emergency medicine; international medicine; Australia; training programs; residency.

References 1. Taylor DMcD, Jelinek GA. A comparison of Australasian and United States emergency medicine training programs. Acad Emerg Med. 1999; 6:324–30. 2. Walpole BG. Gun control—a public health issue. Emerg Med (Australian). 1996; 8:203–4. 3. Garra G, Nashed AH, Capobianco L. Minor head trauma in anticoagulated patients. Acad Emerg Med. 1999; 6:121–4. 4. Cameron PA, Bradt DA, Ashby R. Emergency medicine in Australia. Ann Emerg Med. 1996; 28:342–6. 5. Kuttner R. The American health care system: health insurance coverage. N Engl J Med. 1999; 340:163–8.