U.S. Dental Schools: A Pilot Study Peggy Leong, D.M.D., M.B.A.; Jay Afrow, D.M.D., M.H.A.; Hans Peter Weber, D.M.D.; Howard Howell, D.D.S. Abstract: The objective of this study was to assess the patient safety culture among students, staff, and faculty in seven U.S. dental school clinics when compared to those from a similar study in twenty U.S. hospitals. A survey on patient safety culture developed by the Agency for Healthcare Research and Quality (AHRQ) was used to measure attitudes towards patient safety by anonymous faculty, students, and support staff members who work in the clinics of seven U.S. dental schools. This survey instru- ment was also administered to staff at twenty U.S. hospitals. In three of the twelve sections of the survey (Overall Perceptions of Safety, Management Support for Patient Safety, and Teamwork Across Units), dental school personnel responses rated above the hospital benchmark results. In Section 2 (Frequency of Adverse Events Reported) and Section 4 (Organizational Learning/ Continuous Improvement), average dental school responses were below those recorded for hospital personnel. The overall score from the twelve sections of the survey indicated that patient safety attitudes of dental school participants were higher than those of their hospital counterparts. Dr. Leong is Health Sciences Clinical Professor, Preventive and Restorative Dental Sciences Department, School of Dentistry, University of California, San Francisco; Dr. Afrow is Executive Director of Quality and Dental Director, Wentworth Douglass Hospital, New Hampshire; Dr. Weber is Raymond J. and Elva Pomfret Nagle Professor of Restorative Dentistry and Biomaterials Sciences and Chair of the Department of Restorative Dentistry and Biomaterials Sciences, Harvard School of Dental Medicine; and Dr. Howell is A. Lee Loomis Professor of Periodontology, Acting Head of the Department of Oral Medicine, Infection, and Immunity, and Dean for Dental Education, Harvard School of Dental Medicine. Direct correspondence and requests for reprints to Dr. Peggy Leong, Preventive and Restorative Dental Sciences Department, School of Dentistry, University of California, San Francisco, 707 Parnassus Ave., Box 0758, San Francisco, CA 94143; 415-514-0686 phone; 415-476-0858 fax; [email protected]. Key words: patient safety, survey, dental schools, hospital benchmarks Submitted for publication 6/6/06; accepted 12/3/07 P atient safety has been a concern of the United States health care system since the early twen- tieth century. The Flexner report 1 on medical education, published in 1910, and the Carnegie Foundation report on dental education in the United States and Canada, written by William J. Gies 2 and published in 1926, both spoke of the need for greater attention to patient safety. The Gies report, which brought about dental education as we know it today, made many recommendations, including calls for better cooperation between dentistry and medicine, expansion of dental research, and greater apprecia- tion by dental teachers of the biological and medical side of dentistry. 2 Although many areas of medical and dental care have progressed since then, the oc- currence of errors or failures continues to challenge health care providers. In 1999, the Institute of Medicines report To Err Is Human: Building a Safer Health System focused attention on the number and frequency of errors in inpatient hospitals. This report stated that errors cause between 44,000 and 98,000 deaths every year in American hospitals. The total national cost of preventable adverse events is estimated to be be- tween $17 billion and $29 billion per year, of which over half are health care costs. 3 Another report from Johns Hopkins Childrens Center and the Agency for Healthcare Research and Quality reviewed 5.7 million records of patients under nineteen years of age who were hospitalized in 2000; these records were from twenty-seven states. Of the 52,000 chil- dren identied by the researchers as being harmed by unsafe medical care during their hospital stay, 4,483 suffered a fatal injury. 4 As the complexity of care provided by the health care system increases, the chance of error or failure also increases. Al- though the magnitude and complexity of patient safety issues in dentistry differ from those found in hospitals, attitudes towards those safety issues have not been systematically explored in dental schools, and there is no published research that has quantied the type and number of adverse events that occur in dental care. In general, mainstream organizations working on patient safety problems in medicine have done little to study these issues and determine how they 432 Journal of Dental Education Volume 72, Number 4 may impact the delivery of dental care in the United States. It could be assumed that the morbidity, mor- tality, and nancial impact of human error in the dental environment would be signicantly less than that seen in medicine. This could be the reason why it is not considered a high priority in the health care environment. This lack of scrutiny, however, can give dental providers a false sense of security. Errors can occur in the dental environment, but with the lower morbidity and mortality rates in dentistry, the benet of preventing errors may be mea- sured by increased patient and employee satisfaction, reduced practice costs, improved practice reputation, and less stress on dental providers. The cost of fail- ures may not be only direct costs, but indirect ones also. The amount of lost business because of a poor reputation can be signicant though hard to quantify in a dental practice. In most organizations, the cost of preventing failures is signicantly less than the cost of correcting the error after it occurs. 5 The purpose of this study was to test the hy- pothesis that the patient safety culture in U.S. dental school clinics is less developed than in hospitals by utilizing a survey instrument developed to measure patient safety culture in U.S. hospitals. Materials and Methods Because of its impact on the U.S. dental work- force, U.S. dental schools teaching clinics were chosen as the patient care model for this study. A recruitment letter was sent in May 2005 to all U.S. dental schools listed on the American Dental Educa- tion Association website. The recruitment letter was addressed to the associate/assistant dean for clinical affairs to invite his or her voluntary participation in this study. Participation included completing a survey instrument and taking part in a one-day site visit by the principal investigator and co-investiga- tor. Eight dental schools agreed to participate in this study. Visits were carried out between August and November 2005. Dental schools are unique among U.S. health care educational sites. Unlike medical, nursing, and pharmacy schools, dental schools are the only health care educational sites that provide patient care within the schools. When the students from medi- cine, nursing, and pharmacy interact with patients, they provide the care in sites, such as hospitals, that have received some form of specic patient safety accreditation, such as The Joint Commission or the American Association of Ambulatory Health Centers. As expected, the current dental school accreditation process focuses primarily on the educational out- comes of the school with limited attention to patient safety issues. During the process, one dental school withdrew from participation, so there were seven schools that participated in both the survey and the site visit. The Internal Review Board of Harvard Medical School and the Harvard School of Dental Medicine approved the study. The survey instrument was developed by the U.S. Agency for Healthcare Research and Quality (AHRQ) and is entitled Hospital Survey on Patient Safety Culture. 6 The AHRQ-sponsored develop- ment of this survey is part of its goal of supporting a culture of safety and quality improvement in the nations health care system. This survey was utilized to measure the attitudes towards patient safety issues of students, staff, faculty, and administrators in seven U.S. dental schools. The survey consisted of forty-eight randomly sorted questions from twelve areas of concern. These twelve areas are shown in Table 1. In 2003, a pilot test of the survey was con- ducted, and completed surveys were received from over 1,400 staff from twenty different hospitals across the United States. Data from these pilot tests were analyzed, and average scores were calculated for each of the twelve dimensions of safety culture in order to allow health care organizations to make benchmarking comparisons against these pilot sites. For this study, slight modications were made to the survey instruments language to accommodate the categories of health care workers within a dental school. For example, the personnel descriptions were changed from physician and nurse to dentist and hygienist. The surveys were mailed to the dental schools and returned to the investigators at the beginning of the site visits. Each of the seven participating dental schools received fty copies of the survey with a cover sheet requesting demographic information about the school. Each survey instrument was com- pleted anonymously. The results were compiled into four categories: Dentists, Dental Students, Dental Support Staff, and all Dental Schools combined. The analysis utilized average percentage of positive responses to the fty-one survey questions. Based on the methodology utilized by the AHRQ, the percentage of positive responses dened as agreeing or strongly agreeing with a positive state- ment or disagreeing or strongly disagreeing with a April 2008 Journal of Dental Education 433 Table 1. AHRQs hospital survey on patient safety culture organized into twelve sections 1. Overall Perceptions of Safety A10: It is just by chance that more serious mistakes dont happen around here. A15: Patient safety is never sacriced to get more work done. A17: We have patient safety problems in this unit. A18: Our procedures and systems are good at preventing errors from happening. 2. Frequency of Adverse Events Reported D1: When a mistake is made but is caught and corrected before affecting the patient, how often is it reported? D2: When a mistake is made but has no potential to harm the patient, how often is it reported? D3: When a mistake is made that could harm the patient but does not, how often is this reported? 3. Supervisor Expectations and Actions Promoting Patient Safety B1: My supervisor/manager/instructor says a good word when he/she sees a job done according to established patient safety procedures. B2: My supervisor/manager/instructor seriously considers staff suggestions for improving patient safety. B3: Whenever pressure builds up, my supervisor/manager/instructor wants us to work faster, even if it means taking shortcuts. B4: My supervisor/manager/instructor overlooks patient safety problems that happen over and over. 4. Organizational Learning/Continuous Improvement A6: We are actively doing things to improve patient safety. A9: Mistakes have led to positive change. A13: After we make changes to improve patient safety, we evaluate their effectiveness. 5. Teamwork Within Units A1: People support one another in this unit. A3: When a lot of work needs to be done quickly, we work together as a team to get the work done. A4: In this unit, people treat each other with respect. A11: When one area in the unit gets really busy, others help out. 6. Communication Openness C2: People will freely speak up if they see something that may negatively affect patient care. C4: People feel free to question the decisions or actions of those with more authority. C6: People are afraid to ask questions when something does not seem right. 7. Feedback and Communication About Errors C1: We are given feedback about changes put into place based on event reports. C3: People are informed about errors that happen in this unit. C5: In this unit, we discuss ways to prevent errors from happening again. 8. Nonpunitive Response to Errors A8: Staff feel as if their mistakes are held against them. A12: When an event is reported, it feels as if the person is being written up, not the problem. A16: Staff worry that mistakes they make are kept in their personnel le. 9. Stafng A2: We have enough staff to handle our workload. A5: Staff in this unit work longer hours than is best for patient care. A7: We use more agency/temporary staff than is best for patient care. A14: We work in crisis mode, trying to do too much, too quickly. 10. Management Support for Patient Safety F1: Management in this facility provides a work climate that promotes patient safety. F8: The actions of management in this facility show that patient safety is a top priority. F9: Management in this facility seems interested in patient safety only after an adverse event happens. 11. Teamwork Across Units F2: Units in this facility do not coordinate well with each other. F4: There is good cooperation among units that need to work together. F6: It is often unpleasant to work with staff from other units in this facility. F10: Units in this facility work well together to provide the best care for patients. 12. Handoffs and Transitions of Patients F3: Things fall between the cracks when transferring patients from one unit to another. F5: Important patient care information is often lost during shift changes. F7: Problems often occur in the exchange of information across units in this facility. F11: Shift changes are problematic for patients in this facility. 434 Journal of Dental Education Volume 72, Number 4 negative statement for each group was determined. If the percentage of positive responses was more than 5 percent above the results of the hospital group, the results were considered above average. If the percentage of positive responses was more than 5 percent below the hospital group benchmark, it was considered below average. All results between 5 per- cent above or below the benchmark were considered to be average. 6
Data analysis of this study followed the method used by the Benchmark survey analysis available at the AHRQ website. 6
Results There were a total of 328 surveys completed out of 350 surveys sent to the seven participating U.S. dental schools. The sample included ninety- two dentists, 107 dental students, and 129 support staff. Demographics of the dental schools were as follows: The average class size was sixty-one students per class with a range of thirty to eighty-ve. Two schools were state-supported, two were partially publicly funded, and three were entirely privately funded. In all seven dental schools, the clinic director was a dentist. All of the schools had a person or committee des- ignated to oversee quality assurance for the patient care clinics. The age of the schools ranged from three years to over 100 years. Data derived from the 328 subjects at the seven dental schools were compared to the results from employees at twenty hospitals that participated in a 2003 pilot test of the AHRQs hospital survey on patient safety culture. The survey instrument items were divided into twelve sections for result tabulation, as shown in Table 1. In the actual survey instrument, these items were arranged in a different order to reduce the pos- sibility of the survey format leading the respondents towards preferred responses. The average responses to each section within the dental school community were rst analyzed in total as one group. They were then separated in order to look at each group (dentist, dental student, and dental staff) individually (Table 2). Sections 1 and 10 (Overall Perception of Safety and Management Support for Patient Safety) were rated above average for all three study groups. Section 11 (Teamwork Across Units) was rated higher than the benchmark for the total group of respondents. As shown in Table 2, the results in Section 2 (Frequency of Adverse Events Reported) and Section 4 (Organizational Learning/Continuous Improvement) showed that the average dental school responses were below average compared to the national benchmark. The remaining seven sections had results within ve percentage points of the benchmark results. When asked to give the overall grade for the re- spondents organization on patient safety, 77 percent of the subjects in this study graded their dental school as Very Good to Excellent (Table 3). The national benchmark was 62 percent. The response provided by dental students averaged 7 percent in the Poor to Failing grading compared to 2 percent to 3 percent in the other two groups of respondents. The national benchmark was approximately 8 percent. Discussion There were two sections in the survey where the dental school respondents gave less positive re- sponses than the benchmark group. The response to the Overall Patient Safety Rating was more positive than the benchmark response. Since there are both strengths and weaknesses in using a survey tool to measure the culture of an organization, more in- depth discussions on these issues with some policy implications follow. Frequency of Reporting Patient Safety Problems (Section 2 in Survey) All three dental groups (faculty, staff, and stu- dents) surveyed gave less positive responses to the three questions on the reporting of problems than the medical benchmark. There could be several reasons for the less positive responses including the lack of a user-friendly reporting system in dental school clinics and the lack of feedback to all three dental groups about the usefulness of incident reports and changes made to reduce errors as a result of timely reporting. This challenge has also been noted in the medi- cal community: Both doctors and nurses believe they should report most incidents, but nurses do so more frequently than doctors. To improve incident reporting, especially among doctors, clarication is April 2008 Journal of Dental Education 435 needed of which incidents should be reported, the process needs to be simplied, and feedback given to reporters. 7 In a collaborative hospital study, Evans et al. reported that common barriers to reporting incidents included time constraints, unsatisfactory processes, and deciencies in knowledge, cultural norms, inadequate feedback, beliefs about risk, and a perceived lack of value in the process. 8 Table 2. Comparison of responses among respondent groups percentage of responses that were positive Survey on Patient Safety: Composite Level Benchmarks: Dentists Dental Dental Responses Culture Survey Dimension Average % Positive Responses (n=92) Students Support from Seven in Twenty Hospitals (n=107) Staff U.S. Dental (n=1419) (n=129) Schools (n=328) 1. Overall Perceptions of Safety 56% 79% 63% 67% 68% (+) (+) (+) (+) 2. Frequency of Adverse Events Reported 52% 46% 37% 46% 42% (-) (-) (-) (-) 3. Supervisor Expectations and Actions 71% 78% 67% 75% 72% Promoting Patient Safety (+) (0) (0) (0) 4. Organizational Learning/Continuous 71% 64% 55% 58% 60% Improvement (-) (-) (-) (-) 5. Teamwork Within Units 74% 85% 74% 82% 78% (+) (0) (+) (0) 6. Communication Openness 61% 66% 42% 54% 58% (0) (-) (-) (0) 7. Feedback and Communication About Errors 52% 62% 38% 40% 49% (+) (-) (-) (0) 8. Nonpunitive Response to Errors 43% 53% 35% 49% 47% (+) (-) (+) (0) 9. Stafng 50% 62% 54% 50% 55% (+) (0) (0) (0) 10. Management Support for Patient Safety 60% 83% 66% 73% 67% (+) (+) (+) (+) 11. Teamwork Across Units 53% 69% 60% 53% 62% (+) (+) (0) (+) 12. Handoffs and Transitions of Patients 45% 48% 46% 52% 47% (0) (0) (+) (0) (+)=results above average for benchmark (0)=results average for benchmark (-)=results below average for benchmark If the percentage of positive responses was more than 5% above the results of the hospital group, the results were considered above average. If the percentage of positive responses was more than 5% below the hospital group benchmark, it was consid- ered below average. All results between 5% above or below the benchmark were considered to be average. Table 3. Comparison of patient safety overall grading by individuals Patient Safety Grade Average % Average % Faculty Dental Dental Response Response Dentists Students Support Across Twenty Across Seven Staff U.S. Hospitals U.S. Dental Schools A=Excellent 15% 25% 27% 24% 25% B=Very Good 47% 52% 58% 46% 55% C=Acceptable 30% 18% 13% 23% 17% D=Poor 8% 4% 2% 6% 2% F=Failing <1% 1% 0% 1% 1% Question was Please give your work area in this organization an overall grade on patient safety. 436 Journal of Dental Education Volume 72, Number 4 Dental schools involved in this study should develop policies to reduce the barriers to timely re- porting of safety issues and create a safe environment for such reporting. Proactive Activities (Section 4 in Survey) The dental school survey respondents rated dental schools lower than the medical benchmark in the area of proactive activities toward patient safety. Few of the sites visited had a process in place to summarize and trend patient safety incident data that would allow them to focus on preventive rather than reactive activities. Without the ability to notice posi- tive or negative trending of incident reports, any form of proactive efforts would be without direction. The use of a prospective root cause analysis or failure mode effects analysis (FMEA) originated in the world of industry over thirty years ago and has been adopted into the world of health care. FMEA is a systematic method of identifying and preventing process errors before they occur by evaluating a high risk, high volume, or problem-prone activity before a problem arises. 9 Medical organizations use it in an effort to reduce the possibility of errors. Its adop- tion into dental education could have only a positive impact on patient care and students. Overall Patient Safety Rating The dental school survey respondents rated the overall grade on their organizations patient safety higher than the benchmark. Reasons for this overall positive grade could include lower morbidity resulting from errors due to the nature of dental procedures; lack of knowledge of overall reported incidents; and lack of benchmarking information for respondents to evaluate their organizations performance in patient safety. Despite an overall positive safety rating, par- ticipating dental schools should seek to educate their staff, students, and faculty on the need for improved monitoring, better reporting, and trending of patient safety issues. This work will result in educators positive attitudes based on a true understanding of the safety conditions of their dental clinics and not from a lack of knowledge. Use of the Survey Method There are strengths and weaknesses in using a survey approach in this study. The strengths include the fact that surveys are relatively easy to adminis- ter, are relatively simple to score and code, and can determine the values and relations of variables and constructs. 10 Surveys can be generalized to other members of the population studied and often to other similar populations. They can be reused easily and provide an objective way of comparing responses over different groups, times, and places. Surveys can sometimes be used to predict behavior and can help conrm and quantify the ndings of qualitative research. 10 Weaknesses of the survey method include the fact that surveys are just a snapshot of behavior at one place and time. One must be careful about assuming they are valid in different contexts. Surveys do not provide a description of a situation that is as rich as a case study. They also do not provide evidence for causality between surveyed constructs that is as strong as a well-designed experiment. 10 Given some similarity of direct patient care between most dental school practices and ambulatory care areas of hospitals, we believe this survey instru- ment is a useful tool in the study. However, given the small number of dental schools participating in this study and the resulting small sample size, we realize the studys limited generalizability. Conclusion The data from this study demonstrate that there are areas of perceived weakness in the patient safety culture of the dental schools visited. By identifying these specic areas, it should allow the leadership of these participating organizations to focus their efforts on improving their patient safety culture. Since dental schools train future generations of dental clinicians in the world, we hope this data will help schools initiate a review of their current patient safety programs within their teaching clinics, as well as inspire additional research in best practices for patient safety that will lead to the development of new benchmarks for patient safety for the dental profession. Acknowledgments The authors thank Laura Ebenstein for her administrative assistance and data entry for this study. Two of the authors were funded by a grant from the Department of Restorative Dentistry and Biomaterials Sciences at the Harvard School of Dental Medicine. April 2008 Journal of Dental Education 437 REFERENCES 1. Flexner A. Medical education in the United States and Canada: from the Carnegie Foundation for the Advance- ment of Teaching, bulletin number four, 1910. Bull World Health Organ 2002;80(7):594602. 2. Orland FJ. William John Gies: his contribution to the advancement of dentistry. New York: The William Gies Foundation for the Advancement of Dentistry, 1992. 3. Committee on the Quality of Health Care in America, Institute of Medicine. To err is human: building a safer health system. Washington, DC: National Academy Press, 2000. 4. Miller MR, Zhan C. Pediatric patient safety in hospitals: a national picture in 2000. Pediatrics 2004;113:17416. 5. Katz J, Green E, eds. Evaluating your quality management program. St. Louis: Mosby, 1992. 6. Agency for Healthcare Research and Quality. Hospital survey on patient safety culture, 2005. Comparing your results: preliminary benchmarks. At: www.ahrq.gov/qual/ hospculture/prebenchmk.htm. Accessed: December 22, 2005. 7. Kingston MJ, Evans SM, Smith BJ, Berry JG. Attitudes of doctors and nurses towards incident reporting: a qualitative analysis. Med J Aust 2004;181(1):278. 8. Evans SM, Berry JG, Smith BJ, Esterman A, Selim P, OShaughnessy J, et al. Attitudes and barriers to incident reporting: a collaborative hospital study. Qual Saf Health Care 2006;15(1):3943. 9. Derosier J, Stalhandske E, Bagian JP, Nudell T. Using health care failure mode and effect analysis: the VA Na- tional Center for Patient Safetys prospective risk analysis system. Jt Comm J Qual Improv 2002;27(5):24867. 10. Survey instruments in IS. MISQ Discovery 1998. At: www.isworld.org. Accessed: December 22, 2005.