Overconfidence as a Cause of Diagnostic Error in Medicine

https://doi.org/10.1016/j.amjmed.2008.01.001Get rights and content

Abstract

The great majority of medical diagnoses are made using automatic, efficient cognitive processes, and these diagnoses are correct most of the time. This analytic review concerns the exceptions: the times when these cognitive processes fail and the final diagnosis is missed or wrong. We argue that physicians in general underappreciate the likelihood that their diagnoses are wrong and that this tendency to overconfidence is related to both intrinsic and systemically reinforced factors. We present a comprehensive review of the available literature and current thinking related to these issues. The review covers the incidence and impact of diagnostic error, data on physician overconfidence as a contributing cause of errors, strategies to improve the accuracy of diagnostic decision making, and recommendations for future research.

Section snippets

Incidence and impact of diagnostic error

We reviewed the scientific literature with several questions in mind: (1) What is the extent of incorrect diagnosis? (2) What percentage of documented adverse events can be attributed to diagnostic errors and, conversely, how often do diagnostic errors lead to adverse events? (3) Has the rate of diagnostic errors decreased over time?

Physician overconfidence

“… what discourages autopsies is medicine's twenty-first century, tall-in-the-saddle confidence.”

“When someone dies, we already know why. We don't need an autopsy to find out. Or so I thought.”

—Atul Gawande83

“He who knows best knows how little he knows.”

attributed to Thomas Jefferson84

“Doctors think a lot of patients are cured who have simply quit in disgust.”

—attributed to Don Herold85

As Kirch and Schafii78 note, autopsies not only document the presence of diagnostic errors, they also provide

Strategies to improve the accuracy of diagnostic decision making

“Ignorance more frequently begets confidence than does knowledge.”

—Charles Darwin, 1871145

We believe that strategies to reduce misdiagnoses should focus on physician calibration, i.e., improving the match between the physician's self-assessment of errors and actual errors. Klein128 has shown that experts use their intuition on a routine basis, but rethink their strategies when that does not work. Physicians also rethink their diagnoses when it is obvious that they are wrong. In fact, it is in

Analysis of strategies to reduce overconfidence

The strategies suggested above, even if they are successful in addressing the problem of overconfidence or miscalibration, have limitations that must be acknowledged. One involves the trade-offs of time, cost, and accuracy. We can be more certain, but at a price.202 A second problem is unanticipated negative effects of the intervention.

Recommendations for future research

“Happy families are all alike; every unhappy family is unhappy in its own way.”

—Leo Tolstoy, Anna Karenina209

We are left with the challenge of trying to consider solutions based on our current understanding of the research on overconfidence and the strategies to overcome it. Studies show that experts seem to know what to do in a given situation and what they know works well most of the time. What this means is that diagnoses are correct most of the time. However, as advocated in the Institute

Conclusions

Diagnostic error exists at an appreciable rate, ranging from <5% in the perceptual specialties up to 15% in most other areas of medicine. In this review, we have examined the possibility that overconfidence contributes to diagnostic error. Our review of the literature leads us to 2 main conclusions.

Author disclosures

The authors report the following conflicts of interest with the sponsor of this supplement article or products discussed in this article:

Eta S. Berner, EdD, has no financial arrangement or affiliation with a corporate organization or manufacturer of a product discussed in this article.

Mark L. Graber, MD, has no financial arrangement or affiliation with a corporate organization or manufacturer of a product discussed in this article.

Acknowledgments

We are grateful to Paul Mongerson for encouragement and financial support of this research. The authors also appreciate the insightful comments of Arthur S. Elstein, PhD, on an earlier draft of this manuscript. We also appreciate the assistance of Muzna Mirza, MBBS, MSHI, Grace Garey, and Mary Lou Glazer in compiling the bibliography.

References (211)

  • P. Mongerson

    A patient's perspective of medical informatics

    J Am Med Inform Assoc

    (1995)
  • R.J. Blendon et al.

    Views of practicing physicians and the public on medical errors

    N Engl J Med

    (2002)
  • YouGov survey of medical misdiagnosis. Isabel Healthcare–Clinical Decision Support System, 2005. Available at:...
  • T.E. Burroughs et al.

    Patient concerns about medical errors in emergency departments

    Acad Emerg Med

    (2005)
  • W.M. Tierney

    Adverse outpatient drug events—a problem and an opportunity

    N Engl J Med

    (2003)
  • G.R. Norman et al.

    Expertise in visual diagnosis: a review of the literature

    Acad Med

    (1992)
  • Foucar E, Foucar MK. Medical error. In: Foucar MK, ed. Bone Marrow Patholog, 2nd ed. Chicago: ASCP Press,...
  • J.D. Kronz et al.

    Mandatory second opinion surgical pathology at a large referral hospital

    Cancer

    (1999)
  • L. Berlin et al.

    Perceptual errors and negligence

    Am J Radiol

    (1998)
  • Kripalani S, Williams MV, Rask K. Reducing errors in the interpretation of plain radiographs and computed tomography...
  • G. Neale et al.

    Exploring the causes of adverse events in NHS hospital practice

    J R Soc Med

    (2001)
  • P.M. O'Connor et al.

    Unnecessary delays in accident and emergency departments: do medical and surgical senior house officers need to vet admissions?

    Acad Emerg Med

    (1995)
  • M. Chellis et al.

    Evaluation of missed diagnoses for patients admitted from the emergency department

    Acad Emerg Med

    (2001)
  • A.S. Elstein

    Clinical reasoning in medicine

  • I. Kedar et al.

    Internet based consultations to transfer knowledge for patients requiring specialised care: retrospective case review

    BMJ

    (2003)
  • K.S. McGinnis et al.

    Pathology review of cases presenting to a multidisciplinary pigmented lesion clinic

    Arch Dermatol

    (2002)
  • R.J. Zarbo et al.

    Error detection in anatomic pathology

    Arch Pathol Lab Med

    (2005)
  • J.E. Tomaszewski et al.

    Consensus conference on second opinions in diagnostic anatomic pathology: who, what, and when

    Am J Clin Pathol

    (2000)
  • M. Harris et al.

    Sarcomas in north west EnglandI. Histopathological peer review

    Br J Cancer

    (1991)
  • J. Kim et al.

    Pathology Panel for Lymphoma Clinical Studies: a comprehensive analysis of cases accumulated since its inception

    J Natl Cancer Inst

    (1982)
  • P. Goddard et al.

    Error in radiology

    Br J Radiol

    (2001)
  • L. Berlin

    Defending the ”missed” radiographic diagnosis

    Am J Radiol

    (2001)
  • J.A. Espinosa et al.

    Reducing errors made by emergency physicians in interpreting radiographs: longitudinal study

    BMJ

    (2000)
  • R.L. Arenson

    The wet read. AHRQ [Agency for Heathcare Research and Quality] Web M&M, March 2006

  • C.A. Beam et al.

    Variability in the interpretation of screening mammograms by US radiologists: findings from a national sample

    Arch Intern Med

    (1996)
  • A.S. Majid et al.

    Missed breast carcinoma: pitfalls and pearls

    Radiographics

    (2003)
  • W.H. Goodson et al.

    Causes of physician delay in the diagnosis of breast cancer

    Arch Intern Med

    (2002)
  • R. Smith-Bindman et al.

    Comparison of screening mammography in the United States and the United Kingdom

    JAMA

    (2003)
  • G.D. Schiff et al.

    Diagnosing diagnosis errors: lessons from a multi-institutional collaborative project

    Advances in Patient Safety: From Research to Implementation

    (2007)
  • Shojania K, Burton E, McDonald K, et al. The autopsy as an outcome and performance measure: evidence report/technology...
  • L.A. Pidenda et al.

    Clinical suspicion of fatal pulmonary embolism

    Chest

    (2001)
  • Y. von Kodolitsch et al.

    Clinical prediction of acute aortic dissection

    Arch Intern Med

    (2000)
  • J.A. Edlow

    Diagnosis of subarachnoid hemorrhage

    Neurocrit Care

    (2005)
  • E.C. Burton et al.

    Autopsy diagnoses of malignant neoplasms: how often are clinical diagnoses incorrect?

    JAMA

    (1998)
  • R.H. Perlis

    Misdiagnosis of bipolar disorder

    Am J Manag Care

    (2005)
  • L. Graff et al.

    False-negative and false-positive errors in abdominal pain evaluation: failure to diagnose acute appendicitis and unneccessary surgery

    Acad Emerg Med

    (2000)
  • S.S. Raab et al.

    Clinical impact and frequency of anatomic pathology errors in cancer diagnoses

    Cancer

    (2005)
  • S. Gorter et al.

    Psoriatic arthritis: performance of rheumatologists in daily practice

    Ann Rheum Dis

    (2002)
  • S.S. Arnon et al.

    Human botulism immune globulin for the treatment of infant botulism

    N Engl J Med

    (2006)
  • D. Edelman

    Outpatient diagnostic errors: unrecognized hyperglycemia

    Eff Clin Pract

    (2002)
  • Cited by (700)

    View all citing articles on Scopus

    This research was supported through the Paul Mongerson Foundation within the Raymond James Charitable Endowment Fund (ESB) and the National Patient Safety Foundation (MLG).

    Statement of author disclosures: Please see the Author Disclosures section at the end of this article.

    View full text