Full article available as PDF: Request Reprint
Over the past decade, hospitals have come to embrace the Quality Improvement (QI) model that successfully transformed other industries. Through a variety of tools and techniques aimed at standardizing and improving processes, QI seeks to "shift the curve" towards higher performance (see Figure 1).
Peer review, however, remains an anachronism. It is still commonly conducted within the Quality Assurance (QA) framework. By today’s standards, "Quality Assurance" was misnamed: it hasn’t assured either quality or patient safety. Neither has it served to eliminate substandard care. The QA model presumes that problems in care delivery can be minimized through inspection of untoward events to identify the underlying failure of professional judgment. As a result, peer review has been conducted with the primary objective of determining whether substandard care occurred and, if so, who was responsible - to prevent "the bad apples from spoiling the barrel." This is tantamount to endorsing the false belief that simply trimming one tail of the distribution will substantially improve overall performance (see Figure 2). Thus, in many hospitals, peer review is rendered ineffective because it is threatening to individual careers and economic livelihood.
The other major problem with the QA model is that it confuses the evaluation of performance with the assessment of competence. Competence is an enduring quality that changes slowly over time in the absence of a major health event (e.g., stroke, substance abuse). The Joint Commission standards calling for Focused (initial) and Ongoing Professional Practice Evaluation (FPPE/OPPE), which became effective in January 2008, gave us a new set of buzzwords and may have inadvertently perpetuated this confusion. FPPE/OPPE is really about monitoring competence in the intervals between the mandated 2-year reappointment process. Peer review, which is fundamentally an assessment of clinical performance in a specific context, can provide important data for FPPE/OPPE, particularly in the aggregate.
If the QA model of peer review is fundamentally antithetical to our efforts to improve the overall quality of care, it is also wasteful. It captures only a small fraction of clinical performance data available to the review process. Moreover, it misses the opportunity to:
Given all these limitations and inefficiencies of the QA model, the potential return on investment for improvement in peer review process is substantial.
Figure 3 presents a high-level framework for clinical peer review activity. Any and all of these aspects can and should be studied for systematic improvement. Figure 4 elaborates on the key hospital processes and functions that are interdependent with peer review activity. These interfaces may also be targets for improvement efforts.
In my first national survey of peer review practices, Dr. Evan Benjamin and I applied this model to identify the key drivers of the perception that peer review has a significant ongoing impact on the quality and safety of care at the hospital. (1) The unexpected finding was that the factors we identified make intuitive sense when considered in a QI context. In other words, the more that peer review looks like QI instead of QA, the more effective it is felt to be. I subsequently translated the survey results into a 100-point, 13-item Peer Review Program Self-Evaluation Tool. (2) When applied to our original study population, the mean score was 45, suggesting huge opportunity for improvement across the country. No one scored at A-level, which may explain the dearth of examples of real success stories. The utility of the tool and the magnitude of the performance gap were subsequently validated in a different group of hospitals. (3) The follow up study also demonstrated that a well-designed peer review program can significantly influence objective measures of quality and safety. (4) In addition, it enabled a refinement of the Self-Evaluation Tool which is now available online: Clinical Peer Review Program Self-Evaluation Tool.
A few comments on these points may be useful.
Peer review needs to be well-connected to the organizational quality improvement function. As a practical matter, most review committees don’t have the resources or skills to manage the process improvement stuff that can be uncovered by diligent peer review. Conversely, most hospital QI programs face limitations in terms of the issues they can surface. For example, clinician to clinician issues, the subset of system issues that figured most prominently in our regression analysis, are hard to uncover except through peer review.
In the process of case review, the many programs record an overall quality “level” and some look at component factors, but only rarely are such ratings meaningfully quantified. The scientific literatures shows that clinical performance can be measured during case review by rating multiple elements of performance having specific clinician attribution using reliable scales. This generates at least 10-fold more clinical performance data with minimal additional effort. The added statistical power and scope of the data collected lowers the cost to assess variation.
Sanazaro and Worth have shown that as few as 5 cases can reliably compare performance among physicians when structured ratings are used. (5) Please refer to my whitepaper or PEJ publication on clinical performance measurement for a more detailed discussion. (6)
A few industry pundits have been promoting centralization and reduced case review volume, but have not published data on the outcome of such changes. Our national study found a minimum threshold for review volume among the predictive factors of belief in efficacy. Also, Graber showed that broadening the scope of peer review to include a search for process of care issues quadrupled the number of problems identified and greatly magnified the number of quality improvement projects initiated. (7) So throttling back may be counter-productive.
We can also look at the review volume question from the angle of waste. Since the QA model is only focused on making the standard of care decision in the peer review process, the return on the invested effort is small. Most of the time, deviation is not significant. This is because the commonly used generic screens have low specificity even for substandard care. (8) What has been thrown away is the opportunity to actually measure clinical performance and provide the feedback necessary to “shift the curve”.
The coda on the volume question is that, over time, we need to develop good data on what would be the high payback triggers for peer review in the framework of the QI model for peer review. The most promising strategy is to promote self-reporting of adverse events, near misses, and hazardous conditions. In organizations suffering from the burden of the QA model, the concept of self-reporting might seem ludicrous because no physician would voluntarily seek exposure to criticism and blame. Nevertheless, self-reporting has been nicely demonstrated in a healthcare setting. (9) Moreover, self-reporting has had a major impact on aviation safety over the past 20 years. It was facilitated by offering relative immunity from disciplinary action. Today, the greatest leverage for comparable cultural change in healthcare comes from the Federal protections of the Patient Safety Act. By adopting the QI model and establishing a Patient Safety Organization relationship, any healthcare organization can readily promote self-reporting. (10)