PREP-SETTM is an evidence-based 13-item inventory of fundamental peer review process characteristics. It takes only minutes to complete.
PREP-SETTM measures the extent to which an organization's clinical peer review program conforms to the QI Model. It contains the 11 factors, identified in the 2007 National Peer Review Practices Survey as published in the Journal of Clinical Outcomes Management, which predict higher perceived program impact on the quality and safety of care when controlling for other factors. The 2007 study showed that few if any programs are using reliable methods for measuring clinical performance through case review, even though there are models to do so. Therefore, 2 additional elements were added (Clinical Performance Measurement and Rating Scale reliability) for a total of 13 items. The 11 factors were validated in the 2009 ACPE Clinical Peer Review Outcomes Study published in the American Journal of Medical Quality.
PREP-SETTM was published in the September/October 2009 ACPE Physician Executive Journal:
From the 2009 study, the inter-rater reliability of PREP-SETTM was estimated using 27 paired ratings as 0.61 [0.31-0.80]. The reliability of the mean of 2 independent ratings of the Total Score is estimated as 0.75 [0.47-0.89]. This means that the average of 2 or more independent ratings should have good reliability for serial measurement.
As a practical matter, the larger the organization and the greater the variation in process, the more input should be obtained. The tool asks for a rating of what prevails in the institution. Where there are islands of QI model-style process in a sea of QA-style activity, there may be a tendency to inappropriately up-score. In such organizations, I have found it useful to walk through the tool as a group. This sharing of information can provide a useful platform from which to plan improvements.
I have recently modified the evaluation tool to mitigate widespread misunderstanding of clinical performance measurement. For this purpose, the literature demonstrates the superiority of a methodology termed structured implicit review. The Self-Evaluation Tool previously contained an item labeled Structured Review. Unfortunately, this terminology proved confusing because the vast majority of organizations use structured forms to record the findings from peer review, even though such forms do not meet the standard for structured implicit review. Consequently, I believe that it is best to abandon the term structured implicit review other than for academic purposes. The Program Evaluation Tool now includes items that assess in less ambiguous language whether the peer review process generates reliable clinical performance measures.
If you are considering making improvements, be sure to learn more about how our Peer Review Enhancement ProgramSM (PREPSM) and My PREPTM Toolkit can help.