The “MERIT” Trial of Medical Emergency Teams in Australia: An Analysis of Findings and Implications for the 100,000 Lives Campaign
100,000 Lives Campaign We invite you to join a Campaign to make health care safer and more effective — to ensure that hospitals achieve the best possible outcomes for all patients. IHI and other organizations that share our mission are convinced that a remarkably few proven interventions, implemented on a wide enough scale, can avoid 100,000 deaths between January 2005 and July 2006, and every year thereafter. Complete details on the web at http://www.ihi.org/IHI/Programs/Campaign/. This document is in the public domain and may be used and reprinted without permission provided appropriate reference is made to the Institute for Healthcare Improvement.
The “MERIT” Trial of Medical Emergency Teams in Australia: An Analysis of Findings and Implications for the 100,000 Lives Campaign In the June 18, 2005, issue of The Lancet, the pioneering Australian clinical investigators who first described in detail the use of “Medical Emergency Teams” (METs)—the intervention that we call “Rapid Response Teams” in the 100,000 Lives Campaign— report on a “cluster-randomised trial” of the MET system in 23 Australian hospitals. The study attempted to determine “whether the MET system could reduce the incidence of cardiac arrests, unplanned admissions to intensive units, and deaths among patients in general hospital wards.” (MERIT Study Investigators. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. The Lancet. 2005; 365:2091-2097.) This is a bold study, and a highly commendable example of an attempt at systematic, scientific evaluation of a complex socio-technical innovation. The study report is extremely well-written, and both the report and the MERIT investigators performed a cluster randomized trial, one of the strongest possible study designs to evaluate the impact of quality improvement interventions at the hospital level. Their study design was robust, and their analytic plan was sound and sophisticated. They considered and adjusted for correlations in patient-level data and inter-institutional variability, and they appropriately chose to perform an “intent-to-treat” analysis for their pre-specified primary outcome (cardiac arrest, unplanned ICU admissions, and unexpected death). The study report is extremely well-written and is refreshingly candid about potential limitations of the work. This openness of the study team allows for significant lessons to be learned from its ongoing work. The researchers summarize their main findings as follows: “The MET system greatly increases emergency team calling, but does not substantially affect the incidence of cardiac arrest, unplanned ICU admissions, or unexpected death.” Both prior to and after this publication, key IHI faculty and Campaign staff have been in frequent communication with the MERIT study investigators to understand what they have learned, what their current interpretations are, and what the implications of their study may be for the 100,000 Lives Campaign. The following summarizes some of our findings to date. MERIT STUDY DESIGN: • The study randomized 23 hospitals into two groups: 12 received training in the MET process and were asked to implement the program; 11 did not receive training and were asked to delay introduction of a MET system during the study period. • A baseline period of two months preceded a training and implementation period of four months, followed by a study period of six months.
2
• Outcome variables assessed were (a) cardiac arrests (i.e., no palpable pulse) without a prior “do-not-resuscitate” order, (b) unplanned ICU admissions, and (c) “unexpected” deaths (i.e., deaths without a prior DNR order). In addition, (d) a “composite” score was calculated as the sum of these three events. For all four events, (a) - (d), the actual rate used was “events per 1000 admissions” and the population studied included patients admitted to inpatient general wards (including the coronary care unit, but not the ICU, ORs, or EDs). • Of the 23 hospitals in the study, 17 were teaching hospitals. Bed sizes ranged from 182 to 457.
PRIMARY FINDINGS: • During the study period, MET hospitals were more likely than control hospitals to call emergency teams for help (3.1 calls per 1000 admissions vs. 8.7 calls; (p = 0.0001)). • Mean crude event rates (per 1000 admissions) were as follows during the baseline and study periods: MET HOSPITALS BASELINE
2.61
CONTROL HOSPITALS STUDY PERIOD 1.64
1.60
MET HOSPITALS STUDY PERIOD 1.31
5.29
4.96
4.68
4.19
1.61
1.18
1.65
1.06
7.07
5.86
6.58
5.31
CONTROL HOSPITALS BASELINE (A) Cardiac Arrest (B) Unplanned ICU Admission (C) Unexpected Death Any Primary Outcome (A, B, or C)
3
• In the overall study—among the 23 hospitals—unexpected deaths fell by over 30% between baseline and study periods, as shown in the graphs below. The graphs show three data points: the baseline period of 2 months; the 4month implementation period when 12 of the hospitals were prepared for introduction of the MET system; and the 6-month study period.
6
Control Hospitals
6
5
Unexpected Deaths
Unexpected Deaths
5
MET Hospitals
4
3
2
4
3
2
1
1
0
0
Baseline
Impl
Study
Baseline
Impl
Study
• In patients with primary outcome events, recording of physiologic variables included in specifications for triggering MET calls was erratic. For example, among patients with primary events, a record of vital signs—blood pressure, heart rate, and respiratory rate—within 15 minutes before the event was missing in 62% of cases. • MET calls did not occur reliably when appropriate physiological triggers occurred. For example, in patients who had unplanned ICU admission and whose appropriate physiological variables were measured more than 15 minutes before the event and met criteria for triggering a MET call, MET calls actually occurred only 9% of the time in control hospitals and only 30% of the time in the MET hospitals. • Although the number of primary outcome events was lower in MET than in control hospitals (i.e., -0.26 events per 1000 admissions, a difference of about 4% relative to the baseline rate of 6.82 events per 1000 admissions for the combined hospital groups), the 95% confidence interval for this difference was very wide (-2.449 to 1.921) relative to the baseline rate.
4
ANALYSES: • The substantial decrease in unexpected deaths in both control and treatment hospitals suggests that such a decrease can be achieved in a relatively short period of time. • The majority of patients did not have sufficient physiological monitoring in the general medical ward setting to allow for reliable triggering of MET responses. • Although the MET hospitals did increase their rate of emergency calls, such calls did not occur reliably, even when monitoring occurred and MET criteria were satisfied. Thus, the study suggests incomplete implementation of the MET system. • The empirical rate of MET calls in the study hospitals—8.7 per 1000 admissions—is below the rate of calls that IHI faculty believe they see in effective RRT implementation in our US hospitals, which is close to 15 calls per 1000 admissions. In addition, the MERIT study’s authors tell us that the original, effective MET implementation in two initially successful hospitals as reported in prior Australian literature led to call rates of 23 and 25 per 1000 admissions—three times the rate in the MET hospitals in this trial. • The investigators reported to us that most hospitals in Australia were already moving toward the MET process when the MERIT study began, making it difficult to find even 23 hospitals that had not yet implemented the MET system. This information also suggests that even the control hospitals may have begun before or during the study period to put some elements of the MET system and awareness into place. In addition, several of the experimental hospitals did not, in fact, implement the MET system during the study period. • Of the population of 23 hospitals, 17 were teaching hospitals, raising questions about the applicability of METs in academic vs. community hospital settings. Implementing METs in academic centers may take longer because of the complexity of the academic environment, and may require different strategies than in non-academic settings. • The study design was powered to have a 90% chance of detecting a 30% decline in the primary events, assuming a baseline rate of 30 events per 1000 admissions. The baseline event rate was far lower—6.82 events per 1000— and a recalculation shows that 90% power to detect a 30% decline in outcome events would have required 100 hospitals, rather than 23. In addition, the 100,000 Lives Campaign RRT target is a 10% reduction in deaths, a level of improvement that the MERIT study had very low power to detect.
5
IMPLICATIONS FOR THE 100,000 LIVES CAMPAIGN: • Despite the robust cluster randomized trial design of the MERIT study, the investigators ran into number of issues that tend to arise in this type of study, including (a) incomplete and inconsistent implementation of the MET process in experimental hospitals, (b) high proportion of teaching hospitals in the study, (c) possible unmeasured adoption of early *or rapid response system* response processes by the control hospitals, (d) low rates of monitoring of relevant physiological variables in general ward patients, and (e) extremely low experimental power to detect improvements of the magnitude sought in the Campaign (fewer events than expected, greater inter-institutional variability in event rates than anticipated). These issues, which are discussed candidly by the investigators, may have contributed to their inability to show a significant effect of the MET intervention. • Faced with these issues, it is appropriate—in fact, highly desirable—to explore secondary analyses that might deepen understanding of the study and develop hypotheses to inform future research. Such exploratory analyses are especially important given the most stunning finding of the MERIT study: the dramatic and statistically significant 30% reduction in mortality in intervention and control hospitals over the short study period (see Figure). Something happened in these hospitals that saved lives. Perhaps it was partial implementation of MET-like behaviors in the control hospitals; perhaps there were unmeasured interventions to reduce deaths. The Figure also demonstrates a profound reduction in the variance in mortality rates by the end of the study, precisely what would be expected with a successful quality improvement intervention. While some of this reduction in variance may be explained by the statistical issues, such as the longer study period compared to the baseline period, this is a potentially important finding. • Regardless of the reasons for the 30% reduction in mortality documented in the MERIT study, the implications are clear: dramatic reductions in mortality are achievable within the timeframe of the 100K Campaign. • Although the MERIT study in The Lancet was technically negative in its findings comparing treatment to control hospitals in an “intention to treat” analysis, limitations in both the conclusiveness and the generalizability of the MERIT study (that is, limitations in both internal and external validity) leave us undeterred in our confidence that the RRT process, well-implemented, can reduce in-hospital mortality. Prior published reports, though not of randomized trials, in addition to ongoing time-series analysis from participating hospitals, remain strongly supportive of the effectiveness of RRTs in at least some circumstances.
6
• The implementation issues embedded in the MERIT study emphasize the need for the 100,000 Lives Campaign hospitals to attend carefully to the training, support, encouragement, and monitoring of the RRT process, itself. Good target rates of RRT calls may be in the range of 15 to 25 per 1000 admissions to ensure proper uptake of the RRT process. • Monitoring of patient status and vital signs among general ward inpatients may be a vulnerable part of the RRT process. The RRT cannot be called if no one is noticing the patients’ changing status. Classical monitoring methods and schedules may not be up to the task of properly supporting and triggering the RRT capability. This is an important area for further investigation and development. • The dramatic reduction of unanticipated deaths over time in all 23 subject hospitals in the MERIT study suggests that changes of that magnitude are, one way or another, actually achievable among inpatients in at least some hospitals. Based on our experience and the literature as a whole, IHI believes that RRTs are an important and highly promising intervention for reducing deaths and other untoward events due to “failure to rescue”—that is, for the prompt identification and treatment of patients whose clinical condition is spiraling downwards.
7
