Introduction

Quality and patient safety are integral aspects of modern healthcare. In recent years, we have seen a significant increase in the number of meaningful quality improvement and patient safety efforts across the globe, owing largely to the recognition that: (1) medical errors are a potentially preventable source of patient morbidity and mortality; (2) faulty systems, rather than negligent individuals, are often responsible for safety mishaps and suboptimal quality of care; and (3) thoughtful efforts to improve quality of care and safeguard patient safety can simultaneously improve patient outcomes while decreasing healthcare costs, thus significantly improving the value of care provided.

Trauma and emergency general surgery (EGS) are high-hazard fields, making them fertile ground for quality improvement and patient safety interventions. The high burden of trauma and EGS disease, significant patient morbidity and mortality associated with injury and emergent operations, and substantial societal costs of trauma and emergency surgical care all make such quality improvement and safety efforts needed and highly valuable. For example, trauma constitutes one of the leading causes of death, disability, and healthcare expenditures in the USA. It is estimated that 400,000 US adults are admitted to a hospital and/or die in the hospital from major injury each year, comprising ~US$27 billion in annual health care spending [1]. Similarly, per the American Association for the Surgery of Trauma (AAST), EGS accounted for 27.6 million US hospital admissions from 2001 to 2010 (7.1% of all hospitalizations) during which time nearly 8 million EGS operations were performed [2•, 3]. EGS correlates with higher morbidity and mortality compared to non-EGS operations, and is in and of itself an independent risk factor for morbidity and mortality following surgery [4•, 5,6,7]. In this context, trauma and EGS were early to embrace quality and safety efforts, including the development of multiple quality and performance indicators. With proliferation of such metrics, there have also been several efforts to validate their use and thus standardize the assessment of quality and safety in trauma and EGS.

The purpose of this article is to provide an overview of quality and patient safety indicators in trauma and EGS in the USA and abroad. We start with a historical look at the quality improvement and patient safety movement. We provide an overview of existing quality and safety indicators with a specific focus on those relevant to trauma and EGS. We then review quality and safety benchmarking efforts in both the developed and developing world, describing controversies surrounding several of these indicators. We conclude with a summary of our findings and opportunities for growth and improvement in this important field.

Quality and Patient Safety Indicators in Historical Context

The modern healthcare quality and patient safety movement is rooted in the work of several twentieth century pioneers, including Walter Shewhart, Joseph Juran, and W. Edwards Deming. Through their avant-garde work, several foundational quality improvement concepts emerged, for example statistical process control, continuous quality improvement (CQI), and the Plan-Do-Study-Act (PDSA) cycle. Dr. Earnest Amory Codman, who practiced surgery in the early twentieth century, was perhaps the first surgeon to publicly recognize the importance of continuous quality improvement. His proposed “End-Result System” suggested that all patients must be followed long enough after their surgery to document their postoperative outcomes and that surgeons should continuously and transparently learn from their clinical successes and failures [8]. Dr. Codman co-founded the American College of Surgeons and helped to establish the morbidity and mortality conference.

In the last decade of the twentieth century, the quality improvement and patient safety movement gained momentum and focus [9]. The landmark Harvard Medical Practice Study, published in the New England Journal of Medicine in 1991, found that 3–4% of all hospitalized patients experienced an adverse event, of which a substantial fraction resulted from negligent or substandard care. Interestingly, nearly half of all inpatient adverse events were associated with a surgical procedure [10, 11]. These findings provided a call to action, motivating others to improve patient safety through process improvements and systematic benchmarking efforts.

The Agency for Healthcare Research and Quality (AHRQ, formerly the Agency for Health Care Policy and Research, AHCPR) responded to such calls for action, developing the “Healthcare Cost and Utilization Project Quality Indicators” in 1994. This was one of the first formal, national efforts to measure and track hospital quality using a set of consensus-based quality metrics [12]. The Healthcare Cost and Utilization Project (HCUP) quality indicators (QIs) were intended to be used as screening tools to identify potential quality gaps within and across different healthcare organizations. The initial iteration of AHRQ’s QIs included 33 metrics related to procedure utilization, condition-related admissions, complications, and mortality rates. AHRQ simultaneously created statistical packages that could be used by interested parties (e.g., participating hospitals and departments of public health) to mine the HCUP database for these specific QIs. Notably, many of AHRQ’s QIs apply to EGS but not to patients with traumatic injury.

Building upon these early quality-related efforts, the Institute of Medicine released two landmark reports [13, 14]: To Err is Human and Crossing the Quality Chasm in the early 2000s. The former suggested that between 44,000 and 98,000 Americans die each year as a result of preventable medical errors. These estimates were supported by similar findings in studies performed outside the USA [15]. The latter IOM report in particular provided a framework for system-wide improvement efforts, suggesting that such efforts should be directed towards six key system-wide aims: (i) safety, (ii) effectiveness, (iii) patient-centeredness, (iv) timeliness, (v) efficiency, and (vi) equity. The IOM’s publications are viewed by many as critical moments in the quality improvement and patient safety movement, providing both a catalyst and a roadmap for change. Following these reports, the field of quality and safety was elevated into the consciousness not just of healthcare administrators and policy makers, but also of everyday clinicians and patients.

As quality and safety received greater recognition, in-depth investigations into the quality of care being provided across a wide range of healthcare settings found significant gaps between recommendations and actual care. In another landmark study by McGlynn et al., the authors examined more than 400 quality indicators spanning 30 acute, chronic, and preventive care conditions, concluding that Americans received inappropriate care 45% of the time and that “these deficits … pose serious threats to the health and well-being of the US public” [16].

Subsequently, organized efforts began to address these quality and safety gaps. In a coordinated national effort, the Institute for Healthcare Improvement organized a campaign to save an estimated 100,000 lives during an 18-month period by enlisting over 3000 US hospitals to adhere to six evidence-based interventions (i.e., rapid response teams; bundles for acute myocardial infarction; medication reconciliation; and prevention of central line infections, surgical site infections, and ventilator-associated pneumonia) [17]. These and other collaborative efforts demonstrated the potential of organized quality-focused interventions to improve systems of care and save lives.

Surgical quality improvement has grown in parallel to many of these efforts. The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) is perhaps the most effective of such efforts today. NSQIP grew out of the Department of Veterans Affairs’ early efforts to benchmark quality through a national surgical outcomes database. This early work, led by Dr. Shukri Khuri and others, showed significant reductions in surgical morbidity and mortality in the 1990s simply by measuring and sharing risk-adjusted surgical outcomes across different VA hospitals [18]. Subsequent studies have confirmed similar improvements in surgical quality associated with participation in the ACS NSQIP program [19, 20]. Today, the ACS-NSQIP is a nationally validated, risk-adjusted, outcomes-based quality improvement program that collects, stores, tracks, and analyzes perioperative data from more than 600 participating hospitals [21].

When it comes to trauma and EGS, quality and patient safety efforts have developed alongside the efforts described above. In the next several sections we focus specifically on quality and patient safety indicators, with particular attention to those currently in use in trauma and EGS. We also discuss ongoing efforts to create emergency surgery scoring systems to facilitate risk assessment and benchmarking of care [4•, 22].

Quality and Patient Safety Indicators—the Basics

What Is a Quality Indicator?

A quality or patient safety indicator is a metric that can be used to assess the performance of a healthcare provider, group of providers, organization, or system at a given point in time or over a period of time. Several examples of existing QIs are presented below to provide a sense for the diversity of QIs in the literature. Development of a comprehensive list of all trauma and EGS-related QIs is beyond the scope of this review.

Aspects of a Good Quality Indicator

According to the National Quality Forum (NQF), a good quality indicator is one that satisfies the following criteria [23, 24]:

  1. 1.

    Importance (i.e., size of affected population, impact/severity of patient harm caused by quality deficit)

  2. 2.

    Scientific acceptability (i.e., reliability, validity)

  3. 3.

    Feasibility (i.e., ease of obtaining/calculating data)

  4. 4.

    Usability (i.e., ease of understanding results)

  5. 5.

    Minimal overlap with other quality indicators

Quality indicators must carry the above characteristics to fulfill their role in discerning differences in healthcare outcomes that are attributable to quality of care provided, rather than to patients’ underlying disease processes. While AHRQ’s QIs are the most directly linked to quality and patient safety, patient-reported measures—while more subjective—are also of significant value, as they represent quality of care as seen through a patient’s lens. A vast number of other metrics exist in varying forms, ranging from objective to subjective, administrative to clinical, and patient-specific to organization- and system-wide.

AHRQ’s Healthcare Cost and Utilization Project QIs

AHRQ QIs, as stated above, were initially intended to be used as screening tools to identify healthcare quality gaps. Since their introduction in the 1990s, the number of AHRQ QI’s has increased from 33 to 81 metrics spanning four categories: prevention quality indicators (PQI, n = 17) [25]; inpatient quality indicators (IQI, n = 25) [26]; patient safety indicators (PSI, n = 18) [27]; and pediatric quality indicators (PDI, n = 21) [28]. Several QIs in each category have been endorsed by the National Quality Forum [29]. The current version of AHRQ QIs was updated this year and uses International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes for data collection purposes. AHRQ provides interested parties with ready-to-use statistical packages to facilitate customized quality reporting, benchmarking, and improvement.

Much of the surgical research related to quality indicators has focused on PSIs, which were designed to specifically detect preventable inpatient complications and patient safety events and are often associated with a surgical admission or surgical care. Of note, many of AHRQ’s QIs apply to EGS but not to patients with traumatic injuries. Early studies evaluating the accuracy of PSIs reported high specificities but only moderate sensitivities (19–63%) and moderate positive predictive values (22–74%) [30,31,32,33,34,35,36,37]. In many of these studies, the positive predictive value of key surgical PSIs was found to vary between 44 and 89%, suggesting that they may adequately measure quality; still, others have raised concerns regarding the validity of PSIs. The statistical strength of several PSIs is thought to have improved over time as coding definitions were revised and adjusted to distinguish signal from noise. For example, the PSI #15 (accidental puncture/laceration) algorithm now has a positive predictive value of 85–91% for detecting true cases of accidental puncture or laceration that were not already present at the time of admission [31, 38]. A recent study reviewed the use of PSIs to date in assessing healthcare quality, including trends in PSI rates nationally and issues related to the statistical strength of PSIs [39].

Additional studies have specifically considered the application of AHRQ QIs in trauma care. In one study using coding data from over 50,000 trauma patients in Florida’s State Agency for Health Care Administration, the authors showed that PSIs could be used to benchmark the quality of care at different trauma centers using observed-to-expected (O/E) ratios, for example by calculating “failure to prevent complications” and “failure to rescue from death” from PSI #4, death rate among surgical inpatients with serious treatable conditions [40]. In that same study, the authors suggested that (1) PSI #9 (postoperative hemorrhage or hematoma) had the strongest association with trauma mortality; (2) failure to prevent complications may be a stronger marker of quality than failure to rescue; and (3) PSIs could be useful for benchmarking efforts by screening trauma centers with higher than expected mortality rates for causes of such findings. A prior investigation using the National Trauma Data Bank (NTDB) similarly identified failure to rescue (from death after major complications) as a primary driver of mortality differences between high vs. low-mortality trauma centers [41]. Clouding the picture further, a more recent study looked at complications attributable to trauma surgeons using AHRQ’s PSI algorithms at a single institution over a 1-year period and reported an overall positive predictive value of only 60% for the identified PSIs. While the authors of this last study concluded that PSIs may not reflect quality of care, they also acknowledged that the PSIs triggered meaningful local quality improvement efforts at their institution.

The variability in results related to AHRQ’s QIs, including those applicable to trauma and EGS have led to the suggestion that while PSIs may be appropriate screening tools, caution should be exercised when considering them for more consequential purposes, including public quality reporting and reimbursement [31, 39, 42].

Patient Experience

In contrast to the quality and safety indicators published by AHRQ and described above, “patient experience” has recently emerged as a valuable approach to measuring quality in health care settings. Many are now familiar with the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores, which are widely used and referenced within hospital quality meetings. HCAHPS is a survey instrument delivered to patients following discharge from the hospital. A randomly selected number of trauma patients, like others, may be asked to complete the HCAHPS survey following discharge. Their responses are analyzed and aggregated to determine ways in which the service they received excelled and where things could have been improved. A small number of studies have considered HCAHPS scores as indicators of quality of care for trauma patients. One report suggested that HCAHPS communication scores could be improved through simple measures, for example showing an orientation video to trauma and burn patients upon admission to an inpatient unit. Another study showed improvements in patient satisfaction on HCAHPS-adapted questions among orthopedic trauma patients who received a biosketch card of their caregivers, allowing them to get to know their caregivers [43]. These and other similar efforts illustrate the potential of patient experience and other “softer” or more subjective indicators to increase quality and value in the healthcare environment.

Table 1 presents a quick-reference to some common terms and acronyms related to quality and patient safety indicators.

Table 1 Common terms and acronyms related to quality and patient safety indicators
Full size table

Evolution of Quality and Patient Safety Indicators in Trauma

Trauma has been the focus of significant quality improvement efforts for many decades. The American College of Surgeons Committee on Trauma (COT) was founded in 1922 as the Committee on Fractures, and later renamed COT in 1950 [44]. In the 1960s and 1970s, after the National Academy of Sciences issued a report describing trauma as a national epidemic, the COT spearheaded numerous quality-related efforts. These included developing emergency medical services (EMS) systems around the country, setting standards for trauma centers, and establishing a National Trauma Data Bank (NTDB). The COT continues to set national standards for evaluation of trauma centers, although each state has ultimate authority over center designation. This work to develop state and national trauma systems has proven successful by many measures, with multiple studies confirming that injured patients who present to designated trauma centers are less likely to die compared with similarly injured patients who present to non-trauma centers [45]. The National Study of the Costs and Outcomes of Trauma, published in 2006 [45, 46], studied trauma center effectiveness across 69 level-I trauma and non-trauma centers in urban and suburban hospitals in 12 different states; they found significant reductions in in-hospital mortality (7.6 vs. 9.5%) and 1-year mortality (10.4 vs. 13.8%) for patients treated at level-I trauma centers [45]. The findings argued in favor of efforts to concentrate trauma care through regionalization. The topic of regionalization of trauma care is discussed in a different review in this same special edition.

Among the first standard quality documents developed in trauma is the “The Optimal Resources for the Care of the Injured” [47], published by the ACS-COT in 1979. This document was produced to define the metrics and standards for verification of trauma centers. However, the original metrics have evolved over the last several years, particularly after the Major Trauma Outcomes Study (MTOS) in the 1980s [48]. Around this time, the ACS-COT released a set of quality indicators, known as “audit filters” with the goal of promoting adherence to evidence-based practices. Examples of audit filters included [49]:

  • Trauma patient brought in by ambulance without ambulance report in medical record

  • Trauma patient presenting to ED with Glasgow Coma Scale (GCS) < 13, without subsequent head CT

  • Failure to obtain a definitive airway for trauma patient with altered mental status and GCS ≤ 8

  • Injured patient requiring emergent laparotomy that is delayed until > 2 h after ED arrival

Over the years, the ACS-COT audit filters have been revised multiple times and repeatedly studied. Investigations have shown mixed associations between the ACS-COT audit filters and outcomes of interest (e.g., trauma-related mortality, major complications, and length of stay) [49,50,51]. A large population-based study of these quality indicators from 2012 demonstrated significant associations between six audit filters and in-hospital mortality and/or major complications. For example, patients with admission GCS < 13 without head CT had a fourfold increased risk of mortality [49]. The authors convey optimism about the potential for these process measures to improve quality and to promote best practices, but they also acknowledge the need for large-scale analyses linking these and other process measures to outcomes of interest.

While these ACS-COT efforts focused primarily around establishing standards for trauma care, others have reviewed the literature on trauma quality indicators in depth. A comprehensive review on this topic was published by Stelfox et al. in 2010 [52•]. In this study, the authors identified 1572 quality indicators across 192 articles, classifying trauma QIs along several different domains, including:

  1. 1.

    Donabedian’s “Structure, Process, Outcome” framework, applied to the four phases of trauma care (prehospital, hospital, posthospital, secondary prevention) [53]. First described by Donabedian in the 1960s, “structure” metrics measure the availability of equipment and resources, and the environment in which care is delivered. “Process” metrics look at methods for delivering care, e.g., adherence to policies and guidelines. Finally, “outcomes” of care include morbidity, mortality, length of stay, costs, and other measurable results of care. Using this framework, the authors found that a clear majority of existing trauma QIs fit into the “hospital” and “prehospital” phases, with emphasis on process > outcome > structure measures. Only a small minority of existing trauma QIs (< 5%) occurred in the posthospital phases.

Table 2 presents examples of structure, process, and outcome measures for each of the four phases of trauma care.

Table 2 Trauma quality indicator classification based on Donabedian’s framework and stages of trauma care
Full size table
  1. 2.

    Characteristics of trauma QIs: most QIs identified in this study were developed locally (56.9%), whereas the remainder were developed either by the ACS-COT (26.5%) or other national/international organizations (16.6%). Very few descriptions of QIs included specific details regarding data collection methodologies. Furthermore, the data source for QIs was often unknown (40%), whereas a smaller fraction was generated from trauma registries (27%) or from medical record review (25%). The 10 most frequently identified QIs from these studies were:

  1. 1.

    Peer review of trauma deaths

  2. 2.

    The rate of inpatient mortality

  3. 3.

    The rate of inpatient complications

  4. 4.

    Length of time spent at the injury scene

  5. 5.

    Decreased GCS with delayed or absent head CT

  6. 6.

    Time from arrival in ED to operative treatment

  7. 7.

    Unscheduled return to OR within 24–48 h of the initial operation

  8. 8.

    The rate of missed injuries

  9. 9.

    Decreased GCS without secure airway

  10. 10.

    Length of ED stay

The authors of this comprehensive review highlight the paucity of clearly defined, evidence-based, broadly accepted QIs for evaluating the quality of trauma care, suggesting the need for a common taxonomy of trauma-related QIs, and for large-scale multi-institutional trials to generate stronger evidence in support of the validity of trauma QIs.

Additional Quality Indicators in Trauma

Within the last 5 years, several additional studies have identified quality indicators relevant to trauma. In 2013, Moore et al. [54] used Donabedian’s “Structure, Process, Outcome” framework to identify 14 non-fatal outcomes reported in 40 different studies describing quality within trauma hospitals. Adverse events (e.g., complications, missed injuries, reintubation, unplanned intensive care admission, unplanned return to OR) and resource use (e.g., hospital, ICU, ventilator, length of stay) were the most common non-fatal trauma outcomes reported in these studies. This same research group subsequently developed a consensus-based list of 25 complications, classically categorized by anatomic/physiologic system (i.e., pulmonary, cardiac, gastrointestinal, hematologic, infectious, genitourinary/renal, musculoskeletal, neurologic, vascular, psychiatric) in addition to death [55]. In a follow-up study, the authors evaluated the validity of their consensus-derived complications, and found that the presence of at least one of these complications was associated with a greater than twofold increase in mortality and hospital length of stay for trauma patients [56].

Trauma Quality Improvement Program

Following on the success of ACS-NSQIP, the trauma community developed its own much-needed risk-adjusted outcomes database, the Trauma Quality Improvement Project (TQIP). TQIP was launched in 2006 by the ACS-COT, and has now expanded to include over 450 centers [57,58,59]. The TQIP launch coincided with emerging research that showed significant variability in risk-adjusted mortality rates among different trauma centers [60]. TQIP measures, tracks, and reports validated, risk-adjusted, and outcomes-based data on trauma center performance compared to national benchmarks [57]; it is modeled after NSQIP but with specific modifications to the data collection process that reflect the unique clinical data needs of trauma patients. Data abstractors are trained and required to use the National Trauma Data Standard, which provides standardized definitions of terms to increase data quality [61]. The quality of the data is monitored on a regular basis. Annual reports are provided to participating centers with risk-adjusted O/E in-hospital mortality ratios, complication rates, and rates of adherence to several trauma-specific process measures and comparisons to peer organizations, allowing centers to compare their outcomes against national standards, and allowing for the development of best practice guidelines [58].

More recently, regional collaboratives have been created, such as the Michigan Trauma Quality Improvement Project [62, 63]. Participation in this collaborative requires attendance at quarterly meetings, an annual quality improvement project at each hospital, and regular reviews of each hospital’s outcomes report. Each report is scored and the score is tied to pay-for-participation and pay-for-performance for each hospital [62]. The results have been impressive, with a reported 40% reduction in serious complications over 5 years and a significant reduction in patient mortality and healthcare costs [62]. The value of such regional collaboratives is discussed in a separate paper in this same special edition.

Quality and Patient Safety Indicators in Emergency Surgery

The field of trauma surgery has evolved over the last two decades to merge with the management of EGS and critical care to what is now acknowledged as the field of acute care surgery. In recent years, research has shown that EGS patients carry unique perioperative risk factors [5] and have increased risk of postoperative morbidity and mortality when compared to their non-EGS counterparts [6, 7]. Large-scale national outcomes databases (e.g., NSQIP, UHC) traditionally include a variable for “emergency surgery” (e.g., Yes, No), but often do not benchmark these patients separately for quality improvement purposes. As the definition of EGS continued to evolve [2•] and the evidence for its uniqueness continues to grow, the need for a separate risk-adjusted database that allows for quality benchmarking of EGS has become apparent. Efforts are currently underway to create unique markers of EGS quality. Some hospitals are tracking EGS outcomes with institutional homemade EGS-specific registries [64]. Others are using established national databases to benchmark the quality of EGS care. In one study, the authors describe the initial steps needed to develop a validated list of quality indicators for EGS care [65]; they used a modified Delphi process to identify and validate 25 indicators of high-quality EGS care (n = 13 patient-level QIs, n = 12 hospital-level QIs). Initial investigations demonstrate large variations in compliance with these measures across institutions.

One of the key missing elements of EGS databases has been the absence of a validated scale that assesses severity (or acuity) of disease upon presentation (similar to the Injury Severity Scale (ISS) in trauma patients). Such a score would be very helpful for risk adjustment. To illustrate, two patients of similar age may present to the ED with the same diagnosis (e.g., perforated viscus) requiring the same emergent surgical procedure. And while they may both carry the same underlying comorbidities, they may in fact present with dramatically different acuity levels; one may appear well, while the other may have altered mental status, hemodynamic instability requiring vasopressors, and end-organ dysfunction. It would help to understand how these individuals’ perioperative risk profiles differ. To this end, the Emergency Surgery Score (ESS) was recently developed as a mortality risk calculator to help with these kinds of risk adjustment, and to facilitate preoperative counseling of EGS patients [4•]. This scoring system was derived and validated using the national NSQIP database; it correlates well with postoperative mortality and postoperative complications across several different types of emergency surgical procedures [4•, 22, 66–67].

Table 3 lists AHRQ’s current QIs as with assessment of potential relevance to trauma and emergency general surgery.

Table 3 Agency for Healthcare Research and Quality’s quality indicators (QIs) with description and assessment of candidacy for trauma/emergency general surgery
Full size table

Global Considerations

The surgical community is increasingly recognizing the need to focus on quality improvement and patient safety not only in the USA but globally, and especially in low- and middle-income countries (LMICs). International organizations (e.g., World Health Organization, WHO); non-governmental organizations (e.g., Life Box); private industry (e.g., General Electric which sponsored the Safe Surgery 2020 Project); and academic institutions and organizations (e.g., The Lancet Commission on Global Surgery, LCoGS) are each contributing to efforts to improve the quality and safety of surgical care worldwide.

One of the first projects to demonstrate improvement in surgical quality globally was the WHO’s Safe Surgery Saves Lives initiative. The initiative brought together experts from around the world to focus on surgical quality improvement, starting with establishment of best-practice guidelines and implementation of the Surgical Safety Checklist [68]. The implementation of the WHO checklist was a clear demonstration that improving the quality of care can be achieved using low-technology innovation. The WHO’s quality improvement efforts reached the world of trauma and EGS through the Global Initiative for Emergency and Essential Surgical Care (GIEESC) in LMICs.

In 2015, several events occurred that signaled the global surgical community’s full commitment to surgical quality improvement, especially in trauma and EGS. First, the World Health Assembly unanimously passed Resolution 68.15, “Strengthening emergency and essential surgical care and anesthesia as a component of universal health coverage” [69]. This resolution represented a committed effort by the world community to appeal to world leaders to make policy and organizational changes that prioritize safe emergency and essential surgery. Second, the World Bank published its third edition of the Disease Control Priorities (DCP3) with the first volume of the work dedicated entirely to surgery [70•]. This was the culmination of 2 years of efforts by leaders in surgery, anesthesia, and OBGYN from around the world. It included data from clinicians, economists, politicians, statisticians, ministers of health, epidemiologists, and thought leaders in the field. As part of these efforts, the LCoGS published many important findings including a list of six quality indicators that could be used by countries to estimate the strength of their surgical system, especially when it comes to the provision of emergency surgical care [70•] [Fig. 1]. The indicators are (1) The percentage of the population that lives within 2 h of a hospital that can perform specific operations (defined as cesarean section, laparotomy, and management of open fracture); (2) The number of surgeons, anesthesiologists, and obstetricians per 100,000 people; (3) The number of operations performed per 100,000 people; (4) In-hospital postoperative mortality rate (POMR); (5) The percentage of the population that would be pushed into poverty if they were to need an operation; and (6) The percentage of the population that would experience an economic catastrophe if they were to need an operation. These indicators can be used to provide a cross-sectional survey of the health of a country’s surgical system, and/or followed longitudinally to gauge progress in surgical systems development over time. As such, these indicators represent a first step in defining QI metrics in emergency and essential surgery around the world. The World Bank recognized the importance of these indicators and included them in its list of important health metrics—the World Development Indicators (WDI)—in 2016. This was the first time that the WDI included any surgical metrics.

Fig. 1
figure 1

The Lancet Commission on Global Surgery Indicators. The LCoGS recommended that every country measure at least these six indicators to give an estimate of the strength of their surgical system

Full size image

Another international surgical group known as the G4 Alliance recognized the importance of the LCoGS indicators but also saw the need for expansion of the list of six indicators. The G4 worked with leaders from around the world to build consensus metrics to expand on the work done by the LCoGS [71]. The G4 developed a list of 15 metrics that can be used in surgery and 3 of these were specific to trauma [71] (Table 4), including: (1) estimated portion of seriously injured patients transported by ambulance, (2) national whole blood donation rate, and (3) inpatient trauma mortality rate.

Table 4 The G4 Alliance proposed indicators
Full size table

The Trauma Injury Severity Score (TRISS) score allows for comparison of observed versus expected outcomes in trauma [72]. Similarly, several groups have attempted to develop trauma scoring systems for low resource settings. For instance, the Kampala Trauma Score (KTS) uses five easy metrics to collect variables for risk stratification [73]. The Codman score, which was developed using a database in South Africa, is a broader scoring system for surgical outcomes [74]. Many of these recommendations, guidelines, and metrics have only been developed over the past few years and much work is still needed to refine and implement them in the near future.

Controversies

Given the proliferation of quality and safety indicators in trauma and EGS, it is important to consider which indicators might be best for assessing true quality of care. Table 5 highlights some of the differences between administrative and clinical data obtained for quality indicator purposes.

Table 5 Key considerations (advantages and disadvantages) for quality indicators derived from administrative data
Full size table

Some criticisms of quality indicators include that many are not patient-centered, and that there is a tendency to focus more on classical outcomes (e.g., mortality, postoperative complications) to the exclusion of other, perhaps more meaningful or immediate measures, including functional outcomes, effectiveness of an intervention, patient comfort, and recurrence of disease [54, 75]. In addition, risk-adjustment remains an Achilles’ heel of quality benchmarking, especially in trauma and EGS where risks imposed by disease acuity and time-sensitivity impact outcomes so profoundly. Without robust risk adjustment, a surgeon or a center that cares for patients with higher injury burden and/or higher acuity of disease may be erroneously labeled as providing lower quality care. In time, this could create perverse incentives for many surgeons and hospitals to avoid caring for the highest risk patients and to avoid operating on patients with the most need for an emergent operation but with the least chance of survival.

Database quality also matters significantly when individual and organizational outcomes are shared, in particular if these outcomes are to be used for public reporting. The NTDB and TQIP have full-time staff dedicated to data entry, and in general, these databases are considered high-quality. Despite this, data collection challenges remain. One study showed that certain complications are only reported by 63% of participating hospitals within NTDB [76]. Other databases, in particular those relying solely on administrative data and those relying on automated data capture, may suffer from missing or incorrect data points.

Finally, while several studies have demonstrated improvements in clinical outcomes through the use of quality-focused databases like NSQIP [18, 19], some recent high-profile studies have suggested that data collection in and of itself does not necessarily lead to meaningful improvement. In other words, quality data collection is likely necessary but not sufficient to ensure quality improvement [77,78,79].

Looking Forward

Quality indicators have proliferated over the last two decades, with a large number directly related to trauma and EGS, and several more with applicability in LMICs [54,55,56, 80, 81]. Importantly, this trend reflects broad acknowledgement by trauma and EGS communities regarding the importance of quality improvement and patient safety. Yet, reasonable concerns exist regarding the reliability, validity, and potential impact of several quality-focused metrics in use today. The growing burden of data collection is another risk that must be recognized. Consensus-based guidance from national organizations (e.g., AAST, COT) may help to streamline and motivate these efforts going forward. Additionally, as AHRQ QIs shift towards ICD-10-CM-based coding, a new comprehensive list of trauma-related ICD-10 codes may help to catalyze new trauma-related quality improvement efforts through this well-established administrative data source [82]. Finally, EGS, like trauma, may benefit from a risk-adjusted national database with thoughtful approaches to risk adjustment and quality benchmarking, in order to catalyze much-needed quality improvement discussions and interventions in this emerging field.

Conclusions

As interest in quality and patient safety indicators in trauma and EGS continues to accelerate across the developed and developing world, there remains an unmet need for a common taxonomy of QIs, as well as more structured and standardized approaches to collecting the data necessary for quality comparisons within and across healthcare providers and systems. Key next steps will require bringing together relevant stakeholders to help streamline and rationalize these efforts going forward.