j.jamcollsurg.2017.09.016

ORIGINAL SCIENTIFIC ARTICLE
Postoperative Myocardial Infarction in
Administrative Data vs Clinical Registry:
A Multi-Institutional Study
David A Etzioni, MD, MSHS, FACS, Cynthia Lessow, RN, Liliana G Bordeianou, MD, FACS,
Hiroko Kunitake, MD, FACS, Sarah E Deery, MD, MPH, Evie Carchman, MD, FACS,
Christina M Papageorge, MD, George Fuhrman, MD, FACS, Rachel L Seiler, MPH,
James Ogilvie, MD, FACS, Elizabeth B Habermann, PhD, MPH, Yu-Hui H Chang, PhD,
Samuel R Money, MD, FACS
Previous studies have documented significant differences between administrative data and
registry data in the determination of postoperative MI. The goal of this study was to characterize discordance between administrative and registry data in the determination of postoperative myocardial infarction (MI).
STUDY DESIGN: This study was performed using data from the American College of Surgeons NSQIP merged with
administrative data from 8 different hospitals, between 2013 and 2015. From each of these sources,
the occurrence of a postoperative MI, as ascertained by administrative data and NSQIP data, were
compared. In each situation in which the 2 sources disagreed (discordance), a 2-clinician chart review
was performed to generate a “gold standard” determination as to the occurrence of postoperative MI.
RESULTS:
A total of 43,289 operations met our inclusion criteria for analysis. Within this cohort a total
of 230 cases of MI were identified by administrative data and/or NSQIP data (administrative
rate 0.41%, NSQIP rate 0.42%). A total of 89 discordant ascertainments were identified, of
which 42 were adminþ/NSQIP- and 47 were admin-/NSQIPþ. Accuracy (99.9% for both)
and concordance (kappa ¼ 0.89 [95% CI 0.86 to 0.92] for administrative data, kappa ¼ 0.87
[95% CI 0.84 to 0.91] for NSQIP data) of the 2 systems were similar when compared against
our gold standard (chart review). The majority of errors were related to false negatives, with
sensitivity rates of 81% in both data sources.
CONCLUSIONS: In this multi-institutional study, administrative data and NSQIP demonstrated a similar ability to
determine the occurrence of postoperative MI. These findings do not demonstrate an advantage of
registry data over administrative data in the determination of postoperative MI. (J Am Coll Surg 2017;
-:1e8. 2017 by the American College of Surgeons. Published by Elsevier Inc. All rights reserved.)
BACKGROUND:
Postoperative myocardial infarction (MI) occurs in 1.4%
to 5.0% of patients undergoing noncardiac surgery, and is
associated with a 30-day mortality rate of 11.6%.1,2 As a
potentially preventable adverse outcome of surgery, MI
is a valid target of research and quality improvement
aimed at minimizing patient-level risk.
Given the importance of MI as a focus of analysis, the mechanisms used to ascertain the occurrence of postoperative MI
deserve attention. The occurrence of a postoperative MI can
be identified on the basis of data collected for administrative/
billing purposes (eg Medicare data, discharge data) or registry
data that are abstracted prospectively/separately for the purposes of quality measurement. Each of these 2 mechanisms relies on trained coders to translate an analog medical record into
a digital representation (discharge abstract).
Disclosure Information: Nothing to disclose.
Support for this study: This publication was made possible by funding from
the Mayo Clinic Robert D and Patricia E Kern Center for the Science of
Health Care Delivery.
Received August 22, 2017; Accepted September 19, 2017.
From the Departments of Surgery, Mayo Clinic Arizona, Phoenix, AZ
(Etzioni, Lessow, Money); Massachusetts General Hospital, Boston, MA
(Bordeianou, Kunitake, Deery); University of Wisconsin, Madison, WI
(Carchman, Papageorge); Ochsner, New Orleans, LA (Fuhrman); University of Queensland Medical Center, Queensland, Australia (Seiler); and
Spectrum Health Medical Center, Grand Rapids, MI (Ogilvie); and the
Mayo Clinic Robert D and Patricia E Kern Center for the Science of Health
Care Delivery, Surgical Outcomes Program, Rochester, MN (Etzioni,
Lessow, Habermann, Chang).
Correspondence address: David A Etzioni, MD, MSHS, FACS, Department of Surgery, 5777 E Mayo Blvd, Phoenix, AZ 85054. email: etzioni.
david@mayo.edu
ª 2017 by the American College of Surgeons. Published by Elsevier Inc.
All rights reserved.
1
https://doi.org/10.1016/j.jamcollsurg.2017.09.016
ISSN 1072-7515/17
2
Etzioni et al
Data and Postoperative Myocardial Infarction
The relative ability of these 2 approaches to get the diagnosis of a postoperative MI “right” is unknown. What is
known is that when compared, these 2 systems differ considerably. In 1 study comparing Medicare data with data from
the NSQIP, 84% of postoperative MIs detected in Medicare
data were not considered MIs in the NSQIP dataset.3 This
level of discordance is troubling, especially given the fact
that data from both of these sources are used for public reporting.4,5 Medicare data are additionally used as the basis for
value-based purchasing, and have the potential to redirect a
significant portion of domestic health care expenditure and
patient volume.6
With this study, we analyzed the discordance between
administrative and NSQIP data in the determination of
postoperative MI. Our approach was not hypothesis
driven, but instead, sought to characterize and apportion
the potential reasons why the ascertainment of MI differs
between registry data and administrative data.
METHODS
Cohort
The unit of observation for this analysis was a surgical hospitalization, occurring in 1 of 8 distinct hospitals, between 2013
and October 2015. In developing this cohort, data from the
American College of Surgeons (ACS) NSQIP were used as
a starting point. Within the ACS NSQIP, participating hospitals contribute data regarding a random sample of their patients to the ACS, including preoperative/intraoperative
factors, and 30-day postoperative complications.7
At each site, ACS NSQIP data were merged with 2 specific elements of hospital discharge data: diagnosis codes
(using ICD-9 scheme) and present on admission (POA)
classifications relevant to each code.
Definition of complication
Within administrative data, the occurrence of a postoperative MI was based on the ICD-9 codes 410.X0 (myocardial infarction, episode of care unspecified) and 410.X1
(myocardial infarction, initial episode of care). This set
of codes was selected because it matches the codes
analyzed in another recent comparison of administrative
vs NSQIP-ascertained complications.3
Within NSQIP data, the occurrence of a postoperative MI
was determined based on the variable, “intraoperative or postoperative myocardial infarction.” The occurrence of an MI
was based on clinical criteria as defined in the ACS NSQIP
Operations Manual. Contents of this manual are considered
proprietary, but an abridged version is available online.8
The essence of this definition is: An acute myocardial
infarction that occurred intraoperatively or within 30
days after surgery, as manifested by 1 of the following:
J Am Coll Surg
Documentation of ECG changes indicative of acute MI
(1 or more of the following): ST elevation > 1 mm in
2 or more contiguous leads, new left bundle branch,
and/or new q-wave in 2 of more contiguous leads; new
elevation in troponin greater than 3 times upper level of
the reference range in the setting of suspected myocardial
ischemia; or physician diagnosis of myocardial infarction.
The definition adds that a cardiology consultation that
determines an absence of ischemia/MI is sufficient to
“overrule” other aspects of the patient’s determination.
Data from NSQIP include not only occurrence of the
MI, but also the first date on which an MI was diagnosed.
An MI diagnosed postoperatively and before discharge was
therefore considered a postoperative MI. The primary analysis of this study focuses on agreement between administrative data and NSQIP data. Administrative data, by
definition, is oblivious to post-discharge complications.
Therefore, any MI diagnosed post-discharge in the NSQIP
dataset was considered a non-MI (for NSQIP data).
Chart review
Every operation with discordance between administrative
data and NSQIP data regarding the occurrence of a postoperative MI was reviewed within each participating hospital using
an abstraction form. Abstracted data included presence of a
preoperative MI, highest troponin value, date of highest
troponin value, whether a clinical diagnosis of MI was
made, the date of such diagnosis, whether a cardiology consultation was obtained, the date of the consultation (if one
occurred), and the final cardiology diagnosis (if cardiology
consultation occurred). Each of these reviews was performed
independently by 2 clinicians experienced in perioperative
care, at least 1 of whom had to be an attending surgeon. Surgical case reviewers (SCRs) from NSQIP could not be reviewers. Reviewers were blinded to the type of discordance
and the date of the occurrence within the NSQIP data.
After the dual review was completed, the 2 reviews were
reconciled. In this reconciliation, local reviewers were asked
to consider each data element that differed between the 2 reviewers. In cases in which consensus was unable to be reached,
the data element was considered “uncertain/unclear.”
Exclusion criteria
Operations performed on an outpatient basis (admit
date ¼ discharge date) were excluded, as were those in
which the patient was discharged on October 1, 2015
or later. This date-based exclusion was necessary because
after this date, the ICD-10 scheme replaced the ICD-9
scheme in most hospital discharge datasets.
Within administrative data, the POA indicator was capable
of identifying patients who had an MI that was considered
present on admission. When such an indicator was present,
Vol.
-,
No.
-, -
2017
Etzioni et al
Data and Postoperative Myocardial Infarction
3
Figure 1. Cohort flowchart. Admin, administrative; MI, myocardial infarction; POA, present on admission; postop,
postoperative; preop, preoperative.
the hospitalization was excluded because it was (by definition)
no longer possible to determine the occurrence of a postoperative MI within the administrative dataset. Within NSQIP
data, postoperative occurrences more than 30 days after the
index operation are not abstracted. Therefore, any MI that
occurred during the index hospitalization and more than 30
days postoperatively were excluded from analysis.
Statistical analysis
The primary goal of this study was to analyze concordance between different mechanisms of ascertaining a specific complication. Description of concordance was performed using
Cohen’s kappa, which can range between -1 (very poor agreement) and 1 (perfect agreement). Accuracy, sensitivity, and
specificity were measured according to standard epidemiologic estimations.9 Comparisons of these tests characteristics
were performed using either McNemar’s test or using a
weighted generalized score statistic.10,11 Statistical analyses
were conducted with SAS version 9.4 (SAS Institute Inc).
Institutional review
This research study was reviewed by the IRB of each of
the participating hospitals.
RESULTS
Cohort
After applying our inclusion and exclusion criteria (Fig. 1),
the total cohort for analysis included 43,289 patients, with
individual hospitals contributing between 3,203 and 8,163
observations to the analysis. Cohort characteristics are
included in Table 1. The average age (SD) of analyzed patients was 59.5 15.8 years, and 53.6% were women.
Unadjusted myocardial infarction rates,
concordance
Unadjusted rates of postoperative in-hospital MI in the 2 datasets
were similar, with 0.42% for administrative data and 0.41% for
NSQIP. The Cohen’s kappa value comparing the evaluations
from administrative vs NSQIP determination was 0.71 (95%
CI 0.66 to 0.77). A total of 230 cases of MI were identified by
administrative data and/or NSQIP data. Of these 230 cases,
128 cases (56%) were considered an MI by both systems.
Chart review
Our methods identified 102 cases of discordance, and
each of these underwent a dual-clinician chart review.
4
Etzioni et al
Table 1.
Patient Population
Data and Postoperative Myocardial Infarction
Characteristic
Total n
Age, y, mean SD
Sex, n (%)
Male
Female
Missing
Surgical subspecialty, n (%)
Cardiac
General surgery
Gynecology
Neurosurgery
Orthopaedics
Otolaryngology
Plastics
Thoracic
Urology
Vascular
Length of stay, d
Mean SD
Median
Unadjusted MI diagnosis rate, %
Administrative data
NSQIP data
Year of operation, n (%)
2013
2014
2015
Data
43,276
59.5 15.8
19,919 (46.0)
23,067 (53.3)
290 (0.7)
81
15,593
2,456
3,157
9,448
2,271
1,888
1,034
4,223
3,125
(0.2)
(36.2)
(5.7)
(7.3)
(21.8)
(5.3)
(4.4)
(2.4)
(9.8)
(7.2)
4.28 8.4
3
0.42
0.41
15,398 (35.6)
15,924 (36.8)
11,954 (27.6)
MI, myocardial Infarction.
Of these 102 cases, 13 were excluded from further analysis
due to evidence that a recent MI was part of the patient’s
preoperative risk profile. Of the remaining 89 discordant
cases, 42 were “type A discordance,” for which administrative data identified postoperative MI that NSQIP did
not. Another 47 were “type B discordance,” for which
NSQIP data identified a postoperative MI that administrative data did not. Each of these 89 cases underwent a
dual-clinician review. Of these 89 discordant cases, the
vast majority (79 of 89) were considered false negatives
based on this review and a minority (10 of 89) were
considered false positives. Based on the chart review,
each discordant record could be categorized into 1 of 9
mutually exclusive groups (Table 2).
In the vast majority of false negatives, a cardiology
consultation determined either MI or ischemia in the
presence of elevated troponins (group 1a). Of the false
positives, 8 cases were considered by the NSQIP reviewers to be MI despite a cardiology evaluation that
determined neither MI nor ischemia to be present (group
J Am Coll Surg
3a). Two false positives were considered an MI by
administrative mechanisms despite normal troponins
and the absence of a clinical diagnosis in the record
(group 3b).
False positives/negatives and accuracy
In order to compute the accuracy, sensitivity, and specificity for administrative vs NSQIP data, 3 important assumptions needed to be made. First, that every case in
which both mechanisms (administrative data and
NSQIP) ascertained an MI was truly an MI. Second, in
every case in which both mechanisms determined that
no MI occurred, no MI had occurred. Third, in situations
of discordance that the 2-clinician chart review performed
in this study was correct (“gold standard”). Based on these
assumptions, the operational characteristics of administrative vs NSQIP data could be calculated, and these are reported in Figure 2. A comparison of administrative and
NSQIP against the chart review (gold standard) demonstrates similar kappa values (p ¼ 0.45), accuracy (p ¼
0.45), sensitivity (p ¼ 0.91), and specificity (p ¼ 0.06).
Figure 3 displays unadjusted hospital-specific rates of
MI (administrative, NSQIP) and concordance (kappa
values) between administrative and NSQIP data. Sample
sizes by site are not shown in order to preserve site
identity.
Sensitivity analyses
The approach used in this study to identify complications
using ICD-9 codes is only 1 of several potential strategies.
In order to test the robustness of our approach, 2 sensitivity analyses were performed. First, the administrative
codes used as the basis for ascertaining MI data were
modified. The strategy outlined here, based on ICD-9
codes 410.X0 and 410.X1, is not the only viable approach.
Code 410.X1 denotes a “first episode of care” for an MI.
The 410.X0 code, however, denotes “episode of care unspecified” and may therefore not be appropriate in determining postoperative MI. The performance of
administrative data using an approach that is restricted
to ICD-9 codes 410.X1 was tested as alternative strategy
#1. Second, the position of the MI diagnosis code within
the list of diagnosis codes relevant to the hospitalization
may be related to the accuracy of MI coding. Performance
of administrative data in determining MI based only on
the presence of an MI code in the first 3 listed diagnoses
was tested as alternative strategy #2. The operational characteristics of these 2 alternative strategies relative to the
baseline strategy are shown in Table 3. Neither of the
alternative strategies performed superiorly to the basic
strategy.
Vol.
-,
No.
Table 2.
-, -
2017
Etzioni et al
Data and Postoperative Myocardial Infarction
5
Categorization of Discordant Cases
Chart review finding
Type A discordance
(Adminþ/NSQIP-), (n ¼ 42) n
Type B discordance
(Admin-/NSQIPþ), (n ¼ 47) n
29
3
3
3
29
3
1
5
0
1
2
0
0
0
0
2y
8*
0
False negative - group 1: troponin elevation greater than 3x upper limit
of normal, and:
1a: Cardiology evaluation ¼ ischemia/MI
1b: Cardiology evaluation ¼ unclear/uncertain
1c: No cardiology evaluation, clinical diagnosis of MI
1d: No cardiology evaluation, no clinical diagnosis of MI
1e: No cardiology evaluation, clinical diagnosis of MI ¼ unclear/
uncertain
False negative - group 2: no troponin elevation greater than 3x upper
limit of normal and:
2a: Clinical diagnosis of MI/ischemia in chart (either cardiology or
other physician)
2b: Clinical diagnosis of MI/ischemia unclear/uncertain
False positive - group 3: MI excluded
3a: Troponin elevation, cardiology evaluation ¼ no MI, no ischemia
3b: No troponin elevation, no clinical diagnosis of MI in chart
*NSQIP false positives; all other entries in this column are administrative false negatives.
y
Administrative claim false positives; all other entries in this column are NSQIP false negatives.
Admin, administrative; MI, myocardial infarction.
DISCUSSION
A postoperative MI is a dramatic event, and one that has
clear impact on the short-term and long-term outcomes
for a surgical patient. Intuitively, the occurrence of an
MI is dichotomous; either an MI occurs or does not.
With this study, we assessed the performance of 2
competing mechanismsdadministrative data and registry
datadin determining the occurrence of a postoperative
MI. In this comparison, these 2 very different mechanisms had nearly identical operational characteristics.
At some level, this finding should be reassuring. Both
administrative data and NSQIP data are publicly reported
and therefore they should give similar results. Unadjusted
rates of postoperative in-hospital MI were 0.42% in
administrative data and 0.41% in NSQIP data. Kappa
values between these datasets and a gold standard (as
defined in this study) were high, at 0.89 for administrative
data and 0.87 for NSQIP. The sensitivity for detecting
MI, however, was lower than what might be considered
acceptable, at approximately 81% for both systems.
Previous head-to-head comparisons of NSQIP data and
administrative data have found dramatic differences in
terms of their ascertainment of postoperative MI. In a
comparison of 117,752 Medicare patients, Lawson and
colleagues3 found that 84% of postoperative MIs within
Medicare data were not considered MIs in NSQIP data.
A similar, smaller study in the UK examined concordance
between a National Health Service Database and NSQIP,
and found that 59% of MIs recorded in an administrative
dataset (1,323 operations) were not MIs in NSQIP. Our
study found that 30% of administratively coded MIs were
not considered MIs in the NSQIP dataset.
It would be a mistake, however, to consider these proportions as estimates of a false positive rate within administrative datasets. Instead, these estimates are evidence of
discordance between 2 different approaches to ascertaining the occurrence of postoperative complications. In
this study, we undertook an intensive, 2-clinician review
of each situation in which there was discordance between
the 2 datasets in order to better understand the differences
between these 2 datasets. We found that both approaches
have similar sensitivity, specificity, and accuracy for determining that an MI occurred. In both of these datasets, the
vast majority of errors (79 of 89, 89%) were the result of
false negativesdsituations in which a coder/abstractor
failed to ascertain an MI that was clearly present.
Figure 2. Operational characteristics of administrative vs NSQIP
determination of myocardial infarction relative to gold standard.
Admin, administrative; MI, myocardial infarction.
6
Etzioni et al
J Am Coll Surg
Data and Postoperative Myocardial Infarction
Figure 3. Concordance (administrative vs NSQIP data) by study site. Admin, administrative; MI,
myocardial infarction.
The approach taken in this study is more powerful than
earlier investigations for 3 reasons. First, this study represents
a review of the experience of 8 different hospitals, each with
its own individual administrative and NSQIP coders. Therefore our findings are more representative of the national
experience than a single-institution study would be. Second,
each chart review conducted in this study was focused on a
single complication, and therefore, the scrutiny applied to
characterizing each occurrence of discordance was magnified.
Third, an additional level of scrutiny was applied through
our use of a dual-clinician review. This dual review, combined with the reviewer focus on a single complication, imparts a greater level of diagnostic precision than is present in
either administrative or NSQIP data.
Table 3.
This study has several limitations. First and foremost,
this study was not designed to examine the clinical and
interventional algorithms that are used to diagnose and
treat postoperative MI. Strategies to screen postoperative
patients with a sensitive troponin assay, with the goal of
stratifying and reducing MI risk, are an emerging area of
research, but are not yet widely used in clinical practice.12 In this study, the clinical thresholds used to diagnose MI surely varied between sites, as well as between
physicians within sites. The severity of MI was not
analyzed, nor was the use of revascularization. These aspects of clinical care we considered separate from the
goal of the studydto characterize the accuracy of existing methods in the determination of postoperative MI.
Evaluation of ICD-9 Code Selection Criteria
Strategy
Basic strategy 1: Default definition of MI
(ICD-9 codes 410.X1 and 410.X0)
Alternate strategy 1: Restrict to ICD-9
codes 410.X1
Alternate strategy 2: MI code in first 3
diagnosis code positions only
MI, myocardial infarction.
Operational characteristics relative to gold standard
Kappa (95% CI)
Accuracy, %
Sensitivity, %
Specificity, %
0.89 (0.86e0.93)
99.9
81.2
99.9
0.57 (0.50e0.64)
99.7
40.1
99.7
0.67 (0.61e0.73)
99.8
50.7
99.8
Vol.
-,
No.
-, -
2017
Etzioni et al
Other limitations are also worth mentioning. In order
to analyze rates of accuracy, an assumption was made
that in all cases of agreement between administrative
and NSQIP data, both datasets were “correct.” This
assumption was necessary in order to focus on the relatively small number of cases in which there was discordance, but it is possible that some of these cases in
which concordance was present were mis-classified by
both data sources (“disconcordance”). The selection of
which codes from within the administrative dataset were
used to denote the occurrence of an MI is also a potential
limitation. Several different approaches have been
described,13 and the approach used in this study, when
compared against 2 alternative strategies, yielded the
best performance. Our study may also be criticized for
its reliance on the ICD-9 coding scheme, which was
phased out in 2015. There is no good reason to believe,
however, that coding behavior under the ICD-10 scheme
will be significantly different.
Despite these limitations, this study stands to inform
the ongoing movement toward increasing measurement,
analysis, and reporting of surgical outcomes. Within
this movement, administrative- and registry-based mechanisms have competing/complementary roles in informing clinical leaders regarding surgical quality of care.
Regardless of which approach is used, some aspects of
measurement can be strengthened. First, the status of
any major event such as an MI should be clearly designated as present on admission (POA), present at time
of surgery (PATOS), or postoperative. Both administrative data and NSQIP data have deficiencies in this area.
Administrative data do not have the ability to designate
whether a condition occurred between admission and
operation (eg a patient who had an MI after admission
but before surgery), so there is the potential for preoperative occurrences to be inaccurately considered as a
complication. Data from NSQIP do not capture whether
or not an MI was present at the time of surgery, and therefore may be underestimating risk in patients with a preoperative MI. In this analysis, a significant number of
discordant cases had evidence of a preoperative MI (13
of 102, 13%). Second, the nature of clinical documentation should change to meet evolving information needs.
Well-established criteria exist to diagnose MI.14 Clinical
leadership needs to become more accustomed to applying
these criteria in an explicit/categorical fashion so that
translation of the medical record into a digital format is
straightforward and less prone to errors in translation.
With these steps, the ability of interested parties to better
ascertain the occurrence of postoperative MI would be
improved. These changes would be of significant benefit
to patients, health care policy, and outcomes researchers.
Data and Postoperative Myocardial Infarction
7
CONCLUSIONS
This multi-institutional analysis of postoperative MI in
43,289 patients challenges accepted beliefs regarding the
relative strengths and weaknesses of administrative and
registry data. Registry data, such as NSQIP, are generally
considered superior because of standardized training of
clinical abstractors and the use of stringent clinical definitions to govern abstraction. These data come at a high
price, however, in terms of personnel and program costs.
Administrative data, which are generated primarily for
billing purposes, are widely considered less accurate
because of the absence of universally accepted criteria
and variability in the clinical acumen of administrative
coders/abstractors. The decision as to which of these 2
data sources to use as the basis for quality measurement
is, theoretically, rooted in a tradeoff between cost and accuracy of data. As the quality movement progresses, a
quantitative characterization of this tradeoff is critical.
Author Contributions
Study conception and design: Etzioni, Lessow, Habermann, Chang, Money
Acquisition of data: Etzioni, Lessow, Bordeianou, Kunitake, Deery Carchman, Papageorge, Fuhrman, Seiler,
Ogilvie
Analysis and interpretation of data: Etzioni, Lessow,
Bordeianou, Kunitake, Deery Carchman, Papageorge,
Fuhrman, Seiler, Ogilvie
Drafting of manuscript: Etzioni, Habermann, Chang,
Money
Critical revision: Etzioni, Lessow, Bordeianou, Kunitake,
Deery Carchman, Papageorge, Fuhrman, Seiler, Ogilvie, Habermann, Chang, Money
REFERENCES
1. Devereaux PJ, Xavier D, Pogue J, et al. Characteristics and
short-term prognosis of perioperative myocardial infarction
in patients undergoing noncardiac surgery: a cohort study.
Ann Int Med 2011;154:523e528.
2. Devereaux PJ, Goldman L, Cook DJ, et al. Perioperative cardiac events in patients undergoing noncardiac surgery: a review
of the magnitude of the problem, the pathophysiology of the
events and methods to estimate and communicate risk.
CMAJ 2005;173:627e634.
3. Lawson EH, Louie R, Zingmond DS, et al. A comparison of
clinical registry versus administrative claims data for reporting
of 30-day surgical complications. Ann Surg 2012;256:
973e981.
4. NSQIP Hospital Compare Data. Available at: https://
data.medicare.gov/Hospital-Compare/NSQIP-HOSPITALCOMPARE-DATA/c635-s3cy. Accessed March 8, 2017.
5. Medicare.gov Hospital Compare. Available at: https://www.
medicare.gov/hospitalcompare/search.html. Accessed March
8, 2017.
8
Etzioni et al
Data and Postoperative Myocardial Infarction
6. Hospital Value-Based Purchasing. Available at: https://www.cms.
gov/Outreach-and-Education/Medicare-Learning-NetworkMLN/MLNProducts/downloads/Hospital_VBPurchasing_Fact_
Sheet_ICN907664.pdf. Accessed April 20, 2017.
7. Ingraham AM, Richards KE, Hall BL, Ko CY. Quality
improvement in surgery: the American College of Surgeons
National Surgical Quality Improvement Program approach.
Adv Surg 2010;44:251e267.
8. ACS National Surgical Quality Improvement Program. Available at: http://site.acsnsqip.org/wp-content/uploads/2014/11/
ACS_NSQIP_PUF_User_Guide_2013.pdf. Accessed April 4,
2017.
9. Wikipedia. Confusion matrix. Available at: https://en.
wikipedia.org/wiki/Confusion_matrix. Accessed April 18,
2017.
J Am Coll Surg
10. Kosinski AS. A weighted generalized score statistic for comparison of predictive values of diagnostic tests. Stat Med 2013;32:
964e977.
11. McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika
1947;12:153e157.
12. Devereaux PJ, Biccard BM, Sigamani A, et al. Association of
postoperative high-sensitivity troponin levels with myocardial
injury and 30-day mortality among patients undergoing
noncardiac surgery. JAMA 2017;317:1642e1651.
13. Mentz RJ, Newby LK, Neely B, et al. Assessment of administrative data to identify acute myocardial infarction in electronic
health records. J Am Coll Cardiol 2016;67:2441e2442.
14. Thygesen K, Alpert JS, Jaffe AS, et al. Third universal definition
of myocardial infarction. Circulation 2012;126:2020e2035.