Abstract
Background While national databases provide large datasets that can be used to understand trends over time, their correlation with prospectively collected data from local registries has not been established. The purpose of the study was to compare differences in patient demographics and adverse events for patients undergoing elective posterior spinal fusion (PSF) between a national database and institutional registry.
Methods A retrospective chart review was performed. A total of 14,618 patients (13,678 patients from the National Surgical Quality Improvement Program [NSQIP] database and 940 patients from the institutional registry) who underwent elective 1- to 2-level PSF were included in the study. Preoperative patient demographics and comorbidities of each cohort were compared. In addition, postoperative 30-day complications and readmission were collected. A multivariate analysis was performed to examine for differences in risk factors for 30-day adverse events between the 2 cohorts.
Results A total of 13,678 patients from the NSQIP database and 940 patients from the institutional cohort were included for analysis. Mean age was similar between patient cohorts (60.8 ± 13.1NSQIP vs 58.8 ± 12.9registry), with NSQIP having significantly more patients over the age of 65 (41.4% vs 33.2%, P < 0.001). Overall complication rate was similar between NSQIP (6.8%) and the institutional registry (8.4%). Both found age and female sex to be significant predictors of 30-day adverse events, while obesity, hypertension, and smoking were only found to be predictive in the NSQIP database.
Conclusions Age and female sex were found to be independent risk factors for 30-day adverse events between both cohorts, while only NSQIP found modifiable comorbidities to be significant predictors. Although large databases allow for trends in quality over time, subtleties in practice variation and data collection methods at the individual institution level need to be considered when generalizing findings, especially as it pertains to modifiable factors.
Clinical Relevance Quality metrics and risk factors for patient outcomes are often derived from national databases. This study highlights the differences between study results when outcomes are derived from an institutional registry compared to a national database.
Level of Evidence 3.
- clinical outcomes
- lumbar
- NSQIP
- adverse events
- database outcomes
- posterior spinal fusion
- adult
- institutional registry
- fusion
Introduction
Posterior lumbar fusion is a commonly performed procedure, with rates increasing annually in the United States.1–3 As the health care system examines the sustainability of elective procedures and other trends in resource utilization, determining the value (quality of care/cost) of lumbar fusion has become the subject of much research, with many recent studies utilizing large national databases to examine perioperative complications.4–7 These databases provide substantially larger patient sample sizes than traditional clinical studies, yielding the potential to improve the precision of estimates of complications over time.7 While national databases may be pivotal in developing a better understanding of low frequency complications and associated risk factors, there are concerns regarding the reliability of data collection and reporting, as well as the generalizability of findings to individual practices.8
The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) is one of the most commonly used databases in orthopedic research, utilizing trained data collectors to retrospectively review 30-day perioperative morbidity and complications in elective and urgent surgery.4 Previous NSQIP database studies have suggested diabetes, smoking, and age as risk factors for major complication following elective posterior spinal fusion.9,10 While most national databases have inherent concerns of bias due to the retrospective nature of their data collection, NSQIP is considered to be one of the most reliable databases given that the data are collected by extensively trained surgical clinical reviewers, who analyze each patient chart individually, rather than solely through billing data.4
Bohl et al examined the differences in patient demographics and complications between 2 national databases among patients who underwent lumbar spine fusion.11 Despite similar demographic data for patients in both databases, the rate of comorbidities and adverse events varied in several instances.11 This important finding highlights the potential bias that results from variation in data collection and reporting methods. It is unknown, however, how a national database compares to a single-institution, high-volume registry, which collects data prospectively.
The purpose of this study was to compare variations in patient demographics, incidence of comorbidities, 30-day adverse events, and readmission rates between a high-volume, single-institutional prospective registry, and the NSQIP database for patients undergoing elective 1- to 2-level posterior spinal fusion. The null hypothesis was that there would be no significant differences in patient demographics or comorbidities between cohorts.
Materials and Methods
Study Design
This study was a retrospective analysis of patient data from 2 sources: (1) the NSQIP database from 2014 to 2018 and (2) a single institution prospectively collected quality registry from 2014 to 2019. Institutional review board (IRB) approval was obtained prior to initiation of the study for data collection from the local registry (IRB no. SFH-19-77). NSQIP database utilizes deidentified patient data, making it exempt from IRB approval. Patients of 18 years and older who underwent either 1- or 2-level primary, elective posterior lumbar spinal fusion were included in the study. Patients were excluded if their surgical diagnosis was related to a fracture, tumor, or infection, suggesting a nonelective procedure. In addition, patients undergoing revision surgery, lateral, or anterior interbody fusion were excluded.
Current procedural terminology (CPT) codes were used to identify the patients undergoing posterior spinal fusion, with or without a posterior approach interbody fusion (CPT 22612, 22630, 22633). Patients with 3 or greater levels of fusion were excluded by eliminating records with ≥2 instances of the additional level codes 22614, 22632, or 22634. Once a preliminary search was performed, electronic medical records were reviewed to ensure that each patient met the inclusion criteria.
Similarly for the NSQIP database, these CPT codes were used to identify patients eligible and those excluded. Records with the CPT code 22558 were eliminated to exclude patients undergoing either anterior or lateral interbody fusion. Records with the associated diagnosis codes CPT 63282 or 63265 were eliminated to exclude patients with tumor or infection. All diagnoses were reviewed, and any evidence of revision surgery was excluded.
Institutional Registry
The institutional registry used in this study is a level III registry for joint replacement, spine surgery, sports medicine, and hand surgery cases. In addition, it is currently certified by the American Board of Orthopedic Surgery.
Data sources for the registry include current and archived electronic medical record data, laboratory data, time-driven activity-based costing, and patient-reported outcomes. All data entered are first adjudicated using algorithms searching for variations from expected outcomes (ie, low troponin levels despite coding for myocardial infarction, etc). Any variation is then evaluated by the Clinical Quality Coordinator and/or Chief Quality Officer.
Variables Collected
Patient demographics were gathered from each source. There were 6 common comorbidities recorded for patients in both databases, which were used for analysis. Given the NSQIP database is limited to 30-day adverse and readmission rate, only common complications that were recorded between both sources during that interval were included for analysis (5 adverse events). Length of hospital stay and American Society of Anesthesiologists (ASA) classification were also included in the comparative analysis.
Statistics
Descriptive statistics including mean and standard deviation for continuous variables and frequency and proportion for categorical variables were calculated to characterize patient characteristics of each database. Differences in sociodemographic variables between each database were examined with independent t tests or Χ 2 tests where appropriate. Age- and sex-adjusted rates and ratios for the presence of comorbidity were generated using a generalized linear model with a Poisson distribution and a logit link. Finally, a multivariate analysis was performed to compare risk factors for 30-day adverse events between each database. The results of inferential analyses are reported as relative risk with corresponding 95% CI. All statistical analyses were performed using Stata 15.1 (StataCorp. 2017; Stata Statistical Software: Release 15. College Station, TX: StataCorp LP).
Results
Initial search results for posterior lumbar fusion resulted in 30,133 potential patients from the ACS NSQIP database. After implementing the exclusion criteria, 13,678 patients remained in the sample for analysis. Using the same collection criteria to the institutional registry, 1199 records were initially identified, and 940 remained after applying the exclusion criteria (Figure 1).
With regard to patient demographics, mean overall age in the NSQIP database was 60.8 (±13.1) compared with 58.8 (±12.9) for the institutional registry. Patients in the NSQIP database had a significantly higher proportion of patients aged 65 years or older (41.3% vs 33.2%, P < 0.001) (Table 1). There were no significant differences in regard to proportion of sexes in each cohort. A small, yet statistically significant, difference was found in body mass index (BMI) with NSQIP patients having a slightly higher average BMI (31.2 vs 29.7, P < 0.001). Both cohorts had patients with an ASA classification of IV occurring the least amount; however, the institutional registry had significantly more patients (74.9% vs 47.8%, P < 0.001) with an ASA II classification. Fifty-nine percent of patients were found to have a hospital stay of greater than 2 days in the NSQIP database, compared with 55% in the institutional registry (P = 0.012).
There were 6 common comorbidities collected by both data sources: obesity (defined as BMI >30), hypertension, congestive heart failure, smoking status, chronic obstructive pulmonary disease, and diabetes. Obesity and hypertension were the most common comorbidities in both databases, and congestive heart failure was the least common (Figure 2). With the exception of congestive heart failure, rate of obesity (Δ7.1% between cohorts), hypertension (Δ21.9%), smoking (Δ2.4%), and diabetes (Δ6.6%) were all more common occurrences in the NSQIP database. The relative incidence of having a comorbidity between the different databases was calculated in relation to NSQIP (Figure 3). Diabetes, chronic obstructive pulmonary disease, and obesity were found to have a significantly higher chance of occurrence in the national database compared to the institutional registry.
Common 30-day postoperative adverse events and readmissions were recorded between cohorts. The overall rate of complication was 8.4% in the registry compared with 6.8% in the NSQIP database. The most common adverse event in the NSQIP database was readmission (5.02%), while transfusion (4.7%) was the most common complication in the institutional registry. Figure 4 demonstrates the overall rates between the 2 databases. In relation to the NSQIP database, the risk of having a transfusion or acute kidney injury was significantly higher in the institutional registry (Figure 5).
A multivariate analysis was performed on potential predictors of 30-day adverse events between databases (Table 2). Age, female sex, 2-level fusion, obesity, smoking, and hypertension were all found to be significant risk factors for developing a complication in the NSQIP database. Conversely, only age and female sex were found to be significant predictors of a complication in the institutional registry.
Discussion
In this study, comparing 1- to 2-level elective posterior lumbar fusions between an institutional registry and a national database, patients in both cohorts were found to have relatively similar demographics despite statistical significant differences. The ACS NSQIP database found that obesity, hypertension, and smoking were independent risk factors for 30-day complication, while this was not reproduced using the institutional registry. This study also found that the 30-day adverse event rate of the common variables collected between cohorts was 6.8% in NSQIP compared with 8.4% in the institutional registry. These differences in rates of adverse events and associated risk factors may be explained by the differences in data collection or underlying patient selection.
There were several key differences between the NSQIP database and the institutional cohort in this study. One of the main differences was in the multivariate analysis, which found that certain “modifiable” comorbidities such as obesity, smoking, and hypertension were risk factors for complication in the national cohort. This could be explained by several reasons, one being the heterogeneity that is generated from a national database. This includes variations in surgeon experience, patient selection criteria, and differences in patient sociodemographics that cannot be teased out. In addition, the methods of how these comorbidities were defined are not fully understood. For instance, the degree of smoking or how well controlled the patients with hypertension is preoperatively, is not known in either cohort. These modifiable risk factors may be better preoperatively addressed in the institutional cohort, allowing for decreased risk of postoperative complication compared with the national database. This also could explain how the “nonmodifiable” comorbidities, such as age and female sex were both common risk factors between cohorts.
In addition to differences in associated risk factors, there were also differences found in adverse events. The biggest difference in the current study was found to be the transfusion rate. In a previous study using NSQIP, Katz et al found a transfusion rate of 12.2% in a similar population,12 compared with 5% in the institutional cohort and less than 1% in the NSQIP cohort found in the current study. These differences in findings from previous studies, again, may be secondary to the way that NSQIP data are collected, or the method of using CPT codes to ensure that only nonelective cases were included in analysis. It is also important to distinguish between clinically and statistically significant differences when interpreting large volume data. The current study found the 30-day readmission rate to be significantly more prevalent in the NSQIP database; however, this was only a 2% difference to the local registry, and thus may not be clinically meaningful. These differences bring important attention to the differences in methodology used between a national database and local institution.
National databases are increasingly being utilized in the orthopedic literature to report patient outcomes after surgery.13 These large scale databases offer the opportunity to understand surgical trends, outcomes, as well as health care costs.14 Given the ever changing health care environment and its emphasis on value, it is very likely that these national databases will be utilized to influence clinical practice improvement and future policy.14 Comparing one to a high-volume institution, with the differences found in the current study, demonstrates the potential issues with relying heavily on these national databases. The NSQIP database utilizes trained nurses to review charts, including examination of both inpatient and outpatient notes to identify specific comorbidity and adverse events within 30 days of surgery. Previous audits of the NSQIP database have demonstrated improved interrater consistency between reviewers over time, with disagreement rates as low as 1.58% between reviewers.8 The main drawback of NSQIP, along with other national databases, is the lack of patient-identifiable data with which to directly compare results for a true measure of validity. Due to this limitation, a direct measure of which national databases record data most accurately is not feasible. The local registry used in the current study, utilizes prospective data collection methods using a combination of information automatically generated by the electronic medical record and manually entered into the clinical database by trained personnel. The availability of the electronic medical record allows for confirmation of any discrepancy or missing variable that is not accessible in a national database. In addition, it takes into account the importance of patient optimization, without relying exclusively on previous diagnosis codes for a patient’s past medical history.
McLynn et al recently compared risk factors for venous thromboembolism (VTE) in elective spine surgery between the NSQIP database and a local, institutional registry.13 Akin to the current study, the authors found relatively similar age, gender distribution, and BMI between cohorts.15 However, the incidence of VTE in the institutional cohort was found to be roughly twice that in the NSQIP cohort, with several differences in risk factors identified in their multivariate analysis.15 In addition, their findings using NSQIP showed no reduction in VTE events with pharmacologic prophylaxis, which significantly differs from a prior meta-analysis16 involving studies from multiple local institutions.
There were several limitations to this study. First, there are inherent limitations of using any national database given its method for data collection. This includes a lack of clinical or procedural data, relying on CPT and International Classification of Diseases 9/10 coding, which may contain errors that cannot be verified by looking through the medical chart. Future studies comparing a prospectively collected spine registry to a high-volume institution may be useful in confirming the current findings. Second, the total complication rate was not included, as only complications that were recorded in both the institutional cohort and NSQIP were analyzed. This limits the analysis of several other comorbidities, and adverse events that are collected in either database. Finally, the methods of which adverse events were defined were not standardized between cohorts, also potentially explaining differences found between groups.
Conclusions
In this study, comparing a national database to an institutional cohort of patients undergoing elective 1- to 2-level posterior spinal fusion, the authors found overall relatively similar demographics, with a major difference in the incidence of patients who had an ASA ≥3 in the NSQIP database. In addition, nonmodifiable comorbidities including age and female sex were found to be predictors of adverse events in both cohorts, compared to obesity, hypertension, and smoking which were only found in the NSQIP database. These findings are likely secondary to the heterogeneity of a national database and the improved ability of a high-volume institution to optimize modifiable comorbidities preoperatively. Although large databases offer meaningful information on risk factors and trends in quality over time, subtleties in practice variation and data collection methods at the individual institution level need to be considered when generalizing findings, especially as it pertains to modifiable factors.
Footnotes
Funding The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests The authors report no conflicts of interest in this work.
Disclosures Isaac Moss reports consulting fees from Stryker Corp (Kalamazoo, MI), Pfizer Inc (New York, NY), and Biedermann Motech, Inc (Miami, FL). He also reports royalty fees from SpineArt SA and stock ownership in Orthozon Tech (Stamford, CT) and Spinal Simplicity (Overland Park, KS). No company had any influence on the study design, data acquisition, or manuscript of this study. The remaining authors have no disclosures.
Ethics Approval Institutional review board approval was obtained prior to initiation of the study for data collection from the local registry
- This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2022 ISASS. To see more or order reprints or permissions, see http://ijssurgery.com.