Do PEEK Rods for Posterior Instrumented Fusion in the Lumbar Spine Reduce the Risk of Adjacent Segment Disease?
===============================================================================================================

* Daniel Hirt
* Heather A. Prentice
* Jessica E. Harris
* Elizabeth W. Paxton
* Jessa Alexander
* Daniel T. Nagasawa
* Deven Khosla
* Steven M. Kurtz

## ABSTRACT

**Background:** Polyetheretherketone (PEEK) rods were clinically introduced in the mid-2000s as an alternative to titanium (Ti) rods for posterior instrumented lumbar spine fusion, theorized to reduce the risk of adjacent segment disease (ASD). However, few studies have follow-up beyond 2 years. Consequently, we conducted a matched cohort study using data from Kaiser Permanente's spine registry to compare the 2 rod systems and risk for outcomes.

**Methods:** Patients aged ≥18 undergoing first posterior lumbar fusion for a degenerative diagnosis from 2009 to 2018 using either a PEEK or a Ti rod were identified. Fusions using Ti rods were 2:1 propensity score matched to PEEK rods on the following factors: patient age, body mass index, smoking, American Society of Anesthesiologists classification, diagnosis, interbody use, bone morphogenic protein use, number of levels fused, fusion levels, and operative year. The matched sample included 154 PEEK and 308 Ti fusions. We used Cox regression to evaluate ASD and nonunion, and logistic regression to evaluate 90-day emergency department (ED) visit, readmission, and complication.

**Results:** We did not observe a difference in risk for ASD (hazard ratio = 1.02, 95% confidence interval [CI] = 0.66–1.59) or ED visit (odds ratio [OR] = 0.88, 95% CI = 0.48–1.59). A lower likelihood of readmission (OR = 0.34, 95% CI = 0.13–0.94) was observed following PEEK fusion compared with Ti. No nonunions or 90-day complications were observed for the PEEK group; 5 (2-year cumulative incidence = 0.7%) nonunions and 4 (1.3%) complications were observed for the Ti group.

**Conclusions:** Our multicenter study did not support the hypothesis that PEEK rods are associated with a lower ASD risk. Reasons for readmission need to be identified to better understand the differences observed here. Further study of patients with TLIF using Ti and PEEK rods and posterolateral fusion with Ti and PEEK rods is needed.

**Clinical Relevance:** The present study adds to the literature supporting their midterm effectiveness of PEEK rods compared with Ti rods for both their safety and their effectiveness at the 5–7-year follow-up.

**Level of Evidence:** 3.

*   lumbar spine
*   fusion
*   polyetheretherketone rod
*   titanium rod
*   adjacent segment disease
*   readmission

## INTRODUCTION

In 2005, polyetheretherketone (PEEK) rods received United States regulatory clearance and were clinically introduced for posterior lumbar fusion.1 However, they were also considered potentially suitable for off-label indications, such as dynamic stabilization.2 In the following decade, as clinical debate ensued about fusion versus dynamic stabilization to address adjacent segment disease (ASD), the originally cleared indication for PEEK rods for fusion received little attention in the literature.1 Now, as fusion persists as a dominant treatment of choice for degenerative lumbar spine disorders, there has been a resurgence of interest in PEEK rods for their initially cleared fusion indication.3

As adjuncts to fusion, the theoretical benefits of PEEK rods include compatibility with diagnostic imaging, radiolucency, and lower stiffness, which may improve the rate of fusion and potentially reduce ASD. In theory, a semirigid rod would be superior to a relatively rigid metallic rod in preventing ASD, and numerous studies have tested this hypothesis using biomechanical or animal- or cadaver-based models.4–8 These ex vivo studies suggest that segments instrumented with PEEK more closely mimic intact physiologic loading in the adjacent level, which may reduce the likelihood of ASD.

Previous clinical studies of lumbar fusion using PEEK rods are single center with small numbers of patients and 1–3 years of follow-up.1,3,9–13 In a recent meta-analysis, Selim et al3 found no significant differences in clinical outcomes, including rates of fusion, functional and pain outcome scores, or device-related adverse events between PEEK and metal rods, and concluded that PEEK rods appeared to be clinically successful as adjuncts to lumbar spine fusion. However, little is known about the comparative effectiveness of PEEK versus titanium (Ti) rods with respect to both 90-day complications and longer-term outcomes, including ASD. In particular, since PEEK rods were clinically introduced more than 15 years ago, complications and readmissions within 90 days have emerged as a clinically significant benchmark for quality of care for lumbar fusion surgery the United States,14–16 and previous literature on PEEK rods is silent on this topic. Consequently, we conducted a multicenter matched-cohort study using data from Kaiser Permanente's spine registry to compare PEEK versus Ti rod systems and risk for 90-day complications, such as readmission, as well longer-term outcomes, such as reoperation for nonunion and ASD.

## METHODS

### Study Design, Setting, and Data Source

We conducted a matched retrospective cohort study using data from Kaiser Permanente's spine registry. This large integrated health care system covers more than 12 million members throughout 8 geographical (Colorado, Georgia, Hawaii, the Mid-Atlantic, northern California, the Northwest, southern California, and Washington) regions in the United States.17 This patient population has been shown to be demographically and socioeconomically representative of the geographic region it covers.18,19

Detailed information on the spine registry's coverage, data collection procedures, and quality assurance has been previously published.20–22 In brief, the registry is a surveillance tool for instrumented spine procedures performed within 4 regions (Hawaii, northern California, the Northwest, and southern California) of the health care system where there are institution-owned hospitals. Patient, procedure, implant, surgeon, and hospital information is prospectively collected using electronic intraoperative forms that are completed at the time of the index procedure by the operating surgeon, as well as scanned implant bar codes. This information is then supplemented using data from the integrated electronic health record (EHR), administrative claims data, membership data, and mortality records. All patients included in the registry are longitudinally monitored after the index spine procedure for outcomes using electronic screening algorithms, and all outcomes are manually validated by trained research associates using the EHR. Once added to the registry, all patients are prospectively monitored for outcomes until death or health care membership termination.

### Inclusion and Exclusion Criteria

Patients aged ≥18 years undergoing first posterior instrumented lumbar spine fusion with a Medtronic CD Horizon rod for the diagnosis of adult deformity, spondylolisthesis, or stenosis were included. Operative years 2009–2013 and 2016–2018 were included (the registry does not have information on spine procedures for 2014–2015). Patients with prior spine fusions, fusions including other regions of the spine, or nondegenerative indications were excluded. Lumbar fusions where rods from other manufacturers were used were further excluded.

### Exposure of Interest

The primary exposure was the use of a PEEK rod compared to a Ti rod for posterior instrumented fusion, identified using the spine registry. Implant data are captured into the registry from the implant module of the EHR. At the time of the procedure, the product Global Trade Item Number for any implant is scanned, and this information is linked to a master item list maintained and validated by the Global Data Synchronization Network to obtain reference number, catalog number, implant description, manufacturer, and implant type to populate the implant module.

### Outcome of Interest

The primary outcome of interest was reoperation for ASD. All patients were continuously monitored by the registry for reoperation following the index spine procedure until death, being lost to follow-up through health care membership termination, or study end date (March 31, 2019). During the study period, 302 (10.1%) patients were lost to follow-up through membership termination. The reason for reoperation was recorded by the operative surgeon using operative forms and confirmed via chart review by a trained clinical research associate to ensure registry accuracy.

Secondary outcomes included reoperation for nonunion and 90-day adverse postoperative events. Ninety-day events evaluated were emergency department (ED) visit, readmission, and complication (including deep infection, deep vein thrombosis, and pulmonary embolism). ED visit included any ED encounter, and readmission was defined as any rehospitalization within the integrated health care system in the 90 days following discharge. ED visits and readmissions were identified using encounter information in the integrated EHR. Complications were defined according to the Agency for Healthcare Research and Quality indicators23 and identified using the EHR.

### Covariates

Patient covariates consisted of age in years (continuous), body mass index (BMI) in kg/m2 (continuous), smoking status (smoker, quit, and never), American Society of Anaesthesiologists (ASA) classification (1, 2, 3, and 4), and diagnosis (adult deformity, spondylolisthesis, and stenosis). Surgical characteristics included operative time in minutes (continuous), interbody use, bone morphogenic protein (BMP) use, number of levels fused (1, 2, 3, and 4), fused level (L1–L2, L2–L3, L3–L4, L4–L5, and L5–S1), and operative year.

## STATISTICAL METHODS

To evaluate the association between PEEK rod utilization and outcomes, 2:1 pair matching of instrumented lumbar spine fusions with a Ti rod to procedures with a PEEK rod using propensity scores was performed. The propensity score is estimated by fitting a logistic regression model that regresses treatment assignment on the measured baseline covariates. We created separate levels for nominal covariates with missing values and a missing indicator variable for continuous variables with missing data (ie, BMI) while also imputing the mean.24 Matched pairs were formed using nearest-neighbor pair matching on the logit of the propensity score with a caliper width equal to 0.2 of the standard deviation of the logit. In order to not omit patients who received PEEK rods, if a match could not be found within the caliper, the nearest match was found. The standardized difference for each covariate before (unbalance standardized difference) and after (balanced standardized difference) applying propensity score matching can inform how well the application of propensity scores balanced the distribution of covariates between groups; a standardized difference <0.1 indicates that balance is achieved.25,26

Crude cumulative incidence was calculated as 1 minus the Kaplan-Meier estimator for time-to-event outcomes (operative ASD and nonunion) and the proportion of events over the number at risk for 90-day binary outcomes (ED visit, readmission, and complication). Cox regression models were used to evaluate time-to-event outcomes, and logistic regression models were used to evaluate 90-day binary outcomes. Follow-up time for time-to-event analyses was defined as the time from the index spine procedure to the date of reoperation, date of death, date of membership termination, or study end date, whichever came first; analysis censored patients who died, terminated membership, or reached the study end date. Variance estimation of the treatment effect made use of an approach that accounts for observations being nested within surgeon and matched pairs using robust standard errors.27 Data were analyzed using R (version 3.6.0, R Foundation for Statistical Computing, Vienna, Austria). All tests were 2-sided, and an alpha of 0.05 was the significance threshold for this study.

## RESULTS

The identified sample comprised 2998 lumbar spine fusion patients, 154 (5.1%) utilizing a PEEK rod. Mean age and BMI for the PEEK group were 59.9 (standard deviation [SD] = 14.3) years and 28.2 (SD = 5.4) kg/m2. Most patients who underwent PEEK rod lumbar spine fusion were female (55.8%), never smokers (51.9%), and had an ASA classification of 2 (63.6%); stenosis was the dominant diagnosis (80.5%). Mean operative time was 220.3 (SD = 104.3) minutes; most procedures used interbody (63.0%) and BMP (70.8%). Single-level procedures were most common (63.0%), the primary location being L4–L5 (67.5%). There was imbalance for some of the covariates compared to patients who underwent a Ti rod lumbar spine fusion. After applying 2:1 propensity score matching, the matched Ti rod group (N = 308) was similar to the PEEK group for all covariates, and the imbalance was reduced (standardized difference < 0.1 for all covariates) (Table 1).

View this table:
[Table 1](http://ijssurgery.com//content/15/2/251/T1)

Table 1 
Patient and surgery characteristics of lumbar spine fusion patients by rod material (N = 2998) before and after propensity score matching (2009–2018).

Figure 1 presents the ASD-free probability during postoperative follow-up with the number of risk and number of events. At the 5-year follow-up, the crude reoperation for ASD probabilities were 7.5% (95% confidence interval [CI] = 4.0–14.0) for the PEEK lumbar spine group and 11.5% (95% CI = 10.2–12.9) for the Ti lumbar spine group. Incidence in the matched Ti group is reported in Table 2. No reoperations for nonunion were observed at the 2-year follow-up for the PEEK group, while incidence was 1.3% (95% CI = 0.9–1.8) for the Ti group. After propensity score matching, no difference was observed in risk for reoperation due to ASD when comparing PEEK to Ti lumbar spine procedures (hazard ratio = 1.02, 95% CI = 0.66–1.59) (Table 3).

![Figure 1](http://ijssurgery.com//http://www.ijssurgery.com/content/ijss/15/2/251/F1.medium.gif)

[Figure 1](http://ijssurgery.com//content/15/2/251/F1)

Figure 1 
Reoperation due to adjacent segment disease (ASD)–free probability (left panel = overall, right panel = matched) following lumbar spine fusion using polyetheretherketone (PEEK) rods compared to titanium (Ti) rods.

View this table:
[Table 2](http://ijssurgery.com//content/15/2/251/T2)

Table 2 
Crude incidence of outcomes following lumbar spine fusion by rod material.

View this table:
[Table 3](http://ijssurgery.com//content/15/2/251/T3)

Table 3 
Adjusteda association between rod material and outcomes following lumbar spine fusion.

By 90 days postoperative, there were 16 (10.4%) ED visits, 4 (2.6%) readmissions, and no complications for the PEEK group. Respective frequencies for the Ti group were 422 (14.8%), 217 (7.6%), and 35 (1.2%). Incidence in the matched Ti group is reported in Table 2. After the application of propensity score matching, a lower likelihood for readmission was observed for the PEEK group (OR = 0.34, 95% CI = 0.13–0.94) (Table 3). No difference was observed in likelihood for ED visit when comparing PEEK and Ti lumbar spine fusion patients (OR = 0.88, 95% CI = 0.48–1.59).

## DISCUSSION

The results of this study did not support our main hypothesis in that, after propensity score adjustment, we did not detect a difference in risk of reoperation for ASD following lumbar spine fusion with PEEK rods as compared with Ti rods in routine clinical use across a multicenter health care network. Although PEEK rods, as an individual component of a fusion construct, are more flexible than Ti rods, the overall stiffness of an instrumented level is influenced not only by the presence of PEEK rods but also by the presence of anterior fusion, which was employed in approximately 60% of the cases of the present study. Interestingly, we also did not observe any reoperations due to nonunion in the PEEK group, complicating our ability to statistically compare this secondary outcome. In addition, we will study the subgroup of patients who underwent PEEK rod fusion with posterolateral graft only and compare this group to the patients who had use of interbody grafts to see if there is a difference in ASD rates. The present study adds to the body of clinical outcome studies of instrumented lumbar fusion using PEEK rods3 by broadly supporting their midterm effectiveness compared with Ti rods for both their safety and their effectiveness at the 5–7-year follow-up.

We did find evidence in this study to support our secondary hypothesis, that PEEK rods were associated with a difference in 90-day outcomes, specifically, PEEK rods associated with a lower likelihood of readmissions. Further, no complications were observed in the PEEK group, though event rates were too few to evaluate this outcome in adjusted analysis. As health care reimbursement for spinal procedures in the United States begins to shift from fee-for-service to 90-day episodes of care, it is important to understand the comparative effectiveness of fusion technologies in the perioperative time frame. Our findings should provide reassurance to clinicians and payers that PEEK rods are associated with fewer perioperative complications and readmissions and no observed difference in ED visits compared to Ti rods and do not appear to negatively impact outcomes within a 90-day episode of care. The reasons for return to care are not collected by the registry and merit further consideration in future research identifying reasons for return to care, readmission particularly.

Because we included “all comers” during routine clinical practice in our current study, the outcomes reported here for lumbar fusion with PEEK rods are difficult to quantitatively compare directly with previous short-term studies due to differences in study populations. For example, in the meta-analysis by Selim et al,3 interbody fusion was employed in 94.4% of the aggregated, short-term PEEK rod cases, whereas in our clinical population, interbody fusion was used in only 63% of PEEK rod cases. BMP was also routinely used in both the PEEK and the Ti rod cohorts. Thus, the 100% fusion rate for PEEK rods at 2 years in the present study compares favorably with the 100% aggregate fusion rate reported in the previous meta-analysis. We found a 1.3% incidence of reoperation for nonfusion with Ti rods, which is higher than the aggregate 100% fusion rate reported for the Ti rod group in the meta-analysis.

There are several limitations of this study we would like to highlight for the reader. As a retrospective study using data from a clinical registry, our outcomes were limited to complications and reoperations as measures of safety and effectiveness and did not include patient-reported outcomes, such as pain or Oswestry scores. In terms of the overall study design, we employed propensity score matching to develop statistically balanced groups that specifically address our a priori research hypotheses with an overall clinical population, and, as such, the study design was not well suited to post hoc subgroup analyses. Using our approach, evaluating subgroups would require completely rebuilding the propensity score matching and statistical balancing of the various subgroups to address these new subgroup hypotheses and hence was beyond the scope of our original study design. Although we have employed propensity scores to address confounding associated with unbalanced factors between the PEEK and Ti cohorts, the observational nature of the study naturally limits our ability to attribute causation to the treatment, as there may be unmeasured sources of confounding that we could not address with our study design. The limitations of this study are offset by several notable strengths, including the use of an integrated health care system's spine registry that does not rely solely on coding but instead prospectively collects detailed information from the operating surgeon on all instrumented spine procedures and manually validates outcomes to ensure accuracy of the spine registry. This allows for evaluation of PEEK rods in a real-world clinical context, a relatively large sample size, and midterm follow-up.

## CONCLUSIONS

After correcting for confounding and statistical adjustment, the present study observed no difference in the risk of reoperation for ASD between PEEK and Ti rods at midterm follow-up. Both cohorts exhibited similar safety profiles and midterm registry-based measures of clinical effectiveness. Our preestablished study design was not readily amenable to subgroup analysis to explore, for example, the role of anterior fusion. Similarly, there are unanswered questions regarding the differences in 90-day outcomes that favored the PEEK cohort and will be explored in future research. Overall, based on the clinical outcomes investigated in the present study, PEEK rods remain a reasonably safe and effective treatment option for lumbar fusion in a general patient population and real-world clinical conditions.

## Footnotes

*   **Disclosures and COI:** No external funding was received for any aspect of this work. No outside funding was obtained for this study. This study was approved by the Kaiser Permanente Institutional Review Board.

*   This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2021 ISASS

## REFERENCES

1.  1 .1.  Kurtz SM, 
    
    Kurtz SM, Lanman T. Dynamic stabilization and semirigid PEEK rods for spinal fusion. In: Kurtz SM, ed. *The PEEK Biomaterials Handbook*. 2nd ed. Oxford, England: Elsevier; 2019:281–290.
    
    

2.  2 .Highsmith JM, Tumialan LM, Rodts GE Jr.. Flexible rods and the case for dynamic stabilization. *Neurosurg Focus*. 2007;22(1):E11.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=17613229&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 

3.  3 .Selim A, Mercer S, Tang F. Polyetheretherketone (PEEK) rods for lumbar fusion: a systematic review and meta-analysis. *Int J Spine Surg*. 2018;12(2):190–200.
    
    [Abstract/FREE Full Text](http://ijssurgery.com//lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiaWpzcyI7czo1OiJyZXNpZCI7czo4OiIxMi8yLzE5MCI7czo0OiJhdG9tIjtzOjE5OiIvaWpzcy8xNS8yLzI1MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

4.  4 .Bruner HJ, Guan Y, Yoganandan N, Pintar FA, Maiman DJ, Slivka MA. Biomechanics of polyaryletherketone rod composites and titanium rods for posterior lumbosacral instrumentation. *J Neurosurg Spine*. 2010;13(6):766–772.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=21121756&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 

5.  5 .Ponnappan RK, Serhan H, Zarda B, Patel R, Albert T, Vaccaro AR. Biomechanical evaluation and comparison of polyetheretherketone rod system to traditional titanium rod fixation. *Spine J*. 2009;9(3):263–267.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/j.spinee.2008.08.002&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=18838341&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 
    
    [Web of Science](http://ijssurgery.com//lookup/external-ref?access_num=000263885900010&link_type=ISI) 

6.  6 .Turner JL, Paller DJ, Murrell CB. The mechanical effect of commercially pure titanium and polyetheretherketone rods on spinal implants at the operative and adjacent levels. *Spine*. 2010;35(21):E1076–E1082.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=20838271&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 

7.  7 .Chou WK, Chien A, Wang JL. Biomechanical analysis between PEEK and titanium screw-rods spinal construct subjected to fatigue loading. *J Spinal Disord Tech*. 2015;28(3):E121–E125.
    
    

8.  8 .Gornet MF, Chan FW, Coleman JC, et al. Biomechanical assessment of a PEEK rod system for semi-rigid fixation of lumbar fusion constructs. *J Biomech Eng*. 2011;133(8):081009. doi:[10.1115/1.4004862](http://ijssurgery.com//lookup/doi/10.1115/1.4004862)
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1115/1.4004862&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=21950902&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 

9.  9 .Athanasakopoulos M, Mavrogenis AF, Triantafyllopoulos G, Koufos S, Pneumaticos SG. Posterior spinal fusion using pedicle screws. *Orthopedics*. 2013;36(7):e951–e957.
    
    

10. 10 .De Iure F, Bosco G, Cappuccio M, Paderni S, Amendola L. Posterior lumbar fusion by peek rods in degenerative spine: preliminary report on 30 cases. *Eur Spine J*. 2012;21(suppl 1):S50–S54.
    
    

11. 11 .Colangeli S, Barbanti Brodano G, Gasbarrini A, et al. Polyetheretherketone (PEEK) rods: short-term results in lumbar spine degenerative disease. *J Neurosurg Sci*. 2015;59(2):91–96.
    
    

12. 12 .Ormond DR, Albert L Jr., Das K. Polyetheretherketone (PEEK) rods in lumbar spine degenerative disease: a case series. *Clin Spine Surg*. 2016;29(7):E371–E375.
    
    

13. 13 .Qi L, Li M, Zhang S, Xue J, Si H. Comparative effectiveness of PEEK rods versus titanium alloy rods in lumbar fusion: a preliminary report. *Acta Neurochir (Wien)*. 2013;155(7):1187–1193.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1007/s00701-013-1772-3&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=23708088&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 

14. 14 .Kreitz TM, Tarazona D, Padegimas EM, et al. Comparison of short-term outcomes after lumbar fusion between an orthopedic specialty hospital and tertiary referral center. *Spine (Phila Pa 1976)*. 2019;44(9):652–658.
    
    

15. 15 .Rubel NC, Chung AS, Wong M, et al. 90-day readmission in elective primary lumbar spine surgery in the inpatient setting: a Nationwide Readmissions Database sample analysis. *Spine (Phila Pa 1976)*. 2019;44(14):E857–E864.
    
    

16. 16 .James J. Health policy brief: Medicare hospital readmissions reduction program. *Health Aff (Millwood)*. [http://www.healthaffairs.org/healthpolicybriefs/brief.php?brief_id=102](http://www.healthaffairs.org/healthpolicybriefs/brief.php?brief_id=102). Accessed November 12, 2013.
    
    

17. 17 . Kaiser Permanente. Fast facts about Kaiser Permanente. [https://share.kaiserpermanente.org/article/fast-facts-about-kaiser-permanente](https://share.kaiserpermanente.org/article/fast-facts-about-kaiser-permanente). Accessed August 20, 2018.
    
    

18. 18 .Karter AJ, Ferrara A, Liu JY, Moffet HH, Ackerson LM, Selby JV. Ethnic disparities in diabetic complications in an insured population. *JAMA*. 2002;287(19):2519–2527.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1001/jama.287.19.2519&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=12020332&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 
    
    [Web of Science](http://ijssurgery.com//lookup/external-ref?access_num=000175566200020&link_type=ISI) 

19. 19 .Koebnick C, Langer-Gould AM, Gould MK, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. *Perm J*. 2012;16(3):37–41.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.7812/TPP/12-031&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=23012597&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 

20. 20 .Paxton EW, Kiley ML, Love R, Barber TC, Funahashi TT, Inacio MC. Kaiser Permanente implant registries benefit patient safety, quality improvement, cost-effectiveness. *Jt Comm J Qual Patient Saf*. 2013;39(6):246–252.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=23789161&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 

21. 21 .Guppy KH, Paxton EW, Harris J, Alvarez J, Bernbeck J. Does bone morphogenetic protein change the operative nonunion rates in spine fusions? *Spine (Phila Pa 1976)*. 2014;39(22):1831–1839.
    
    

22. 22 .1.  Ratliff J, 
    2.  Albert T, 
    3.  Cheng J, 
    4.  Knightly J, 
    
    Guppy KH, Harris J, Bernbeck JA, Brara HS. Impact of quality assessment on clinical practice, Kaiser Permanente. In: Ratliff J, Albert T, Cheng J, Knightly J, eds. *Quality Spine Care*. Cham, Switzerland: Springer; 2019:315–339.
    
    

23. 23 .Agency for Healthcare Research and Quality. Patient safety indicators technical specifications. [http://www.qualityindicators.ahrq.gov/modules/PSI\_TechSpec.aspx](http://www.qualityindicators.ahrq.gov/modules/PSI_TechSpec.aspx). Accessed October 31, 2017.
    
    

24. 24 .Rosenbaum PR. *Observational Studies*. New York, NY: Springer; 2009.
    
    

25. 25 .Normand ST, Landrum MB, Guadagnoli E, et al. Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. *J Clin Epidemiol*. 2001;54(4):387–398.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/S0895-4356(00)00321-8&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=11297888&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 
    
    [Web of Science](http://ijssurgery.com//lookup/external-ref?access_num=000167835100009&link_type=ISI) 

26. 26 .Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. *Multivariate Behav Res*. 2011;46(3):399–424.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1080/00273171.2011.568786&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=21818162&link_type=MED&atom=%2Fijss%2F15%2F2%2F251.atom) 
    
    [Web of Science](http://ijssurgery.com//lookup/external-ref?access_num=000291533400002&link_type=ISI) 

27. 27 .Cafri G, Wang W, Chan PH, Austin PC. A review and empirical comparison of causal inference methods for clustered observational data with application to the evaluation of the effectiveness of medical devices. *Stat Methods Med Res.* 2018. [https://doi.org/10.1177/0962280218799540](https://doi.org/10.1177/0962280218799540)