Clinical StudyFocus: Artificial Intelligence and Machine LearningUpdated external validation of the SORG machine learning algorithms for prediction of ninety-day and one-year mortality after surgery for spinal metastasis
Introduction
The spinal column is the most common site of bony metastasis, with approximately 5% to 10% of patients with cancer developing metastatic epidural compression [1]. As survival rates of many malignancies increase in an aging population, the prevalence of spinal metastatic disease is similarly expected to increase. After a prospective study by Patchell and colleagues in 2005 demonstrated the superiority of surgical decompression with radiotherapy versus radiotherapy alone, the rate of surgery for spinal metastases has also increased [2,3]. Surgical decompression and stabilization are typically performed to alleviate pain from mechanical instability or to relieve neurologic compromise [4]. Yet spinal surgery in this patient population is not without risk. Complications associated with surgery for spinal metastatic disease are a significant source of morbidity and mortality and include wound infections, neurologic impairment, venous thromboembolism, and instrumentation failure [5,6]. Patients with metastatic disease generally have multiple medical comorbidities with associated immunosuppression and are less likely to recover from postoperative complications.
Given these considerations, the benefits of alleviating pain and improving ambulation must be weighed against the risk of complications. It is thus important to obtain accurate short-term and long-term prognoses when determining a treatment plan. Unfortunately, physicians’ clinical prediction for life expectancy in the setting of metastatic cancer is often not accurate [[7], [8], [9]]. Scoring systems and prognostic calculators in the literature largely fail to estimate survival probability at different time points and perform inconsistently across different tumor types [4,[10], [11], [12], [13], [14], [15], [16], [17], [18], [19]].
SORG previously developed machine learning (ML) algorithms to preoperatively predict risk of 90-day and 1-year mortality in patients undergoing surgery for spinal metastasis; these have been made available in an open access web application [20]. These algorithms are accurate with high discriminatory capability and are internally valid. For these tools to be of clinical utility, however, they must be externally validated in multiple independent populations. We aim to evaluate the performance of these algorithms on an independent contemporary patient cohort from a geographically distinct institution and assess their external validity.
Section snippets
Patient sample
Our institutional review board approved retrospective review of electronic medical records. Individual patient consent was waived as the study was limited to retrospective review of medical charts. Inclusion criteria for the study were: (1) age greater than 18 years; (2) initial decompression and/or fusion for spinal metastatic disease; (3) surgery between 2004 and 2020. Patients who underwent surgical intervention for primary spinal tumors as well as those who had revision procedures for
Results
Overall, 298 patients underwent surgery for spinal metastatic disease. Two hundred and twelve (71.1%) were treated in 2010 or later, and 124 (41.6%) were treated since 2015. The majority of patients (57.4%) were male. The median age was 61 years. With respect to primary tumor histology groups, 34.9% were slow growth, 38.9% were moderate growth, and 26.2% were rapid growth. One hundred and four patients (35.0%) had presence of lung or liver metastasis and 23 patients (7.8%) had metastatic
Discussion
With the increasing prevalence of malignancy in the aging population and advances in medical and radiation oncology, the incidence of spinal metastatic disease is likely to increase. Expected postoperative survival is a critical consideration when weighing the risks and benefits of operative intervention for spinal metastasis. There have been multiple scoring systems proposed to predict survival of patients and determine appropriateness of surgery, yet there remains little consensus as to which
Declarations of Competing Interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (43)
- et al.
Direct decompressive surgical resection in the treatment of spinal cord compression caused by metastatic cancer: a randomised trial
Lancet
(2005) - et al.
Trends in the surgical treatment for spinal metastasis and the in-hospital patient outcomes in the United States from 2000 to 2009
Spine J
(2014) - et al.
Predicting survival for metastatic spine disease: a comparison of nine scoring systems
Spine J
(2018) - et al.
Surgical site infection in spinal metastasis: incidence and risk factors
Spine J
(2018) - et al.
How accurate are physicians’ clinical predictions of survival and the available prognostic tools in estimating survival times in terminally ill cancer patients? A systematic review
Clin Oncol (R Coll Radiol)
(2001) - et al.
Reassessment of scoring systems and prognostic factors for metastatic spinal cord compression
Spine J
(2015) - et al.
Tokuhashi score and other prognostic factors in 260 patients with surgery for vertebral metastases
Orthop Traumatol Surg Res
(2015) - et al.
A novel risk calculator to predict outcome after surgery for symptomatic spinal metastases; use of a large prospective patient database to personalise surgical management
Eur J Cancer
(2019) - et al.
Modeling 1-year survival after surgery on the metastatic spine
Spine J
(2015) - et al.
Are allogeneic blood transfusions associated with decreased survival after surgical treatment for spinal metastases?
Spine J
(2016)
External validation of the SORG 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease
Spine J
Does the SORG algorithm generalize to a contemporary cohort of patients with spinal metastases on external validation
Spine J
Geographic variation in the surgical management of lumbar spondylolisthesis: characterizing practice patterns and outcomes
Spine J
Substantial effective sample sizes were required for external validation studies of predictive logistic regression models
J Clin Epidemiol
Diagnosis and management of metastatic spine disease: a review
J Neurosurg Spine
Risk factors for wound-related reoperations in patients with metastatic spine tumor
J Neurosurg Spine
The relative accuracy of the clinical estimation of the duration of life for patients with end of life cancer
Cancer
How accurate are physicians in the prediction of patient survival in advanced lung cancer?
Oncologist
External validity of the Tokuhashi score in patients with vertebral metastasis
J Cancer Res Clin Oncol
Cited by (21)
Development and external validation of predictive algorithms for six-week mortality in spinal metastasis using 4,304 patients from five institutions
2022, Spine JournalCitation Excerpt :We previously developed algorithms for prediction of 3-month and 1-year mortality in patients undergoing surgery for spinal metastatic disease [28]. These algorithms have been externally both domestically and internationally [29–32]. Existing studies for shorter postoperative survival suffer from a lack of disease-specific variables such as primary tumor histology, relatively low sample size, or lack of assessment of prediction model metrics such as calibration, decision curve analysis, and Brier score [33–35].
Prognostic significance of lab data and performance comparison by validating survival prediction models for patients with spinal metastases after radiotherapy
2022, Radiotherapy and OncologyCitation Excerpt :SORG-MLA exhibited the best performance among the three SPMs across all performance metrics, including discrimination, calibration, and decision curve analyses. The performance was in accordance with that observed during previous external validations, and this is the first study to support the use of SORG-MLA for patients receiving radiotherapy [30–33]. Compared with the other two competing SPMs, SORG-MLA has some advantages.
A machine learning algorithm for predicting prolonged postoperative opioid prescription after lumbar disc herniation surgery. An external validation study using 1,316 patients from a Taiwanese cohort
2022, Spine JournalCitation Excerpt :Furthermore, decision-curve analysis shows this model is suitable for clinical use. External validation is an often-overlooked element in the process of incorporating prediction models in current clinical practice [44,46–48,51–58]. While there has been an abundance of ML prediction models in orthopedics as of late [47,59–62], a study by Groot et al [63].
FDA device/drug status: Not applicable.
Author Disclosures: AAS: Nothing to disclose. AVK: Nothing to disclose. HYP: Nothing to disclose. WLS: Nothing to disclose. LJM: Nothing to disclose. RGE: Nothing to disclose. ANS: Nothing to disclose. DYP: Consluting: Seaspine (C), Globus (B), Nuvasive (B), Stryker (C). JHS: Scientific Advisory Board: Chordoma Foundation (Nonfinancial); Speaking and/or Teaching Arrangements: AO Spine (Travel Expense Reimbursement, Outside 12-Month Requirement), Stryker Spine (B, Outside 12-Month Requirement). FJH: Nothing to disclose.
Funding disclosure: There was no financial support for this study.