Real-World Implementation of Artificial Intelligence/Machine Learning for Managing Surgical Spine Patients at 2 Academic Health Care Systems
============================================================================================================================================

* Ghaith Habboub
* Sigurd Berven
* Christopher Ames
* Thomas Peterson
* Thomas Mroz

## Abstract

Decision-making in spine surgery is complex due to patients’ heterogeneity and complexity of spinal pathologies and the various surgical options applied to a given pathology. Artificial intelligence/machine learning algorithms provide an opportunity to improve patient selection, surgical planning, and outcomes. The purpose of this article is to present the experience and applications of in spine surgery at 2 large academic health care systems.

*   artificial intelligence
*   predictive modeling
*   spine surgery

## Background

Surgical decision-making in spine surgery is complex due to heterogeneity in patients, pathologies, and surgical options. Spine patients may present with significant variability in self-reported health status, comorbidities, and surgical approaches to care.1–6 The heterogeneity of patient presentations and surgical approaches has important implications for the cost and outcomes of care.7 

Patient-collected data are growing in both size and complexity. Consequently, clinicians are limited in their ability to consider all patient data in decision-making regarding risk and outcome assessment. Precision medicine approaches to care may empower patients and physicians to make informed decisions regarding appropriate care and hold promise to optimize outcomes of care.7 Artificial intelligence/machine learning (AI/ML) are valuable analytic techniques for the development of models to help personalize decision-making. AI/ML are tools that provide correlations between variables that may not be apparent in traditional multivariate analysis and may reduce the biases of hypothesis-driven modeling.8 In contrast, AI/ML techniques are limited by important issues including poor reporting methodologies, increased clinician burden, fairness, privacy/anonymity, explainability, and interpretability.6,9,10 Particularly for the medical domain, interpretability is critical in providing trust between the AI/ML algorithm and the health care system.11 Although similar algorithms with similar accuracy can behave differently in deployed settings, global and local interpretability methods can help in model selection and generalization.12,13 It is crucial to think about AI/ML algorithms as devices with end users and process flow. These devices require documentation, regulations, and maintenance.14,15 The regulation of ML algorithms is evolving given limited precedence.

AI/ML has been used across multiple aspects of spine surgery16 including spinal medical imaging,17,18 predicting surgical outcome,19–22 identifying patients’ characteristics in deformity patients,23,24 enhancing robotics,25,26 forecasting length of stay and discharge disposition,27–29 capturing surgeons decision-making,13 and predicting mortality and complications.23,24,30 Increasing availability of patient data through electronic health records is leading to integrated clinical prediction models for precision medicine at the point of care.31 However, the development, implementation, and maintenance of these tools can vary widely depending on institutional culture, level of patient interaction, model accuracy, and perceived trust.32 

The level of automation of AI/ML can be categorized into 3 groups: assistive, autonomous information, and autonomous decision.15 Assistive describes a device with an overlapping role between a provider and the device, where providers confirm or approve device-provided information in overlapping cases. Autonomous information provides separation between the device and the provider; the latter will act on the interpretation made by the device. The final category, autonomous decision, makes decisions without provider input (Figure 1a). Approximately one-half of medical devices using ML are assistive devices. Increasing the autonomy of these devices would require close monitoring of model performance and more comprehensive regulatory approval to deploy them. Control theory, including closed loop control, would provide the appropriate safety framework to increase autonomy without decreasing risk (Figure 1b). The closed loop control system relies on feedback from the system at agreed-upon intervals. The feedback should match with the expected performance. Differences between the expected and the observed metrics would require adjustment of the system. These adjustments can vary from automated to manual model re-training and assessing data skewness.

![Figure 1](https://www.ijssurgery.com/http://www.ijssurgery.com/content/ijss/17/S1/S11/F1.medium.gif)

[Figure 1](https://www.ijssurgery.com/content/17/S1/S11/F1)

Figure 1 
(a) Illustration of the level of autonomy by artificial intelligencey/machine learning group. (b) Illustration of the machine learning (ML) control system.

Real-world application of AI/ML for modeling outcomes, complications, and costs of care is a difficult task that has been spearheaded by Ames et al and the International Spine Study Group.33–35 The cost of care for adult spinal deformity is highly variable, and cost outliers are an important threat to the sustainability of complex spine surgery, especially in smaller hospitals. Ames et al used regression tree and random forest models to predict cost outliers in deformity surgery.36 Adult spinal deformity is characterized by significant variability regarding management approaches, complications, and outcomes. Durand et al used AI/ML-based models to predict whether adults with spinal deformity were treated operatively or non-operatively with 86% accuracy.37 Patient clustering developed with AI/ML may be useful in the classification of patients and in predicting appropriate surgical approaches and expected outcomes of care.38,39 Predicting complications is important in guiding appropriate care for patients with spinal deformity. Scheer et al used traditional statistical techniques to develop a predictive model for complication in spinal deformity with 87% accuracy.40,41 The application of AI/ML with larger datasets holds promise for more precise and patient-specific prediction tools regarding complications, outcomes, and appropriate care in spinal deformity.42,43 Model development and validation will be an iterative process with the introduction of new data points and techniques.

The successful algorithms in the industry are the algorithms that tend to run in the background with fewer user interactions. These include energy management, fraud detection, spam filter, and similar algorithms.44–46 Improving the effectiveness of AI/ML algorithms would require designing a solution as a system with appropriate control and safety logic, a transparent expected goal, a higher level of autonomy, and clear documentation of the users and maintenance team. In the current article, we present 2 real-world AI/ML applications at academic centers for augmenting and complementing surgeons’ knowledge in decision-making, patient selection, and patient optimization.

## Health Care Utilization Metrics at the Cleveland Clinic

At the Cleveland Clinic, an AI/ML tool is utilized to improve patient outcome while minimizing health care expenses. To create this tool, a real-time database was generated of over 55,970 surgical encounters between 2007 and 2022 of patients who underwent surgical intervention at the Cleveland Clinic Health Care System in Ohio and Florida. Patient outcomes, imaging, laboratory results, vitals, medications, costs, and other metrics were all considered in the analytics. From this dataset, limitations were identified for patient-reported outcomes (PROs)/quality of life measures as a pure measure for making decisions. For example, up to 10% of the patients reported fluctuations in their PROs from improving to worsening or worsening to improving during the first 12 months following surgery. PRO metric instability creates difficulty in solely relying on PRO for a visit-to-visit analysis. Consequently, we developed a health care utilization metric to complement patient performance for analysis. This metric is a linear combination of a patient’s encounters with the health care system, use of opioids, office visits, MyChart messages, physical therapy, imaging, postoperative epidural injections, etc. One purpose of this metric development was to objectify postoperative performance using a translatable and transferable language that other health systems and payers would understand. The known heterogeneity of PRO use and adoption across the nation is well documented. The health care utilization metric describes a different aspect of a patient’s recovery but does not replace PRO, with a correlation coefficient of −0.34 between utilization and PRO (Figure 2). In addition to utilization, early and delayed costs are collected.

![Figure 2](https://www.ijssurgery.com/http://www.ijssurgery.com/content/ijss/17/S1/S11/F2.medium.gif)

[Figure 2](https://www.ijssurgery.com/content/17/S1/S11/F2)

Figure 2 
Graphical representation of health care utilization and the relationship to patient-reported outcomes. Graphs a and b show utilization in relation to PROMIS. Graph c shows the utilization in relation to PROMIS and stratified by MCID. MCID, minimum clinically important difference; PROMIS, Patient-Reported Outcomes Measurement Information System; post-op, postoperative.

Three target objectives are optimized per patient before and after surgical intervention (Figure 3). The target objectives are to (1) maximize PROs at 1-year follow-up, (2) minimize health care utilization, and (3) minimize costs. As opposed to usual classification tasks, multiobjective optimization adds additional hyperparameters to be selected, which are the important aspects of the 3 objectives relative to each other. Optimization happens at 3 levels (patient, surgeon, and hospital), with each level having specified modifiable features. Decision-making is based on a search algorithm of the simulated outcome under varying circumstances using causal inference logic as described by Judea Pearl in his book on causality.31 The current system is an assistive device to complement surgeon input. The autonomy of the system can increase to assist with insurance approval and medical preoperative optimization.

![Figure 3](https://www.ijssurgery.com/http://www.ijssurgery.com/content/ijss/17/S1/S11/F3.medium.gif)

[Figure 3](https://www.ijssurgery.com/content/17/S1/S11/F3)

Figure 3 
Illustration of the optimization objectives and the level of optimization. PRO, patient-reported outcome.

## Predictive Modeling for Surgical Interventions at UCSF

The assessment and development of AI/ML tools in the University of California, San Francisco (UCSF) Spine Service are a major focus of UCSF’s clinical research initiatives. Accurate information regarding the length of stay and discharge disposition has important implications regarding patient counseling and hospital budgeting. An assistive AI/ML tool for the accurate preoperative identification of patients at risk for extended length of stay after surgery can provide substantial benefits, including more transparent communication on expected benefits and risks of surgery, postoperative planning, cost savings, pre-emptive administrative action, and optimization of modifiable patient risk factors.47–49 Internal work for adopting predictive models led to a comparison of results for commonly used AI/ML prediction tools: the Risk Assessment and Prediction Tool (RAPT) score50,51 and the American College of Surgeon’s (ACS) National Surgical Quality Improvement Program (NSQIP).29 The RAPT score has been used to predict patient outcomes following surgery. It is a cumulative scaled score ranging from 1 to 12 and is composed of components that correspond to patient community support, the extent of home care, gait aid, and preoperative functional ability.50,51 While the RAPT score can be calculated by hand, other tools such as the ACS NSQIP require an online calculator, which utilizes 21 manually input preoperative factors to predict both length of stay and discharge status. To compare these tools, we selected a subset of adult elective spinal fusion patients previously described in Arora et al that had available RAPT scores (1251 patients) or ACS NSQIP scores (420 patients); 140 patients had both scores.29 For predicting patient-specific length of stay in the hospital after surgery, the Pearson’s correlation for ASC NSQIP was 0.461 (*P* < 2e-16; Figure 4A) and for the RAPT score was −0.361 (*P* < 2e-16; Figure 4B). Although both of these scores were significantly correlated, the ACS NSQIP was significantly more correlated with the true length of stay than the RAPT score (Fisher’s r-to-z transformation; *P* = 0.0192). However, comparing ACS NSQIP to RAPT directly, Pearson’s correlation was −0.286, meaning that one patient can have very different scores using different AI/ML models (Figure 4C). This indicates that more work should be done to properly understand how and for what types of patients these models differ, which could potentially be used to improve care.

![Figure 4](https://www.ijssurgery.com/http://www.ijssurgery.com/content/ijss/17/S1/S11/F4.medium.gif)

[Figure 4](https://www.ijssurgery.com/content/17/S1/S11/F4)

Figure 4 
Graphical representation of Pearson’s correlation with the true length of stay for patients scored with the American College of Surgeon’s (ACS) National Surgical Quality Improvement Program (NSQIP; *r* = 0.461, *P* < 2e-16; A) and the Risk Assessment and Prediction Tool (RAPT) score (*r* = −0.364, *P* < 2e-16; B). When comparing ACS NSQIP to RAPT, the models were found to be correlated but highly divergent on a patient-by-patient basis (*r* = −0.286, *P* < 2e-16; C).

## Discussion

In this study, we report real-world applications of AI/ML-based tools for managing surgical spine patients at 2 academic health care systems and identify major pitfalls. First, using data from the Cleveland Clinic, we demonstrate the need for a health care utilization metric when utilizing AI/ML tools since PRO data alone were shown to be insufficient for studying patient outcomes. This information is then used for optimizing patient outcomes for overall cost in addition to traditional metrics. Finally, using data from the UCSF Spine Service, we demonstrate that existing tools for predicting patient-specific outcomes are often inaccurate and can be in disagreement depending on the tool utilized. Thus, more research is needed to optimize how and which AI/ML tools for predicting outcomes are implemented in new clinical settings.

The application of predictive models and algorithms based upon AI/ML offers promise to improve the management of patients in spine clinics and inpatient settings. Accurate models will empower patients and surgeons to make informed choices regarding optimal care pathways. For health care systems, accurate models will also guide appropriate resource utilization and improve sustainability in providing cost-effective care for patients. However, careful consideration must be made to provide meaningful insights for clinicians. Moreover, the effective adoption and application of AI/ML-based algorithms will require ongoing validation within and between health care systems.

## Footnotes

*    **Funding**  The authors received no financial support for the research, authorship, and/or publication of this article.

*    **Declaration of Conflicting Interests**  The authors report no conflicts of interest in this work.

*    **Disclosures**  Sigurd Berven reports consulting fees for Medtronic, Globus, Innovasis, Camber, Accelus, and Kuros; royalties from Medtronic, Stryker, and Elsevier; support for attending meetings/travel from AO Spine; Participation on a data safety monitoring board or advisory board for Zimmer; and stock/stock options from Globus, Propio, GreenSun, and Novapproach. Christopher Ames reports royalties/licenses from Stryker, DePuy Synthes, Next Orthosurgical, Medicrea, Biomet Zimmer Spine, Nuvasive, and K2M; consulting fees from DePuy Synthes, Medtronic, Medicrea, K2M, Agada Medical, and Carlsmed; leadership/fiduciary roles in ISSG (Executive Committee), Operative Neurosurgery, Neurospine (Editorial Board), SRS Safety and Value Committee (Chair), and Global Spinal Analytics (Director); research interests in Titan Spine, Depuy Synthes, and ISSG; and grant funding from SRS. Thomas Mroz reports royalties/licenses from Stryker; payment for serving as faculty from AO Spine; participation on a data safety monitoring board or advisory board for Medtronic (Medtronic TLIF Study); and leadership role for the *Global Spine Journal* (Deputy Editor). The remaining authors have nothing to report.

*   This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2023 ISASS. To see more or order reprints or permissions, see [http://ijssurgery.com](http://ijssurgery.com).

## References

1.  1. Nayak NR , Stephen JH , Piazza MA , Obayemi AA , Stein SC , Malhotra NR . Quality of life in patients undergoing spine surgery: systematic review and meta-analysis. Global Spine Journal. 2019;9(1):67–76. doi:10.1177/2192568217701104 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1177/2192568217701104&link_type=DOI) 

2.  2. Mroz TE , Lubelski D , Williams SK , et al . Differences in the surgical treatment of recurrent lumbar disc herniation among spine surgeons in the United States. The Spine Journal. 2014;14(10):2334–2343. doi:10.1016/j.spinee.2014.01.037 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.spinee.2014.01.037&link_type=DOI) 
    
    [PubMed](https://www.ijssurgery.com/lookup/external-ref?access_num=24462813&link_type=MED&atom=%2Fijss%2F17%2FS1%2FS11.atom) 

3.  3. Lubelski D , Williams SK , O’Rourke C , et al . Differences in the surgical treatment of lower back pain among spine surgeons in the United States. Spine. 2016;41(11):978–986. doi:10.1097/BRS.0000000000001396 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1097/BRS.0000000000001396&link_type=DOI) 
    
    [PubMed](https://www.ijssurgery.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F17%2FS1%2FS11.atom) 

4.  4. Cooper ME , Torre-Healy LA , Alentado VJ , et al . Heterogeneity of reporting outcomes in the spine surgery literature. Clin Spine Surg. 2018;31(4):E221–E229. doi:10.1097/BSD.0000000000000578 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1097/BSD.0000000000000578&link_type=DOI) 

5.  5. Lubelski D , Alentado VJ , Williams SK , et al . Variability in surgical treatment of spondylolisthesis among spine surgeons. World Neurosurg. 2018;111:e564–e572. doi:10.1016/j.wneu.2017.12.108 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.wneu.2017.12.108&link_type=DOI) 

6.  6. Groot O , Schwab JH , Kapoor N , Bongers M . 82. poor reporting of methods and performance measures by machine learning studies in orthopedic surgery: a systematic review. The Spine Journal. 2021;21(9):S40. doi:10.1016/j.spinee.2021.05.107 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.spinee.2021.05.107&link_type=DOI) 

7.  7. Alvin MD , Lubelski D , Alam R , et al . Spine surgeon treatment variability: the impact on costs. Global Spine Journal. 2018;8(5):498–506. doi:10.1177/2192568217739610 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1177/2192568217739610&link_type=DOI) 

8.  8. Habboub G , Grabowski MM , Kelly ML , Benzel EC . The embedded biases in hypothesis testing and machine learning. Neurosurg Focus. 2020;48(5):2020.2.FOCUS191016. doi:10.3171/2020.2.FOCUS191016 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.3171/2020.2.FOCUS191016&link_type=DOI) 

9.  9. McGreevey JD , Mallozzi CP , Perkins RM , Shelov E , Schreiber R . Reducing alert burden in electronic health records: state of the art recommendations from four health systems. Appl Clin Inform. 2020;11(01):001–012. doi:10.1055/s-0039-3402715 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1055/s-0039-3402715&link_type=DOI) 

10. 10. Vellido A . Societal issues concerning the application of artificial intelligence in medicine. Kidney Dis. 2019;5(1):11–17. doi:10.1159/000492428 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1159/000492428&link_type=DOI) 

11. 11. Hakkoum H , Abnane I , Idri A . Interpretability in the medical field: a systematic mapping and review study. Applied Soft Computing. 2022;117:108391. doi:10.1016/j.asoc.2021.108391 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.asoc.2021.108391&link_type=DOI) 

12. 12. Molnar C.  Interpretable Machine Learning A Guide for Making Black Box Models Explainable. [https://christophm.github.io/interpretable-ml-book/](https://christophm.github.io/interpretable-ml-book/) 
    
    

13. 13. Habboub G , Tish S , Chakravarthy V , Mroz T , Ghogawala Z , Benzel E . Interpretable artificial swarm intelligence in complex neurosurgical decision making: part 1. World Neurosurg. 2020;141:518–520. doi:10.1016/j.wneu.2020.07.005 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.wneu.2020.07.005&link_type=DOI) 

14. 14. Gerke S , Babic B , Evgeniou T , Cohen IG . The need for a system view to regulate artificial intelligence/machine learning-based software as medical device. Npj Digit Med. 2020;3(1). doi:10.1038/s41746-020-0262-2 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1038/s41746-020-0262-2&link_type=DOI) 

15. 15. Lyell D , Coiera E , Chen J , Shah P , Magrabi F . How machine learning is embedded to support clinician decision making: an analysis of FDA-approved medical devices. BMJ Health Care Inform. 2021;28(1):e100301. doi:10.1136/bmjhci-2020-100301 
    
    [Abstract/FREE Full Text](https://www.ijssurgery.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYm1qaGNpIjtzOjU6InJlc2lkIjtzOjEyOiIyOC8xL2UxMDAzMDEiO3M6NDoiYXRvbSI7czoyMDoiL2lqc3MvMTcvUzEvUzExLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

16. 16. Peterson TA , Krogue JB . Precision medicine: artificial intelligence applied to evaluation and management of spinal disorders. Complication Avoidance, and Management. 2021:1561–1566.
    
    

17. 17. Ahmadian H , Mageswaran P , Walter BA , et al . Toward an artificial intelligence. Int j Numer Method Biomed Eng. 2022;38. doi:10.1002/cnm.3601 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1002/cnm.3601&link_type=DOI) 

18. 18. Lee GW , Shin H , Chang MC . Deep learning algorithm to evaluate cervical spondylotic myelopathy using lateral cervical spine radiograph. BMC Neurol. 2022;22(1):147. doi:10.1186/s12883-022-02670-w 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1186/s12883-022-02670-w&link_type=DOI) 

19. 19. Ghogawala Z , Dunbar M , Essa I . Artificial intelligence for the treatment of lumbar spondylolisthesis. Neurosurg Clin N Am. 2019;30(3):383–389. doi:10.1016/j.nec.2019.02.012 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.nec.2019.02.012&link_type=DOI) 

20. 20. Mourad R , Kolisnyk S , Baiun Y , et al . Performance of hybrid artificial intelligence in determining candidacy for lumbar stenosis surgery. Eur Spine J. 2022;31(8):2149–2155. doi:10.1007/s00586-022-07307-7 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s00586-022-07307-7&link_type=DOI) 

21. 21. Wirries A , Geiger F , Hammad A , Oberkircher L , Blümcke I , Jabari S . Artificial intelligence facilitates decision-making in the treatment of lumbar disc herniations. Eur Spine J. 2021;30(8):2176–2184. doi:10.1007/s00586-020-06613-2 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s00586-020-06613-2&link_type=DOI) 

22. 22. Xie N , Wilson PJ , Reddy R . Use of machine learning to model surgical decision-making in lumbar spine surgery. Eur Spine J. 2022;31(8):2000–2006. doi:10.1007/s00586-021-07104-8 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s00586-021-07104-8&link_type=DOI) 

23. 23. Han SS , Azad TD , Suarez PA , Ratliff JK . A machine learning approach for predictive models of adverse events following spine surgery. Spine J. 2019;19(11):1772–1781. doi:10.1016/j.spinee.2019.06.018 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.spinee.2019.06.018&link_type=DOI) 

24. 24. Rudisill SS , Hornung AL , Barajas JN , et al . Artificial intelligence in predicting early-onset adjacent segment degeneration following anterior Cervical Discectomy and fusion. Eur Spine J. 2022;31(8):2104–2114. doi:10.1007/s00586-022-07238-3 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s00586-022-07238-3&link_type=DOI) 

25. 25. Rasouli JJ , Shao J , Neifert S , et al . Artificial intelligence and robotics in spine surgery. Global Spine Journal. 2021;11(4):556–564. doi:10.1177/2192568220915718 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1177/2192568220915718&link_type=DOI) 

26. 26. Liounakos JI , Chenin L , Theodore N , Wang MY . Robotics in spine surgery and spine surgery training. Operative Surg. 2021;21(2):35–40. doi:10.1093/ons/opaa449 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1093/ons/opaa449&link_type=DOI) 

27. 27. Ogink PT , Karhade AV , Thio QCBS , et al . Predicting discharge placement after elective surgery for lumbar spinal stenosis using machine learning methods. Eur Spine J. 2019;28(6):1433–1440. doi:10.1007/s00586-019-05928-z 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s00586-019-05928-z&link_type=DOI) 

28. 28. Valliani AA , Feng R , Martini ML , et al . Pragmatic prediction of excessive length of stay after cervical spine surgery with machine learning and validation on a national scale. Neurosurgery. 2022;91(2):322–330. doi:10.1227/neu.0000000000001999 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1227/neu.0000000000001999&link_type=DOI) 

29. 29. Arora A , Lituiev D , Jain D , et al . Predictive models for length of stay and discharge disposition in elective spine surgery: development validation, and comparison to the ACS NSQIP risk calculator. Spine (Phila Pa 1976). 2023;48(1):E1–E13. doi:10.1097/BRS.0000000000004490 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1097/BRS.0000000000004490&link_type=DOI) 

30. 30. Karhade AV , Thio QCBS , Ogink PT , et al . Predicting 90-day and 1-year mortality in spinal metastatic disease: development and internal validation. Neurosurg. 2019;85(4):E671–E681. doi:10.1093/neuros/nyz070 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1093/neuros/nyz070&link_type=DOI) 

31. 31. Karhade AV , Schwab JH , Del Fiol G , Kawamoto K . SMART on FHIR in spine: integrating clinical prediction models into electronic health records for precision medicine at the point of care. The Spine Journal. 2021;21(10):1649–1651. doi:10.1016/j.spinee.2020.06.014 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.spinee.2020.06.014&link_type=DOI) 

32. 32. Watson J , Hutyra CA , Clancy SM , et al . Overcoming barriers to the adoption and implementation of predictive modeling and machine learning in clinical care: what can we learn from US academic medical centers JAMIA Open. 2020;3(2):167–172. doi:10.1093/jamiaopen/ooz046 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1093/jamiaopen/ooz046&link_type=DOI) 

33. 33. Joshi RS , Lau D , Ames CP . Artificial intelligence and the future of spine surgery. Neurospine. 2019;16(4):637–639. doi:10.14245/ns.1938410.205 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.14245/ns.1938410.205&link_type=DOI) 

34. 34. Joshi RS , Lau D , Scheer JK , et al . State-of-the-art reviews predictive modeling in adult spinal deformity: applications of advanced analytics. Spine Deform. 2021;9(5):1223–1239. doi:10.1007/s43390-021-00360-0 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s43390-021-00360-0&link_type=DOI) 

35. 35. Mallow GM , Siyaji ZK , Galbusera F , et al . Intelligence-based spine care model: a new era of research and clinical decision-making. Global Spine Journal. 2021;11(2):135–145. doi:10.1177/2192568220973984 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1177/2192568220973984&link_type=DOI) 

36. 36. Ames CP , Smith JS , Gum JL , et al . Utilization of predictive modeling to determine episode of care costs and to accurately identify catastrophic cost nonwarranty outlier patients in adult spinal deformity surgery: a step toward bundled payments and risk sharing. Spine. 2020;45(5):E252–E265. doi:10.1097/BRS.0000000000003242 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1097/BRS.0000000000003242&link_type=DOI) 

37. 37. Durand WM , Daniels AH , Hamilton DK , et al . Artificial intelligence models predict operative versus nonoperative management of patients with adult spinal deformity with 86% accuracy. World Neurosurg. 2020;141:e239–e253. doi:10.1016/j.wneu.2020.05.099 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.wneu.2020.05.099&link_type=DOI) 

38. 38. Ames CP , Smith JS , Pellisé F , et al . Artificial intelligence based hierarchical clustering of patient types and intervention categories in adult spinal deformity surgery: towards a new classification scheme that predicts quality and value. Spine (Phila Pa 1976). 2019;44(13):915–926. doi:10.1097/BRS.0000000000002974 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1097/BRS.0000000000002974&link_type=DOI) 

39. 39. Durand WM , Lafage R , Hamilton DK , et al . Artificial intelligence clustering of adult spinal deformity sagittal plane morphology predicts surgical characteristics, alignment, and outcomes. Eur Spine J. 2021;30(8):2157–2166. doi:10.1007/s00586-021-06799-z 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s00586-021-06799-z&link_type=DOI) 

40. 40. Scheer JK , Osorio JA , Smith JS , et al . Development of validated computer-based preoperative predictive model for proximal junction failure (PJF) or clinically significant PJK with 86% accuracy based on 510 ASD patients with 2-year follow-up. Spine (Phila Pa 1976). 2016;41(22):E1328–E1335. doi:10.1097/BRS.0000000000001598 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1097/BRS.0000000000001598&link_type=DOI) 

41. 41. Scheer JK , Smith JS , Schwab F , et al . Development of a preoperative predictive model for major complications following adult spinal deformity surgery. J Neurosurg Spine. 2017;26(6):736–743. doi:10.3171/2016.10.SPINE16197 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.3171/2016.10.SPINE16197&link_type=DOI) 

42. 42. Joshi RS , Lau D , Ames CP . Artificial intelligence for adult spinal deformity: current state and future directions. Spine J. 2021;21(10):1626–1634. doi:10.1016/j.spinee.2021.04.019 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.spinee.2021.04.019&link_type=DOI) 

43. 43. Joshi RS , Haddad AF , Lau D , Ames CP . Artificial intelligence for adult spinal deformity. Neurospine. 2019;16(4):686–694. doi:10.14245/ns.1938414.207 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.14245/ns.1938414.207&link_type=DOI) 

44. 44. Nakabi TA , Toivanen P . Deep reinforcement learning for energy management in a microgrid with flexible demand. Sustainable Energy, Grids and Networks. 2021;25:100413. doi:10.1016/j.segan.2020.100413 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.segan.2020.100413&link_type=DOI) 

45. 45. Zekić-Sušac M , Mitrović S , Has A . Machine learning based system for managing energy efficiency of public sector as an approach towards smart cities. International Journal of Information Management. 2021;58:102074. doi:10.1016/j.ijinfomgt.2020.102074 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/j.ijinfomgt.2020.102074&link_type=DOI) 

46. 46. Bin Sulaiman R , Schetinin V , Sant P . Review of machine learning approach on credit card fraud detection. Hum-Cent Intell Syst. 2022;2(1–2):55–68. doi:10.1007/s44230-022-00004-0 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1007/s44230-022-00004-0&link_type=DOI) 

47. 47. Maitra S , Mikhail C , Cho SK , Daubs MD . Preoperative maximization to reduce complications in spinal surgery. Global Spine Journal. 2020;10(1_suppl):45S–52S. doi:10.1177/2192568219882349 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1177/2192568219882349&link_type=DOI) 

48. 48. Zambouri A . Preoperative evaluation and preparation for anesthesia and surgery. Hippokratia. 2007;11(1):13–21.
    
    [PubMed](https://www.ijssurgery.com/lookup/external-ref?access_num=19582171&link_type=MED&atom=%2Fijss%2F17%2FS1%2FS11.atom) 

49. 49. Menendez JY , Omar NB , Chagoya G , et al . Patient satisfaction in spine surgery: a systematic review of the literature. Asian Spine J. 2019;13(6):1047–1057. doi:10.31616/asj.2019.0032 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.31616/asj.2019.0032&link_type=DOI) 

50. 50. Oldmeadow LB , McBurney H , Robertson VJ . Predicting risk of extended inpatient rehabilitation after hip or knee arthroplasty. J Arthroplasty. 2003;18(6):775–779. doi:10.1016/s0883-5403(03)00151-7 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1016/S0883-5403(03)00151-7&link_type=DOI) 
    
    [PubMed](https://www.ijssurgery.com/lookup/external-ref?access_num=14513453&link_type=MED&atom=%2Fijss%2F17%2FS1%2FS11.atom) 
    
    [Web of Science](https://www.ijssurgery.com/lookup/external-ref?access_num=000185572500015&link_type=ISI) 

51. 51. Sconza C , Respizzi S , Grappiolo G , Monticone M . The risk assessment and prediction tool (rapt) after hip and knee replacement: a systematic review. Joints. 2019;7(2):41–45. doi:10.1055/s-0039-1693459 
    
    [CrossRef](https://www.ijssurgery.com/lookup/external-ref?access_num=10.1055/s-0039-1693459&link_type=DOI)