Analysis of the Quality, Reliability, and Educational Content of YouTube Videos Concerning Spine Tumors
=======================================================================================================

* Michelle A. Richardson
* Won Park
* David N. Bernstein
* Addisu Mesfin

## Abstract

**Background** Given the high volume of user traffic to YouTube, it is important that the medical information disseminated on this platform is of high quality. Unfortunately, previous studies have demonstrated this to not be the case. We aimed to evaluate the quality and educational content of YouTube videos concerning spine tumors using 2 previously validated assessment tools.

**Methods** The first 50 videos returned by the keyword search “spine tumor” were included in the study. The *JAMA* benchmark criteria (range: 0–4) were used to assess video reliability, whereas the Global Quality Score (GQS) (range: 0–5) was used to determine educational quality and content.

**Results** Videos were primarily authored by academic medical groups (80%), while content was primarily related to disease-specific information (44%) or the patient experience (24%). Surgical treatment options and nonsurgical management were discussed in 66% and 50% of all videos, respectively. Sixty percent of videos reported benefits of treatment, while 44% reported potential risks or complications. The average *JAMA* score and GQS were 3.1 ± 0.27 (range: 3–4) and 2.6 ± 1.3 (range: 1–5), respectively. Multivariate linear regression analyses revealed that video duration (β = 0.00697, *P* = 0.04) and number of views (β = 0.000018, *P* = 0.001) were positively associated with *JAMA* score. Video duration and number of dislikes were associated with higher GQS (β = 0.041, *P* = 0.025) and lower GQS (β = −0.189, *P* = 0.04), respectively. Lastly, number of days since upload was associated with lower Video Power Index (β = −0.003, *P* = 0.003).

**Conclusions** The reliability, quality, and educational content of YouTube videos were poor to suboptimal. Physicians should be wary of the education provided by YouTube on spine tumors and guide patients in seeking out additional sources of information.

**Clinical Relevance** YouTube videos are commonly viewed by patients seeking health information on spine tumors. While certain videos may provide useful information, the absence of an editorial process allows videos with poor reliability and low quality to be uploaded. We believe these findings may be useful to physicians seeking ways to better guide their patients with the most appropriate educational tools throughout their disease management.

*   metastasis
*   patient education
*   quality
*   reliability
*   spine tumor
*   YouTube

## INTRODUCTION

Spine tumors can be classified as either primary or metastatic tumors. Primary spine tumors originate within the spine, whereas metastatic spine tumors are due to the spread of cancerous lesions in other tissues.1 Primary spinal tumors (extradural, intradural extramedullary, and intradural intramedullary) include schwannomas, meningiomas, ependymomas, astrocytomas, chordomas, hemangioblastomas, osteosarcomas, osteoid osteomas, and neurofibromas.2–4 Although primary spine tumors are rare, metastatic spine tumors are a common manifestation of many different types of cancers, namely of the lung, breast, prostate, kidney, and gastrointestinal tract.1,5 Estimates of the rate of spinal metastasis range from 50% to 70% of all cancer patients.1 Patients with spine tumors most commonly present with back or neck pain with a slow onset that gradually worsens until it finally persists through the night and at rest.1 Metastatic tumors can result in pathologic fractures and epidural compression leading to neurological deficits, radiculopathy, and instability.1,6,7 Furthermore, complications from metastatic spine surgery are common and are most frequently related to surgical wounds.8,9 

As with patients with various diagnoses, patients diagnosed as having a spine tumor often turn to the internet to learn more about their disease. In fact, a survey conducted by the Pew Research Center found that 90% of American adults utilized the internet in 2019.10 This was an enormous increase from 2000, when internet use among US adults was only at 52%.10 More importantly, a separate study conducted in 2013 found that 59% of US adults used the internet specifically for health information.11 YouTube, a site frequently visited by internet users, has many videos that provide health information for a variety of spine conditions. The site currently has more than 2 billion users, and more than 1 billion hours of video are watched every day.12 Due to the high volume of user traffic to YouTube, it is of paramount importance that the medical information disseminated by the site be of high quality in order to avoid misinformation. Unfortunately, previous studies have demonstrated this to not be the case.13–17 

While quality-based analyses of YouTube content involving the spine have been conducted for kyphosis14 and scoliosis,15 there is a paucity of data for videos regarding spine tumors. The objective of the present study was to evaluate the overall quality, reliability, and educational content of YouTube videos related to spine tumors using 2 previously validated assessment tools.

## METHODS

Institutional review board approval was not required as only publicly available data were used. The first 50 YouTube videos identified by the keyword search “spine tumor” on 7 July 2020 were included in the study. The number of videos evaluated was determined by previously published orthopedic YouTube studies.13–17 

The following video characteristics were recorded for each video: (1) video title, (2) duration, (3) number of views, (4) number of likes, (5) number of dislikes, (6) number of comments, (7) days since upload, (8) like ratio (no. of likes/[no. of likes + no. of dislikes] × 100), (9) view ratio (no. of views/day), (10) the Video Power Index (VPI), a measure of video popularity (like ratio × view ratio/100), (11) gender of the physician or patient, (12) race of the physician or patient, (13) authorship, (14) type of content, (15) type of tumor described, and (16) whether the tumor described was primary or metastatic. All video characteristics subject to change with time including number of views, likes, dislikes, comments, and days since upload were collected in the span of 1 day, on 7 July 2020.

Video authorship was divided into categories as described by Belayneh and Mesfin18 and included: (1) academic medical group, (2) biomedical industry, (3) private medical group, (4) insurance company, and (5) others. Video content was categorized by the following: (1) advertisement, (2) disease-specific information, (3) nonsurgical management, (4) patient experience, (5) surgical technique or approach, and (6) others.

Additionally, each video was evaluated for the inclusion of the following: (1) description of the tumor, (2) diagnosis, (3) surgical treatment options, (4) nonsurgical alternatives, (5) inclusion criteria, (6) exclusion criteria, (7) benefits of treatments, (8) complications or risks of treatment, and (9) postoperative or post-treatment course.

Two tools were used to assess the video reliability, educational content, and quality of videos. The *JAMA* benchmark criteria consist of a maximum of 4 points, with higher scores indicating greater video reliability (Table 1). The Global Quality Score (GQS) consists of a maximum of 5 points, with higher scores indicating greater educational content and video quality (Table 2). Both the *JAMA* benchmark criteria and the GQS have been used in previous literature and shown to be valid assessment tools.

View this table:
[Table 1](http://ijssurgery.com//content/16/2/278/T1)

Table 1 
The *Journal of the American Medical Association* benchmark criteria.

View this table:
[Table 2](http://ijssurgery.com//content/16/2/278/T2)

Table 2 
Global Quality Score criteria.

### Statistical Methods

Descriptive statistics were calculated and reported for all video characteristics and scores. Continuous variables were reported as means with standard deviation or ranges, while categorical variables were reported as counts with percentages. Video characteristic associations with the *JAMA* score, GQS, and VPI were determined by multivariable linear regression. Intraobserver reliability was 1.0 (95% CI 1.0–1.0) for the *JAMA* score and 0.95 (95% CI 0.87–0.98) for the GQS. All statistical analyses were performed with Stata version 16.1 (StataCorp, College Station, TX, USA). Statistical significance was set to *P* < 0.05.

## RESULTS

Fifty videos were included in the study. The mean video duration was 8.1 ± 12 minutes (range: 0.87−57 minutes), and the mean number of views was 21,629 ± 82,734 views (range: 243−582,611). Additional video characteristics are summarized in Table 3. Videos were authored by academic medical groups in 40 videos (80%), the biomedical industry in 1 video (2%), and private medical groups in 4 videos (4%) (Table 3). Content was primarily related to disease-specific information (44%) or the patient experience (24%) (Table 3). Surgical treatment options and nonsurgical management were discussed in 66% and 50% of all videos, respectively (Table 4). Sixty percent of videos reported benefits of treatment, while 44% reported potential risks or complications (Table 4).

View this table:
[Table 3](http://ijssurgery.com//content/16/2/278/T3)

Table 3 
Video characteristics of included YouTube videos.

View this table:
[Table 4](http://ijssurgery.com//content/16/2/278/T4)

Table 4 
No. (%) reporting specific information for each authorship category.

The mean ± SD *JAMA* score and GQS were 3.1 ± 0.27 (range: 3–4) and 2.6 ± 1.3 (range: 1–5), respectively. Multivariate linear regression analyses revealed that video duration (β = 0.00697, *P* = 0.04) and number of views (β = 0.000018, *P* = 0.001) were positively associated with *JAMA* score. Video duration and number of dislikes were associated with higher GQS (β = 0.041, *P* = 0.025) and lower GQS (β = −0.189, *P* = 0.04), respectively. Number of days since upload was associated with lower VPI (β = −0.003, *P* = 0.003).

## DISCUSSION

The advancement of the internet has provided new, easily accessible sources for patients seeking additional health information. While YouTube is a commonly used source for this purpose, the reliability, quality, and educational content of videos may be poor for patient use.14,19,20 It is important for healthcare providers to consider both the large online audience and lack of quality of health-related YouTube videos when caring for patients.14,21,22 Thus, we aimed to evaluate the quality, reliability, and educational content of YouTube videos concerning spine tumors. In this study of 50 YouTube videos, we (1) determined the average *JAMA* score and GQS, suggesting poor to suboptimal educational quality and reliability; (2) determined that YouTube videos often provide incomplete information, with many missing descriptions of the inclusion and exclusion criteria for treatment, potential complications, or post-treatment sequelae; and (3) demonstrated significant associations between video characteristics and the *JAMA* score, GQS, and VPI.

Overall, the reliability, quality, and educational content of YouTube videos analyzed in this study were poor to suboptimal. The GQS of 2.6 was low, yet consistent with findings in previous studies. For example, Kunze et al reported a GQS of 2.12 for a meniscus study13 and 2.3 for a posterior cruciate ligament study.16 These low scores may be explained by the lack of an editorial process for YouTube videos. Interestingly, the *JAMA* score, at 3.1, was higher than those in previous studies that have reported scores of 2.02,16 1.55,13 1.36,14 and 1.32.15 This finding may be attributed to the high percentage (80%) of videos authored by academic medical groups in the present study. Indeed, in an assessment of websites regarding scoliosis, Mather et al found that academic-affiliated sources had the highest scores for both quality and accuracy.23 Similarly, Belayneh and Mesfin found that academic websites on lateral lumbar interbody fusion were the most comprehensive of all sources studied.18 While the academic medical group had the highest mean *JAMA* score in a study by Kunze et al, only 4% of videos were authored by this group, resulting in an overall mean *JAMA* of 1.36 for all 50 videos studied. Despite the higher *JAMA* score found in this study suggesting greater reliability than those found in previous studies, patients and physicians should be aware of the poor quality and educational content these videos may contain.

The results of this study also demonstrate the incomplete information that is presented in YouTube videos. Only 32% and 22% of videos reported at least one inclusion criteria and exclusion criteria for the described treatment, respectively. Furthermore, while more than half the videos discussed surgical treatment (66%) and the benefits of treatment (60%), only 44% and 42% of videos mentioned complications or post-treatment sequelae. While these findings may be partly attributed to authors attempting to keep the videos concise, we must also consider the consequences of these uneven percentages. Indeed, patients may be biased to receiving surgery if videos disproportionately focus on the benefits of surgical treatment. Interestingly, Belayneh and Mesfin had similar findings when investigating the content of websites on lateral lumbar interbody fusions, as benefits were reported by 69% of websites, whereas complications were only reported by 36%.18 The skewed and incomplete information found in YouTube videos may be explained by considering their role in recruiting patients. This may additionally explain why the mean *JAMA* score, which includes criteria such as author credentials and affiliations, is slightly higher than the mean GQS. Including such details may have the added benefits of self-promotion. Patients should consider the incomplete nature of YouTube videos when learning about their condition, and physicians should encourage their patients to find additional, more comprehensive sources to supplement such videos.

Several associations between video characteristics and the *JAMA* score, GQS, and VPI were identified. First, we found that video duration was positively associated with both the *JAMA* score and GQS. Considering that GQS measures the quality, flow, and usefulness to patients, longer videos may be less rushed and have more opportunity to thoroughly discuss the disease in a way that is easily understood by patients. In addition, number of views was associated with higher *JAMA* score, suggesting that patients may be drawn to videos that provide information including credentials, affiliation details, and references or sources for content. The visibility of this information may provide patients with a reassurance that the education they are receiving is one that is reliable and can be trusted. Furthermore, the number of dislikes was negatively associated with GQS. This makes sense intuitively, as the highest score on the GQS indicates that the video is of “excellent quality and flow; highly useful to patients.” Thus, if the video is hard to follow and does not provide useful information to viewers, the video would acquire more dislikes. Lastly, we identified a negative association between number of days passed since video publishing date and VPI. This finding suggests that videos that have been posted more recently attract more online attention. When looking up “spine tumor” videos, patients may prefer to watch videos that have later publication dates, as they are more likely to feature the most updated information as well as latest medical advancements. More work is needed to verify whether these suggested explanations are correct.

Readers should consider several limitations when interpreting the findings presented in this study. First, both the *JAMA* score and the GQS were scored by a single, trained observer. However, both scores had excellent intraobserver reliability, which has been found to be appropriate in a previous study.13 Second, our findings do not represent the content of all YouTube videos regarding spine tumors. Nonetheless, by selecting the first 50 videos returned by the keyword search “spine tumor,” we believe we identified videos most commonly viewed by patients. Related, the active nature of the internet means that certain video characteristics, including the top 50 videos as well as the number of likes, dislikes, and comments, are constantly subject to change. We aimed to address this limitation by collecting all such variable data within a single day. Despite these limitations, we believe that our findings provide valuable information on the quality and educational content of the most commonly viewed YouTube videos regarding spine tumors.

## CONCLUSION

YouTube videos are commonly viewed by patients seeking health information on spine tumors. While certain videos may provide useful information, the absence of an editorial process allows videos with poor reliability and low quality to be uploaded. Therefore, patients should be wary of the education provided by YouTube videos on spine tumors and seek out additional sources of information. We believe these findings may be useful to physicians seeking ways to better guide their patients with the most appropriate educational tools throughout their disease management.

## Footnotes

*   **Funding** The author(s) received no financial support for the research, authorship, and/or publication of this article.

*   **Disclosures and COI** The author(s) report no conflicts of interest or financial disclosures with respect to the research, authorship, and/or publication of this article.

*   This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2022 ISASS. To see more or order reprints or permissions, see [http://ijssurgery.com](http://ijssurgery.com).

## References

1.  1. Ciftdemir M , Kaya M , Selcuk E , Yalniz E . Tumors of the spine. World J Orthop. 2016;7(2):109–116. doi:10.5312/wjo.v7.i2.109 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.5312/wjo.v7.i2.109&link_type=DOI) 

2.  2.PDQ Adult Treatment Editorial Board. Adult Central Nervous System Tumors Treatment (PDQ®). Patient Version; 2002.
    
    

3.  3. D’Amore T , Boyce B , Mesfin A . Chordoma of the mobile spine and sacrum: clinical management and prognosis. J Spine Surg. 2018;4(3):546–552. doi:10.21037/jss.2018.07.09 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.21037/jss.2018.07.09&link_type=DOI) 

4.  4. Qiu B , Joo P , Ajabnoor R , Boyce B , Mesfin A . Surgical management of aggressive hemangiomas of the spine. J Clin Neurosci. 2020;78:67–72. doi:10.1016/j.jocn.2020.06.012 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/j.jocn.2020.06.012&link_type=DOI) 

5.  5. Mesfin A , Buchowski JM , Gokaslan ZL , Bird JE . Management of metastatic cervical spine tumors. J Am Acad Orthop Surg. 2015;23(1):38–46. doi:10.5435/JAAOS-23-01-38 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.5435/JAAOS-23-01-38&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=25538129&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom) 

6.  6. Ge L , Arul K , Mesfin A . Spinal cord injury from spinal tumors: prevalence, management, and outcomes. World Neurosurg. 2019;122:e1551–e1556. doi:10.1016/j.wneu.2018.11.099 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/j.wneu.2018.11.099&link_type=DOI) 

7.  7. Mika A , Mesfin A . Update on the management of sacral metastases. JBJS Rev. 2018;6(7):e8. doi:10.2106/JBJS.RVW.17.00130 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.2106/JBJS.RVW.17.00130&link_type=DOI) 

8.  8. Williams DM , Thirukumaran CP , Oses JT , Mesfin A . Complications and mortality rates following surgical management of extradural spine tumors in New York State. Spine (Phila Pa 1976). 2020;45(7):474–482. doi:10.1097/BRS.0000000000003294 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1097/BRS.0000000000003294&link_type=DOI) 

9.  9. Mesfin A , Baldwin A , Bernstein DN , et al . Reducing surgical site infections in spine tumor surgery: a comparison of three methods. Spine (Phila Pa 1976). 2019;44(24):E1428–E1435. doi:10.1097/BRS.0000000000003177 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1097/BRS.0000000000003177&link_type=DOI) 

10. 10. Anderson M , Perrin A , Jiang J , Kumar M . *10% of Americans Don’t Use the Internet. Who Are They?*  [Pew Research Center. Published 2019. Accessed September 7, 2020]. [https://www.pewresearch.org/fact-tank/2019/04/22/some-americans-dont-use-the-internet-who-are-they/](https://www.pewresearch.org/fact-tank/2019/04/22/some-americans-dont-use-the-internet-who-are-they/).
    
    

11. 11. Fox S , Duggan M . *Health Online 2013*  [Pew Research Center. Published 2013. Accessed September 7, 2020]. [https://www.pewresearch.org/internet/2013/01/15/health-online-2013/](https://www.pewresearch.org/internet/2013/01/15/health-online-2013/).
    
    

12. 12.YouTube for Press. *YouTube*  [Published 2020. Accessed September 7, 2020]. [https://www.youtube.com/about/press/#:~:text=Over2 Billion logged-in,in more than 100 countries](https://www.youtube.com/about/press/#:~:text=Over2%20Billion%20logged-in,in%20more%20than%20100%20countries).
    
    

13. 13. Kunze KN , Krivicich LM , Verma NN , Chahla J . Quality of online video resources concerning patient education for the Meniscus: a youtube-based quality-control study. Arthroscopy. 2020;36(1):233–238. doi:10.1016/j.arthro.2019.07.033 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/j.arthro.2019.07.033&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom) 

14. 14. Erdem MN , Karaca S . Evaluating the accuracy and quality of the information in Kyphosis Videos shared on YouTube. Spine (Phila Pa 1976). 2018;43(22):E1334–E1339. doi:10.1097/BRS.0000000000002691 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1097/BRS.0000000000002691&link_type=DOI) 

15. 15. Staunton PF , Baker JF , Green J , Devitt A . Online curves: a quality analysis of scoliosis videos on YouTube. Spine (Phila Pa 1976). 2015;40(23):1857–1861. doi:10.1097/BRS.0000000000001137 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1097/BRS.0000000000001137&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom) 

16. 16. Kunze KN , Cohn MR , Wakefield C , et al . YouTube as a source of information about the posterior cruciate ligament: a content-quality and reliability analysis. Arthrosc Sports Med Rehabil. 2019;1(2):e109–e114. doi:10.1016/j.asmr.2019.09.003 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/j.asmr.2019.09.003&link_type=DOI) 

17. 17. Kwok TMY , Singla AA , Phang K , Lau AYS . YouTube as a source of patient information for varicose vein treatment options. J Vasc Surg Venous Lymphat Disord. 2017;5(2):238–243. doi:10.1016/j.jvsv.2016.10.078 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/j.jvsv.2016.10.078&link_type=DOI) 

18. 18. Belayneh R , Mesfin A . Analysis of internet information on lateral lumbar interbody fusion. Orthopedics. 2016;39(4):e701-7. doi:10.3928/01477447-20160419-05 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.3928/01477447-20160419-05&link_type=DOI) 

19. 19. Shah AK , Yi PH , Stein A . Readability of orthopaedic oncology-related patient education materials available on the internet. J Am Acad Orthop Surg. 2015;23(12):783–788. doi:10.5435/JAAOS-D-15-00324 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.5435/JAAOS-D-15-00324&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=26519430&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom) 

20. 20. Wang D , Jayakar RG , Leong NL , Leathers MP , Williams RJ , Jones KJ . Evaluation of the quality, accuracy, and readability of online patient resources for the management of articular cartilage defects. Cartilage. 2017;8(2):112–118. doi:10.1177/1947603516648737 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1177/1947603516648737&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom) 

21. 21. Addar A , Marwan Y , Algarni N , Berry G . Assessment of “YouTube” content for distal radius fracture immobilization. J Surg Educ. 2017;74(5):799–804. doi:10.1016/j.jsurg.2017.03.002 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1016/j.jsurg.2017.03.002&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom) 

22. 22. Cassidy JT , Fitzgerald E , Cassidy ES , et al . YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc. 2018;26(3):840–845. doi:10.1007/s00167-017-4514-x 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1007/s00167-017-4514-x&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom) 

23. 23. Mathur S , Shanti N , Brkaric M , et al . Surfing for scoliosis: the quality of information available on the Internet. Spine (Phila Pa 1976). 2005;30(23):2695–2700. doi:10.1097/01.brs.0000188266.22041.c2 
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1097/01.brs.0000188266.22041.c2&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=16319757&link_type=MED&atom=%2Fijss%2F16%2F2%2F278.atom)