Availability and Readability of Spinal Cord Injury Online Information Materials for Spanish Speaking Population in Neurosurgical Academic Programs: A Nationwide Study
======================================================================================================================================================================

* Jacques Lara-Reyna
* Jorge A. Roa
* Kurt A. Yaeger
* Konstantinos Margetis

## ABSTRACT

**Background:** Recent publications have demonstrated that information has been transmitted inappropriately to the lay person in different pathologies. This limitation is also observed in Spanish language. We evaluate the availability and readability of online patient education material (PEM) on spinal cord injury (SCI) information for the Spanish-speaking population from academic neurosurgery residency programs in the United States.

**Methods:** This is a descriptive analysis of online SCI PEM from neurosurgical residency programs websites. We assess the availability of information in Spanish using a modification of a previously published classification. To assess accessibility, we calculated the time spent and the number of clicks to find the information in Spanish. We calculated the readability of the material using the “Indice Flesch-Szigriszt” (INFLESZ), which determines the difficulty of readability of health-related material in Spanish.

**Results:** A total of 116 accredited neurosurgery residency programs comprised our cohort. Ten (9%) programs had available “mirrored” information in Spanish from its original version in English, 9 (8.1%) used a translation software, 79 (71.2%) provide interpreter services, and 3 (2%) did not have written information or information about translation services. A mean of 72.9 seconds (SD +/− 71.2) were required to have access to the Spanish information or contact information for translation services. Twelve (57.1%) websites with written Spanish information had an INFLESZ score above 55.00, which translates as an appropriate readability level for the general population.

**Conclusions:** More than half of the academic neurosurgery programs or affiliated hospital websites do not provide written informative material about SCI in Spanish. When available, the information is not always transmitted with a level of readability appropriate for the layperson. Most of the websites provide translation or interpreter services that are not directly related to SCI.

*   spinal cord injury
*   residency
*   availability
*   readability
*   Spanish

## INTRODUCTION

According to the National Spinal Cord Injury Statistical Center, an estimated annual incidence of spinal cord injury (SCI) is approximately 54 cases per 1 million people (17,730 new cases a year), with prevalence of approximately 291,000 people living with SCI.1

According to the US Census Bureau, in 2019 there were 60,481,746 Hispanic/Latino individuals representing 18.4% of the total population, and 41,757,391 individuals that speak Spanish at home, which represents 13.5% of the total US population.2

Nowadays, the internet has become an essential tool for the distribution of health-related information, for both professionals and the general public. Thirty-five percent of US adults have looked for information regarding health issues that they may have experienced.3 The American Medical Association and the National Institutes of Health (NIH) have recommended that patient education material (PEM) should be written in a sixth-grade level to ensure proper understandability in the general population.4–6

Recent publications have shown that the level of understandability and readability of PEM is higher than the accepted guidelines, which may compromise the full comprehension of the disease by the affected patient.4,7–9 The main aim of this study is to assess the availability and readability of online PEM related to SCI focused on Spanish-speaking population from US academic neurosurgery training programs.

## METHODS

After communication with the institutional review board (IRB), it was determined that this project is not a Human Subjects Research and does not require a formal review by the IRB. A descriptive analysis of publicly available information about SCI was obtained from the neurosurgery residency training program websites across the country. The list of programs was obtained from the Accreditation Council for Graduate Medical Education (ACGME) (available at: [https://apps.acgme.org/ads/public/](https://apps.acgme.org/ads/public/)). Information of SCI was obtained using the residency program website searching tool with the keywords: “spinal cord injury,” “trauma,” “spine fracture.” If the information provided in the academic website (usually with the domain .edu) was not adequate or insufficient, we performed an additional search in the affiliated hospital website (usually with the domains .org or .com). The English content was classified as “satisfactory” (more than 1 written line explaining the concept of SCI, more than 2 words mentioning the symptoms, more than 2 words explaining diagnosis modalities, more than 2 words indicating treatment modalities, more than 1 line about prognosis/recovery), “deficient” (as the absence of the aforementioned criteria), and “no available information.” For the purpose of this project, we consider a written translation to Spanish as the optimal PEM option. Once the PEM was identified in English, we evaluated the availability of the same information in Spanish, searching for links for direct translation of the content. If no direct translation link was available, we performed a manual navigation through the labels: “Patients & Services,” “Translator/Interpreter Services,” “International Patients,” “Non Discrimination Act,” “Patient Rights.” In order to assess the accessibility of the Spanish information, we calculated the length of time (in seconds) and the number of clicks that took to access the information. For the purpose of this study, a threshold of 5 minutes (300 seconds) was set, before considering that the website did not have easily accessible material in Spanish or easily accessible information about translation services. By setting up this timing, we assessed if the website is easy to navigate and designed in an intuitive way. To assess the characteristics of the written Spanish-language material, we used a variation of the classification presented by Kroshus et al10 who classified the content in: “mirrored,” “partial,” “use of a translation application or software,” “use of external information of SCI in Spanish,” “direct phone or in person translation services,” and “no Spanish content.”10 In order to assess readability of this material in Spanish, we use the previously validated “Indice Flesch-Szigriszt” (INFLESZ), which grades the content in Spanish in “muy dificil” (very hard) with points between 0 and 40, “algo dificil” (somewhat difficult) 40–55, “normal” with 55–65, “bastante facil” (somewhat easy) 65–80, and “muy facil” (very easy) with points between 80 and 100.11 Barrio-Cantalejo et al11 define appropriate readability for health texts as an INFLESZ score above 55.00. This score corresponds to information written for basic education level (ranging from secondary school, to comics or primary school level of readability).11 Scores below 55.00 indicates that the level of readability ranges between a Baccalaureate to scientific publication material.11,12 A free INFLESZ software calculator is available at the following address: [https://legibilidad.blogspot.com/2015/01/el-programa-inflesz.html](https://legibilidad.blogspot.com/2015/01/el-programa-inflesz.html) and additionally as an online version of the software in [https://legible.es/](https://legible.es/).

The first author performed a second collection of data using the same variables in a different period of time (4 months after) to perform an intrarater analysis. Cohen κ test with values < 0 indicating no agreement, 0.01 to 0.20 as none to slight, 0.21 to 0.40 as fair, 0.41 to 0.60 as moderate, 0.61 to 0.80 as substantial, and 0.81 to 1.00 as almost perfect agreement as described by McHugh.13

## RESULTS

From August to December 2020, a total of 116 accredited neurological surgery residency training programs were registered in the ACGME database across the United States.14–16 A total of 111 institutional residency program websites in English were available for review (we excluded 2 residency programs in different cities that are part of the same network and the information is provided in a single institutional website). Three programs did not have available websites. Thirty-two (28.8%) had “satisfactory” information about the definition of SCI, 19 (17.1%) “deficient,” and 60 (54.1%) did not have available information. Forty-five (40.5%) had information about the causes of SCI. Thirty-one (27.9%) websites had “satisfactory” levels of information related to the symptoms, 19 (17.1%) with “deficient,” and 59 (53.2%) no available information. Twenty-six websites (23.4%) had a “satisfactory” level of information about diagnosis, 15 (13.5%) “deficient,” and 69 (62.2%) no information. Twenty-three (20.7%) had a “satisfactory” level of treatment modalities information, 31 (27.9%) “deficient,” and 56 (50.5%) none. Eight (7.2%) had “satisfactory” levels of information regarding the prognosis and recovery, 22 (19.8%) “deficient,” and 80 (72.1%) no available information (Figure).

![Figure](http://ijssurgery.com//https://www.ijssurgery.com/content/ijss/15/5/1039/F1.medium.gif)

[Figure](http://ijssurgery.com//content/15/5/1039/F1)

Figure 
Available information of SCI in original English language from academic or affiliated hospital websites. From 116 accredited programs, 3 did not have websites, and 2 were excluded due to being part of a larger institution and provided information from the main institutional website.

After identifying the information related to SCI from the original English website, it took an average of 72.9 seconds (SD +/− 71.2 seconds), and 5 clicks (SD +/− 5.03 clicks) to find information in Spanish or information about interpreter services.

Ten (9%) websites had a “mirrored” content from the native English website. Eight websites (7.2%) had “partial” information in Spanish, defined as a variety of content in Spanish but not all of it. In our cohort, this “partial” information available was not related to SCI. Nine (8.1%) websites used a link to an online translation software. Two (1.8%) websites were linked to an external resource in Spanish. Seventy-nine (71.2%) websites provided a phone number or offered in-person translation services. Three (2.7%) websites did not have available Spanish content after 5 minutes of navigation though the website. However, in these 3 websites we found interpreter/languages services after the threshold of 5 minutes by using the Searching Tool (Table 1).

View this table:
[Table 1](http://ijssurgery.com//content/15/5/1039/T1)

Table 1 
Adaptation of Spanish language website Classification Rubric proposed by Kroshus et al.10 The first 4 categories represent written online PEM.

It required a mean of 31 seconds (SD +/− 49.7 seconds) and 2.4 clicks (SD +/− 3.4 clicks) to find Spanish written information, compared with 85.8 seconds (SD +/− 72.2 seconds) and 5.8 click (SD +/− 5.2 clicks) for the Interpreter/translation services.

In Table 2, we summarize these findings by the States with the highest percentage of habitants who speak Spanish at home.

View this table:
[Table 2](http://ijssurgery.com//content/15/5/1039/T2)

Table 2 
Distribution of states by percentage of Spanish speaker population. Boldface indicates states without accredited neurological surgery residency programs.

For the analysis of the INFLESZ, we excluded 8 websites with partial information in Spanish that do not have information related to SCI. The mean INFLESZ score for the available written information in Spanish (n = 21, 18.4%) was 51.83 (SD +/− 16.16), which is translated into “somewhat difficult” to understand in Spanish. Twelve (57.1%) of the available websites with Spanish information of SCI had an INFLESZ score above 55.00, which is considered a “normal” level of readability, and 9 (42.9%) had a score below 55.00.

Cohen κ test was 0.58 when comparing the groups in the variation of the classification presented by Kroshus et al,10 which translates into a moderate level of agreement. The moderate level of agreement was attributed to changes in the website contents that happened in the time period between the 2 reviews.

## DISCUSSION

Spinal cord injury (SCI) represents a significant cause of long-term morbidity and associated complications worldwide. In the United States, it was reported that 13.2% of people suffering SCI since 2015 were Hispanic in origin.1 The proper distribution of PEM is a key factor to promote adequate understandability of medical conditions.

Overall, we found that between 50% and 72% of the visited neurosurgical academic programs or affiliated hospital websites did not provide sufficient information about SCI regarding the definition, causes, symptoms, diagnosis, treatment, and prognosis. From these, 24.3% had available written information in Spanish (either using a mirrored or partial material, or digital translation application); 71.2% had available interpreters services that were not specific to the topic, and 2.7% did not have easily accessible information for interpreter services or Spanish information after 5 minutes of searching.

The Section VI of 1964 Civil Right Act was the beginning of substantial changes in the law to avoid discrimination based on several factors, including language, in all federally funded institutions.17,18 Federal Health services include (but not limited to) institutions receiving funding from Medicaid, State Children's Health Insurance Program, Medicare, NIH grants, and Centers for Disease Control and Prevention.18 Graduate Medical Education residency programs across the United States also receive federal funding through Medicaid, Medicare, and Veterans Health Administration, exceeding $15 billion a year.19

We consider the written materials (pamphlets, websites, audiovisual) as an effective way to distribute information to a patient in a not-in person setting. This is in addition to the interpreter language services that by default, all federally funded institutions should provide to the users. Having this in consideration, we found that only 18.9% of academic programs provide online information regarding SCI PEM nationwide, which could be considered suboptimal for an expanding Spanish-speaking population, that is planned to reach 111 million by 2060.20

The 3 states with the highest percentage of people that speak Spanish at home are Texas, California, and New Mexico at 29.2%, 28.8%, 26.5% of the population, respectively.2 There are 17 accredited neurological surgery residency programs in these states (Texas: 7; California: 10; New Mexico: 0). A total of 4 (23.6%) have available written information in Spanish, 12 (70.6%) provide interpreter or translation services, and 1 (5.9%) did not have easily accessible information about Spanish services. On the other hand, the 3 states with the highest number of residency programs are New York (12), California (10), and Pennsylvania (8). From these, 5 (16.7%) provide written SCI information in Spanish, 20 (66.7%) interpreter services, and 5 (16.7%) no easily accessible information.

It has been reported that language barriers between patients and providers can lead to delays in medical care, lack of understanding of health-related issues, increase in medical cost, more hospital admissions, and longer emergency department stays.18,21 Several measures have been proposed to prevent adverse effects during medical visits with language barriers, including in-person trained medical interpreter, ad hoc hospital bilingual staff, and the use of line or video interpretation. However, these measures may increase the overall cost.21

We also found that it was faster to find written information in Spanish compared with information about the interpreter services (ie, phone number for direct line). An interpretation of this is that the websites with written information implement a more user-friendly approach that allows easy access to the information.

It has been demonstrated in the literature that the level of understandability and readability of the online information for Spanish-speaker patients is often written in a more complex level than accepted. Examples of the above are publications in concussion, burns, dermatology, otolaryngology, and endodontics.5,6,10,22–24 Regarding SCI in particular, Bea-Muñoz et al22 performed a search using the keywords “recommendations,” “advice,” “guide,” “manual,” “self-care,” “education,” and “information”; adding “spinal cord injury,” “paraplegia,” and “tetraplegia” to each word in Spanish using Google. They chose the first 50 results of each search, and they found that 75% of the online material had a level “somewhat difficult” to interpret by lay people using INFLESZ score.22 They also presented that most of the online material related to SCI in Spanish had a medium quality, defined as the presence of institutional certificates, seals, or labels indicating the website complies with specific codes of ethics.12 Our results are somewhat better, because we found 57% of the academic programs websites with available written Spanish information to have a “normal” level of readability (INFLESZ above 55.00).

One of the limitations is that we did not perform an analysis of the level of understandability of the original material in English and compare with the material in Spanish. Also, the 5 minutes of set up timing is an arbitrary limit. This limit does not strictly rule out that the websites lacked SCI information or instructions about translation services, but it may indicate that the design of the website is not user-friendly to allow easy access to the information.

## CONCLUSIONS

The majority of the academic neurosurgery programs across the United States do not provide access to written PEM for the Spanish-speaking population regarding SCI. When available, the information is not always transmitted with a proper level of readability for the layperson. The majority of websites provide contact information about interpreter or translation services that are not directly related to SCI. Overall, it appears that there is room for improvement on the SCI PEM offered for the Spanish-speaking population in the websites of the US academic neurosurgery residency programs.

## Footnotes

*   **Disclosures and COI:** The authors received no funding for this study and report no conflicts of interest. After communication with the institutional review board (IRB), it was determined that this project is not a Human Subjects Research and does not require a formal review by the IRB.

*   This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2021 ISASS

## REFERENCES

1.  1 .National Spinal Cord Injury Statistical Center. Spinal cord injury (SCI) facts and figures at a glance. *J Spinal Cord Med*. 2016;39(3):370–371.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1080/10790268.2016.1177348&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=27193177&link_type=MED&atom=%2Fijss%2F15%2F5%2F1039.atom) 

2.  2 .US Census Bureau. Census.gov. [https://www.census.gov/en.html](https://www.census.gov/en.html). Accessed September 24, 2020.
    
    

3.  3 .Fox S, Duggan M. Health online 2013. *Health*. 2013;2013:1–55.
    
    

4.  4 .Lopez Ramos C, Williams JE, Bababekov YJ, Chang DC, Carter BS, Jones PS. Assessing the understandability and actionability of online neurosurgical patient education materials. *World Neurosurg*. 2019;130:e588–e597.
    
    

5.  5 .Votta K, Metivier M, Romo S, et al. Readability of Spanish language online information for the initial treatment of burns. *Burns*. 2018;44(4):956–961.
    
    

6.  6 .Nassif SJ, Wong K, Levi JR. The Índice Flesch-Szigriszt and Spanish Lexile Analyzer to evaluate Spanish patient education materials in otolaryngology. *Laryngoscope*. 2018;128(1):E21–E26.
    
    

7.  7 .Long WW, Modi KD, Haws BE, et al. Assessing online patient education readability for spine surgery procedures. *Clin Spine Surg*. 2018;31(2):E146–E151.
    
    

8.  8 .Ahmed OH, Sullivan SJ, Schneiders AG, McCrory PR. Concussion information online: evaluation of information quality, content and readability of concussion-related websites. *Br J Sports Med*. 2012;46(9):675–683.
    
    [Abstract/FREE Full Text](http://ijssurgery.com//lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYmpzcG9ydHMiO3M6NToicmVzaWQiO3M6ODoiNDYvOS82NzUiO3M6NDoiYXRvbSI7czoyMDoiL2lqc3MvMTUvNS8xMDM5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  9 .Eltorai AEM, Cheatham M, Naqvi SS, et al. Is the readability of spine-related patient education material improving? An assessment of subspecialty websites. *Spine*. 2016;41(12):1041–1048.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F15%2F5%2F1039.atom) 

10. 10 .Kroshus E, Gonzalez LA, Chrisman SPD, Jimenez N. Availability of concussion information in Spanish for parents of youth athletes. *Health Promot Pract*. 2019;20(3):372–380.
    
    

11. 11 .Barrio-Cantalejo IM, Simón-Lorda P, Melguizo M, Escalona I, Marijuán MI, Hernando P. [Validation of the INFLESZ scale to evaluate readability of texts aimed at the patient]. *An Sist Sanit Navar*. 2008;31(2):135–152.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=18953362&link_type=MED&atom=%2Fijss%2F15%2F5%2F1039.atom) 

12. 12 .Bea-Muñoz M, Medina-Sánchez M, Flórez-García MT. Quality of websites with patient information about spinal cord injury in Spanish. *Spinal Cord*. 2016;54(7):540–545.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F15%2F5%2F1039.atom) 

13. 13 .McHugh ML. Interrater reliability: the kappa statistic. *Biochem Med*. 2012;22(3):276–282.
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fijss%2F15%2F5%2F1039.atom) 

14. 14 .Association of American Medical Colleges. Careers in medicine. [https://apps.aamc.org/cim-residency/#!/residency-search](https://apps.aamc.org/cim-residency/#!/residency-search). Accessed July 11, 2020.
    
    

15. 15 .Accreditation Council for Graduate Medical Education. Neurological surgery. [https://www.acgme.org/Specialties/Overview/pfcatid/10/Neurological-Surgery](https://www.acgme.org/Specialties/Overview/pfcatid/10/Neurological-Surgery). Accessed July 11, 2020.
    
    

16. 16 .Accreditation Council for Graduate Medical Education. ACGME - Accreditation Data System (ADS). [https://apps.acgme.org/ads/Public/Programs/Search](https://apps.acgme.org/ads/Public/Programs/Search). Accessed September 23, 2020.
    
    

17. 17 . US Code. U.S.C. Title 42—The Public Health and Welfare. [https://www.govinfo.gov/content/pkg/USCODE-2009-title42/html/USCODE-2009-title42-chap21-subchapV-sec2000d-1.htm](https://www.govinfo.gov/content/pkg/USCODE-2009-title42/html/USCODE-2009-title42-chap21-subchapV-sec2000d-1.htm). Accessed September 25, 2020.
    
    

18. 18 .Chen AH, Youdelman MK, Brooks J. The legal framework for language access in healthcare settings: Title VI and beyond. *J Gen Intern Med*. 2007;22(suppl 2):362–367.
    
    [CrossRef](http://ijssurgery.com//lookup/external-ref?access_num=10.1007/s11606-006-0099-7&link_type=DOI) 
    
    [PubMed](http://ijssurgery.com//lookup/external-ref?access_num=17356969&link_type=MED&atom=%2Fijss%2F15%2F5%2F1039.atom) 
    
    [Web of Science](http://ijssurgery.com//lookup/external-ref?access_num=000244718600013&link_type=ISI) 

19. 19 .Eden J, Berwick D, Wilensky G. GME financing. In: Institute of Medicine; Board on Health Care Services; Committee on the Governance and Financing of Graduate Medical Education, eds. Graduate Medical Education That Meets the Nation's Health Needs. Washington, DC: The National Academies Press; 2014. [https://www.ncbi.nlm.nih.gov/books/NBK248024/](https://www.ncbi.nlm.nih.gov/books/NBK248024/). Accessed September 27, 2021.
    
    

20. 20 .US Census Bureau. Hispanic population to reach 111 million by 2060. [https://www.census.gov/library/visualizations/2018/comm/hispanic-projected-pop.html](https://www.census.gov/library/visualizations/2018/comm/hispanic-projected-pop.html). Accessed September 27, 2020.
    
    

21. 21 .Slade S, Sergent SR. Language Barrier. Treasure Island, FL: StatPearls Publishing; 2021. [https://www.ncbi.nlm.nih.gov/books/NBK507819/](https://www.ncbi.nlm.nih.gov/books/NBK507819/). Accessed September 27, 2021.
    
    

22. 22 .Bea-Muñoz M, Medina-Sánchez M, Flórez-García MT. [Readability and internet accessibility of informative documents for spinal cord injury patients in Spanish]. *An Sist Sanit Navar*. 2015;38(2):255–262.
    
    

23. 23 .Miguéns-Vila R, Ledesma-Ludi Y, Rodríguez-Lozano F, Varela-Centelles P, Seoane-Romero JM, Castelo-Baz P. Disparities between English and Spanish in readability of online endodontic information for laypeople. *J Am Dent Assoc*. 2018;149(11):960–966.
    
    

24. 24 .Mazmudar RS, Sheth A, Tripathi R, Scott JF. Readability of online Spanish patient education materials in dermatology. *Arch Dermatol Res*. 2021;313(3):201–204.