Stroke Severity Score based on Six Signs and Symptoms The 6S Score: A Simple Tool for Assessing Stroke Severity and In-hospital Mortality

Article information

J Stroke. 2014;16(3):178-183
Publication date (electronic) : 2014 September 30
doi : https://doi.org/10.5853/jos.2014.16.3.178
aDepartment of Clinical Neurological Sciences, London Health Sciences Centre, Western University, London, ON, Canada.
bStroke Center. Favaloro Foundation University Hospital, Favaloro University, Buenos Aires, Argentina.
Correspondence: Luciano A. Sposato. Room C7-123 UH, London, ON, Canada, N6A 5A5. Tel: +1-519-685-8500, ext. 34264, Fax: +1-519-663-3196. lsposato@uwo.ca/lucianosposato@gmail.com
*Juan Manuel Racosta and Federico Di Guglielmo equally contributed as first authors.
Received 2014 July 11; Revised 2014 September 11; Accepted 2014 September 11.

Abstract

Background and Purpose

Ascertaining stroke severity and predicting risk of in-hospital mortality is crucial to advise patients and families about medical decisions. We developed and tested the validity of a new stroke score, the 6S Score (Stroke Severity Score based on Six Signs and Symptoms), for quantifying ischemic stroke severity and predicting in-hospital mortality.

Methods

We prospectively assessed 210 consecutive acute ischemic stroke patients. The cohort was further divided into a derivation (n=120) and a validation (n=90) sample. From a total of 10 stroke signs and symptoms, we selected those with likelihood ratio's P<0.005. We tested the validity of the score for predicting in-hospital mortality by using receiver operating characteristic curves. We used a scatterplot and the Spearman's test to evaluate the correlation between the 6S Score and the National Institutes of Health Stroke Scale as a marker of stroke severity. We used principal component and exploratory factor analyses for assessing qualitative aspects of the 6S Score.

Results

The C statistic for in-hospital mortality was 0.82 for the 6S Score and 0.86 for the National Institutes of health Stroke Scale, respectively, with no significant differences between each other (P=0.79). The correlation between both scores was strong (Spearman's rho 0.68, P<0.001). The factor analyses showed a good balance between left/right hemispheres and anterior/posterior circulations.

Conclusions

The 6S Score may constitute a tool for easily assessing stroke severity and predicting stroke mortality. Further research is needed for further assessing its external validity.

Introduction

Stroke accounted for approximately 1 of every 18 deaths in the United States in 2008 and a substantial proportion of those deaths occurred during hospitalization.1 Ascertaining stroke severity and predicting risk of in-hospital mortality is crucial for objectively assessing stroke outcome and for advising patients and families about potential medical decisions.

A variety of scores have been developed for assessing acute ischemic stroke severity and outcome in the acute setting.2,3,4,5 The most widely used and validated stroke scale is the National Institutes of Health Stroke Scale (NIHSS), which has been demonstrated to accurate establish stroke severity and prognosis in the context of clinical practice and clinical trials.6,7 However, its use can be challenging for non-stroke health care personnel because of its relative complexity. In fact, some attempts have been made to develop simplified versions of NIHSS, but these versions are seldom used.8,9 With the aim of overcoming the relatively complexity of the NIHSS, some international stroke registries have used the number of neurological deficits on admission as a proxy of stroke severity.10,11 Although the ideal scenario is the one of a widespread use of the NIHSS and other well established scales, a simpler and easier to administer score based on the presence or absence of readily identifiable neurological signs and symptoms would be useful for every-day clinical practice in regions and centers where health care professionals are not familiar or certified for administering the NIHSS.

The aim of this exploratory study was to develop and test the validity of a new and simple stroke score, the 6S (Stroke Severity Score based on Six Signs and Symptoms), for quantifying the severity and predicting in-hospital mortality of acute ischemic stroke.

Methods

Patients and data collection

We prospectively assessed 210 patients with diagnosis of acute ischemic stroke consecutively admitted to our hospital between January 1st, 2007 and December 31st, 2011. We excluded 64 patients with diagnosis of transient ischemic attack (TIA) and 28 with intracerebral hemorrhage (ICH). We also excluded 142 cases of ischemic stroke occurred within 30 days after a surgical procedure and 15 patients treated with recombinant tissue plasminogen activator. We defined acute ischemic stroke as new focal or global neurological deficit, with the accompanying evidence of acute cerebral infarction on magnetic resonance imaging (MRI) or computed tomographic (CT) scanning.12 Infarct size was classified into ≤15 mm or >15 mm. We prospectively recorded the NIHSS score on admission, during hospital stay, and upon discharge for each patient. Neurologists administering the NIHSS were trained by using the NIHSS videos. We documented the following nine signs and symptoms upon admission in patients with less than 24 hours since acute ischemic stroke onset: (1) impaired level of consciousness (LOC), (2) motor deficit, (3) sensory deficit, (4) aphasia, (5) dysarthria, (6) visual defects (either amaurosis or hemianopia), (7) neglect, (8) ataxia, and (9) cranial nerve deficits excluding peripheral facial nerve palsy. We also collected demographic data and information about vascular risk factors, comorbidities, and neuroimaging studies (either CT or MRI).

The outcome measure was in-hospital mortality defined as death occurring between admission and before discharge.

Development of 6S score and statistical analysis

We divided the original cohort of 210 acute ischemic stroke patients into two groups. The derivation sample was named "Group A" and comprised 120 randomly selected cases. The random selection of cases was done with the "RAND" function in Microsoft Excel (Microsoft Excel 2011 for Macintosh, Redmond, WA, USA). The validation sample, or "Group B", included the remaining 90 patients.

We performed a univariate analysis for identifying signs and symptoms related to in-hospital mortality in the derivation sample and we calculated the positive likelihood ratio of the selected signs and symptoms for in-hospital mortality. Finally, we used those signs and symptoms with a likelihood ratio's P<0.2 to develop the score by assigning the nearest integer (i.e. 2 for a likelihood ratio of 2.3 and 3 for a likelihood ratio of 2.6).

Validity for prediction of in-hospital mortality

We tested the validity of the 6S Score by developing receiver operating characteristic curves (ROC curves) in the validation sample. We used the area under the curve (C statistic) of the independent variables (signs and symptoms) for in-hospital mortality with their respective 95% confidence interval (95% CI) and significance values based on assumptions of nonparametric distribution, to estimate the sensibility and the specificity of the score. Due to the widespread use of the NIHSS we expected to find no differences in terms of prediction of in-hospital mortality between the 6S score and the NIHSS. To this end, we compared the ROC curves of the 6S Score and the NIHSS by using the method developed by Hanley & McNeil.13 We used the Jonckheere-Terpstra trend test to assess the differences in trends of in-hospital mortality across tertiles of 6S Scores.

Analysis

We considered the NIHSS as the gold standard for stroke severity. We used a two-tailed Spearman test and a scatterplot to examine the correlation between the 6S Score and the NIHSS for assessing stroke severity. We used principal component and exploratory factor analyses to compare the underlying clinimetric structure of both cohorts, and to assess the homogeneity of the 6S score in terms of stroke topography (i.e. left vs. right hemispheric lesions and anterior vs. posterior vascular territories). We first developed a scree plot to identify the number of factors with eigenvalues greater than 1.14 We used aphasia as marker of impairment of the anterior circulation within the left hemisphere. The presence of high values (loading) for aphasia in the principal component analysis would thus reflect an imbalance towards the left hemisphere and/or the anterior circulation. Statistical analyses was done with IBM SPSS Statistics 20 package for Macintosh®.

Ethical considerations

The Institutional Ethics Committee approved the study protocol. We conducted the study in accordance with Good Clinical Practice, all applicable subject privacy and confidentiality requirements, and the guiding principles of the declaration of Helsinki. We anonymized data by removing personal details, thus written informed patient consent was not necessary according to regulatory laws.

Results

As shown in Table 1, six of the nine investigated signs and symptoms were included in the 6S score: impaired level of consciousness (4 points), impairment of cranial nerves (3 points), visual defects (2 points), sensory deficit (2 points), motor deficit (2 points), and aphasia (2 points). As a result, the maximum and minimum possible values assigned to a patient could range between 15 and 0, respectively.

Table 1

Univariate analysis

The area under the curve of the ROC for in-hospital mortality was 0.82 for the 6S Score (Figure 1) and 0.86 for the NIHSS, respectively. The comparison between ROCs of the 6S Score and the NIHSS showed no differences (P=0.79). There was a significant increasing in-hospital mortality trend across tertiles of 6S Score (P<0.001, Figure 2).

Figure 1

The receiving operating curves for in-hospital mortality are shown for the 6S Score and the National Institutes of Health Stroke Scale.

Figure 2

Increase in risk of mortality by tertiles of risk score.

The 6S Score showed a good correlation with the NIHSS: Spearman's rho=0.68, R2=0.56, P<0.001.

The scree plot showed that the number of factors with eigenvalues >1, explaining considerable amounts of variance, were three (Figure 3). The principal component and exploratory factor analyses revealed a similar distribution of the factors among signs and symptoms across the derivation and validation cohorts (Table 2). These analyses also showed low values for aphasia and high values for impairment of cranial nerves. There were no differences in the mean 6S Score values between left and right hemispheric infarcts (3.4±2.0 vs. 4.7±2.8, P=0.12) or between anterior vs. posterior circulation (3.9±2.6 vs. 4.7± 2.4, P=0.41).

Figure 3

Scree plot for the number of factors.

Table 2

Principal component and exploratory factor analyses

There were no differences in demographic data, vascular profiles, stroke severity and outcome between the derivation and the validation cohorts (Table 3).

Table 3

Baseline characteristics of the derivation and validation cohorts

Discussion

Predicting mortality in the acute setting of ischemic stroke remains a challenge in clinical practice and continues to stimulate researchers to develop new and more accurate prognostic tools. While the goal of improving accuracy seems to have been achieved, the "dark side" of precision may be the complexity of administering those scores. The NIHSS needs specific training and investigators are usually required to be certified before administering the scale in almost every clinical trial.15 The iScore is a relatively new tool that has shown excellent prediction of 30-day and 1-year stroke mortality rates and is being increasingly used.16 However, it requires the use of the Canadian Stroke Scale for assessing stroke severity, laboratory data on admission (i.e. glucose), identifying stroke subtypes (i.e. lacunar vs. non-lacunar), and knowing the premorbid status and some pre-existing conditions (i.e. preadmission disability, and history of cancer, renal dialysis, myocardial infarction, smoking, congestive heart failure, and atrial fibrillation), which may result in barrier for its utilization in the acute setting.16 As a result, applying these widely accepted and highly-reliable scores in centers where training is not available or where resources are scarce constitutes a challenge and a barrier for a more generalized use.10,11 Developing the simplest possible score for assessing stroke severity and mortality would still be useful less experienced clinicians around the world.

In the present exploratory study, we developed a simple prognostic score for assessing stroke severity and in-hospital mortality in the acute setting of acute ischemic stroke. The 6S Score showed a good level of discrimination for in-hospital ischemic stroke mortality with no significant differences when compared with the NIHSS (C statistic=0.82 vs. 0.86, P=0.79). These values are similar to those found in a model derived and validated within the Get With The Guidelines-Stroke Program (C statistic=0.85) and in a model only including the NIHSS within the same study (C statistic=0.83), meaning that the NIHSS was probably reliably administered in our study.17 We also found a increasing in-hospital mortality rates with increasing tertiles of the 6S Score (P<0.001).

In terms of stroke severity, there was a good correlation between the 6S Score and the NIHSS (Spearman's rho=0.68, R2=0.56, P<0.001), and the underlying clinimetric structure of the 6S Score was consistent across the derivation and validation cohorts, as demonstrated by the similar composition of factors in both cohorts in the factor analyses. One of the problems of scores assessing stroke severity is that they may assign higher values (more severity) to those affecting the left hemisphere.18 Aphasia, the only lateralizing symptom in the 6S Score, showed relatively low values in the factor analysis, meaning that the overall impact of left hemispheric strokes was balanced. Moreover, there were no differences when comparing scores of patients with strokes localized within the left or right hemispheres. Another relative limitation of some scores is that they may assign higher scores for anterior circulation strokes than for those involving the posterior circulation.19 In the factor analyses of the 6S Score, the impairment of cranial nerves, as a marker of impairment of the posterior circulation, showed relatively high values (i.e. 0.930), meaning that the weight of posterior circulation manifestations in the overall result was significant. This balance between anterior and posterior circulation strokes was further demonstrated with the lack of difference in the mean 6S Score values between strokes involving the anterior and posterior circulation.

This study has several limitations. First, the number of patients was low. Clinically accepted and currently used scores have been derived and validated from cohorts with larger numbers of patients.5,16 However, our study is exploratory in nature, and intended to generate the basis for further confirmation of a potentially useful tool for the rapid assessment of ischemic stroke short-term prognosis. Moreover, we used the factor analyses, aimed at qualitatively evaluating the robustness of the component of the score. Second, the score lacks external validation, which is crucial before being considered as a potential score for use in clinical practice. Third, we did not include variables that may have significantly improved the discrimination and calibration of the 6S Score (i.e. prior comorbidities, results of neuroimaging studies, etiology of the ischemic stroke). However, this was in line with our objective to develop the simplest score possible by using readily recognizable neurological signs and symptoms. Fourth, our cohort is hospital-based. As such, it could be not representative of other populations. As mentioned before, an external validation is mandatory before considering this score as a reliable clinical instrument. Fifth, in-hospital stroke mortality may not reflect the prognosis at other points in time such as 30-day or 1-year mortality, which may be more relevant from the public health perspective. Finally, as long-term follow-up data were not available, we were not able to assess the usefulness of the score for predicting functional outcome (i.e. modified Rankin scale at 3 months).

In summary, we have developed the 6S Score a simple tool for assessing ischemic stroke severity and in-hospital mortality on the basis of signs and symptoms noted upon admission. The score is not intended to replace any of the currently used prognostic tools but is aimed at providing physicians not trained in the use of more sophisticated scales with a readily available tool to be used for clinical purposes. It is also intended to be used when clinical, laboratory, or neuroimaging data needed for other scores, are not fully available. Further research is necessary for validating our exploratory results.

Notes

The authors have no financial conflicts of interest.

References

1. Centers for Disease Control and Prevention. Vital Statistics Public Use Data Files - 2008 Mortality Multiple Cause Files Accessed September 18, 2012. Available at: http://www.cdc.gov/nchs/data_access/Vitalstatsonline.htm#Mortality_Multiple.
2. Hantson L, De Weerdt W, De Keyser J, Diener HC, Franke C, Palm R, et al. The European Stroke Scale. Stroke 1994;25:2215–2219. 7974548.
3. Côté R, Battista RN, Wolfson C, Boucher J, Adam J, Hachinski V. The Canadian Neurological Scale: validation and reliability assessment. Neurology 1989;39:638–643. 2710353.
4. Scandinavian Stroke Study Group. Multicenter trial of hemodilution in ischemic stroke-background and study protocol
5. Brott T, Adams HP Jr, Olinger CP, Marler JR, Barsan WG, Biller J, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989;20:864–870. 2749846.
6. Adams HP Jr, Davis PH, Leira EC, Chang KC, Bendixen BH, Clarke WR, et al. Baseline NIH Stroke Scale score strongly predicts outcome after stroke: A report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Neurology 1999;53:126–131. 10408548.
7. Meyer BC, Hemmen TM, Jackson CM, Lyden PD. Modified National Institutes of Health Stroke Scale for use in stroke clinical trials: prospective reliability and validity. Stroke 2002;33:1261–1266. 11988601.
8. Meyer BC, Lyden PD. The modified National Institutes of Health Stroke Scale: its time has come. Int J Stroke 2009;4:267–273. 19689755.
9. Tirschwell DL, Longstreth WT Jr, Becker KJ, Gammans RE Sr, Sabounjian LA, Hamilton S, et al. Shortening the NIH Stroke scale for use in the prehospital setting. Stroke 2002;33:2801–2806. 12468773.
10. Heuschmann PU, Kolominsky-Rabas PL, Misselwitz B, Hermanek P, Leffmann C, Janzen WC, et al. German Stroke Registers Study Group. Predictors of inhospital mortality and attributable risks of death after ischemic stroke: the German Stroke Registers Study Group. Arch Intern Med 2004;164:1761–1768. 15364669.
11. Sposato LA, Esnaola MM, Zamora R, Zurrú MC, Fustinoni O, Saposnik G. ReNACer Investigators. Argentinian Neurological Society. Quality of ischemic stroke care in emerging countries: the Argentinian National Stroke Registry (ReNACer). Stroke 2008;39:3036–3041. 18703802.
12. Easton JD, Saver JL, Albers GW, Alberts MJ, Chaturvedi S, Feldmann E, et al. Definition and evaluation of transient ischemic attack: A scientific statement for healthcare professionals from the american heart association/american stroke association stroke council; council on cardiovascular surgery and anesthesia; council on cardiovascular radiology and intervention; council on cardiovascular nursing; and the interdisciplinary council on peripheral vascular disease. The american academy of neurology affirms the value of this statement as an educational tool for neurologists. Stroke 2009;40:2276–2293. 19423857.
13. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839–843. 6878708.
14. Kaiser HF. The application of electronic computers to factor analysis. Educ Psychol Meas 1960;20:141–151.
15. Lyden P, Brott T, Tilley B, Welch KM, Mascha EJ, Levine S, et al. Improved reliability of the NIH Stroke Scale using video training. NINDS TPA Stroke Study Group. Stroke 1994;25:2220–2226. 7974549.
16. Saposnik G, Kapral MK, Liu Y, Hall R, O'Donnell M, Raptis S, et al. Investigators of the Registry of the Canadian Stroke Network. Stroke Outcomes Research Canada (SORCan) Working Group. IScore: a risk score to predict death early after hospitalization for an acute ischemic stroke. Circulation 2011;123:739–749. 21300951.
17. Smith EE, Shobha N, Dai D, Olson DM, Reeves MJ, Saver JL, et al. Risk score for in-hospital ischemic stroke mortality derived and validated within the Get With the Guidelines-Stroke Program. Circulation 2010;122:1496–1504. 20876438.
18. Lyden P, Claesson L, Havstad S, Ashwood T, Lu M. Factor analysis of the National Institutes of Health Stroke Scale in patients with large strokes. Arch Neurol 2004;61:1677–1680. 15534178.
19. Heldner MR, Zubler C, Mattle HP, Schroth G, Weck A, Mono ML, et al. National Institutes of Health stroke scale score and vessel occlusion in 2152 patients with acute ischemic stroke. Stroke 2013;44:1153–1157. 23471266.

Article information Continued

Figure 1

The receiving operating curves for in-hospital mortality are shown for the 6S Score and the National Institutes of Health Stroke Scale.

Figure 2

Increase in risk of mortality by tertiles of risk score.

Figure 3

Scree plot for the number of factors.

Table 1

Univariate analysis

Table 1

Table 2

Principal component and exploratory factor analyses

Table 2

Table 3

Baseline characteristics of the derivation and validation cohorts

Table 3