J Stroke Search

CLOSE


J Stroke > Volume 21(3); 2019 > Article
Sánchez van Kammen, Greving, Kuroda, Kashiwazaki, Morita, Shiokawa, Kimura, Cognard, Januel, Lindgren, Koivisto, Jääskeläinen, Ronkainen, Pyysalo, Öhman, Rahi, Kuhmonen, Rinne, Leemans, Majoie, Vandertop, Verbaan, Roos, Berg, Boogaarts, Moudrous, Wijngaard, Hove, Teo, George, Hackenberg, Abdulazim, Etminan, Rinkel, and Vergouwen: External Validation of the ELAPSS Score for Prediction of Unruptured Intracranial Aneurysm Growth Risk

Abstract

Background and Purpose

Prediction of intracranial aneurysm growth risk can assist physicians in planning of follow-up imaging of conservatively managed unruptured intracranial aneurysms. We therefore aimed to externally validate the ELAPSS (Earlier subarachnoid hemorrhage, aneurysm Location, Age, Population, aneurysm Size and Shape) score for prediction of the risk of unruptured intracranial aneurysm growth.

Methods

From 11 international cohorts of patients ≥18 years with ≥1 unruptured intracranial aneurysm and ≥6 months of radiological follow-up, we collected data on the predictors of the ELAPSS score, and calculated 3- and 5-year absolute growth risks according to the score. Model performance was assessed in terms of calibration (predicted versus observed risk) and discrimination (c-statistic).

Results

We included 1,072 patients with a total of 1,452 aneurysms. During 4,268 aneurysm-years of follow-up, 199 (14%) aneurysms enlarged. Calibration was comparable to that of the development cohort with the overall observed risks within the range of the expected risks. The c-statistic was 0.69 (95% confidence interval [CI], 0.64 to 0.73) at 3 years, compared to 0.72 (95% CI, 0.68 to 0.76) in the development cohort. At 5 years, the c-statistic was 0.68 (95% CI, 0.64 to 0.72), compared to 0.72 (95% CI, 0.68 to 0.75) in the development cohort.

Conclusions

The ELAPSS score showed accurate calibration for 3- and 5-year risks of aneurysm growth and modest discrimination in our external validation cohort. This indicates that the score is externally valid and could assist patients and physicians in predicting growth of unruptured intracranial aneurysms and plan follow-up imaging accordingly.

Introduction

The prevalence of unruptured intracranial aneurysms in the adult population is around 3% [1], and the number of incidentally found intracranial aneurysms is increasing [2]. After diagnosis of an unruptured intracranial aneurysm, the risk of aneurysm rupture has to be carefully weighed against the risk of complications from preventive aneurysm treatment. In many patients, the risk of treatment complications from preventive treatment is estimated to outweigh the risk of aneurysm rupture. In a number of these patients, the initial treatment of choice is conservative management with follow-up imaging [3]. Follow-up imaging is often recommended because some aneurysms grow during follow-up, indicating aneurysm instability with an increased risk of rupture [4,5]. If aneurysm growth is detected during follow-up, preventive treatment should be reconsidered [3]. However, current international guidelines lack clear recommendations regarding the optimal timing for follow-up imaging [6,7].
The ELAPSS (Earlier subarachnoid hemorrhage, aneurysm Location, Age, Population, aneurysm Size and Shape) score was developed to guide clinicians in predicting absolute 3- and 5-year growth risks per aneurysm, according to six simple patient- and aneurysm characteristics (Table 1) [8]. The total score ranges from 0 to 40, with a higher score indicating increasing 3- and 5-year growth risks. The score had accurate calibration and modest discrimination. However, since prediction models often perform worse in new data than in the development data, a prediction score is preferably externally validated before it is implemented in clinical practice [9]. Therefore, we performed a multicenter study to externally validate the ELAPSS score.

Methods

Study population

We obtained individual patient data from 11 cohorts of patients with unruptured intracranial aneurysm(s) from the following geographic regions: Japan (Toyama, Tokyo, and Otawara), Finland (Tampere, Kuopio, and Turku), the Netherlands (Nijmegen, Amsterdam, and the Hague), France (Toulouse), United Kingdom (Glasgow), Germany (Mannheim and Düsseldorf) (Supplementary Table 1). The Kuopio University Hospital also participated in the study to develop the ELAPSS score [8], but for the validation study a more recent and not previously analysed patient cohort was used.

Inclusion and exclusion criteria

Following the criteria of the previous study [8], we included cohorts with consecutive patients ≥18 years with ≥1 unruptured saccular intracranial aneurysm with ≥6 months of radiological follow-up by means of magnetic resonance angiography, computed tomography angiography, or calibrated digital subtraction angiography. Extradural aneurysms, fusiform aneurysms, or aneurysms that were part of an arteriovenous malformation were excluded.

Data collection

The following variables were retrieved by the local investigators. At time of aneurysm detection: date of first imaging, age, sex, history of subarachnoid hemorrhage, aneurysm location, and aneurysm size and shape; and during follow-up: occurrence of aneurysm growth, date of first imaging with aneurysm growth, date of last imaging without aneurysm growth, occurrence of rupture, date of rupture, and date of last imaging assessment. Aneurysm size was assessed by the local investigators or retrieved from radiologic reports on 0.1 to 1 mm scales, depending on local facilities. The local investigators also assessed aneurysm shape. Irregular aneurysm shape was defined as multiple lobes, blebs or wall protrusions.

Outcome assessment

Aneurysm growth was defined as growth in one direction of ≥1 mm or an indisputable change in aneurysm shape on follow-up imaging. Because aneurysm growth is thought to be an irregular process with periods of stability followed by periods of slow or rapid growth, it is not possible to determine the exact time of aneurysm growth [10]. The time of aneurysm growth was estimated as the time point halfway between the last imaging prior to aneurysm growth and the first imaging with aneurysm growth. Follow-up data for patients were censored at the time of aneurysm growth, at the time of the last imaging assessment before surgical or endovascular treatment or rupture, or at the time of the last imaging assessment during follow-up.

Approval

The Institutional Review Board of the University Medical Center Utrecht decided that no formal approval was needed to conduct this study.

Statistical analysis

Data were missing on aneurysm size for 13 aneurysms (<1%). These aneurysms were excluded from the analysis. The original regression equations were applied to calculate 3- and 5-year growth risks for each aneurysm [8]. We assessed calibration with calibration plots and discriminatory performance of the model with the c-statistic with 95% confidence interval (CI). Calibration at 3 and 5 years was examined by dividing patients in quintiles according to their predicted risk. The mean predicted risk per quintile group was subsequently plotted against the observed risk per quintile group. Calibration over time was assessed across risk groups that were predefined as low-risk (0 to 9 points on the ELAPPS score), intermediate-risk (10 to 19 points), and high-risk (≥20 points). Analyses were performed in R version 3.4 (R Foundation for Statistical Computing, Vienna, Austria). Results are reported in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement [11].

Results

Between February 1996 and May 2017, a total of 1,072 patients with 1,452 aneurysms and 4,268 aneurysm-years of follow-up were followed. Median follow-up time was 2.0 years (interquartile range, 1.1 to 4.1). Patient- and aneurysm characteristics at time of aneurysm detection in the development and validation cohort are shown in Tables 2 and 3. Patients in the validation cohort were younger than patients in the development cohort and were less often Japanese.
Aneurysm growth was observed in 199 aneurysms (14%) in 155 patients (14%). Growth rates were comparable to those in the development cohort (Figure 1). Aneurysm rupture occurred in 14 patients (1.3%). In 10 of these 14 patients (71%) with aneurysm rupture, growth was detected before rupture.
Calibration at 3 years showed that the ELAPSS score slightly underestimated growth risk, in particular in the lowest risk quintile, but overall observed risks were within the range of the expected risks (Figure 2A). For calibration at 5 years again observed risks were within the range of the expected risk, but some underestimation occurred in patients in the lowest-risk quintile, and an overestimation occurred in the highest risk quintile (Figure 2B). Calibration across ELAPSS risk categories over time was accurate, but 5-year growth risk was somewhat underestimated in the low- and intermediate-risk categories (Figure 3). Supplementary Figure 1 shows observed growth risks in the development and validation cohort according to ELAPSS risk categories. Higher growth risks were observed in the low- and intermediate-risk categories in the validation cohort compared to the development cohort.
The c-statistic at 3 years was 0.69 (95% CI, 0.64 to 0.73), compared to 0.72 (95% CI, 0.68 to 0.76) in the development cohort, and at 5 years 0.68 (95% CI, 0.64 to 0.72), compared to 0.72 (95% CI, 0.68 to 0.75) in the development cohort.

Discussion

In this validation study, we found that the ELAPSS score showed accurate calibration for 3- and 5-year risks of aneurysm growth, comparable to that in the development study [8]. Discrimination was modest but also comparable to that of the development study, making the score overall an externally valid tool with modest discriminative power.
Our findings are in line with a recent study that found increasing ELAPSS score to be associated with aneurysm growth in a cohort of 353 patients with 431 unruptured intracranial aneursyms [12]. The reason for the underestimation of the 5-year growth risk for low-risk aneurysms in our validation cohort may be in part because median follow-up time in the validation cohort was only 2.1 years, which limits reliability of the Kaplan-Meier estimated observed probabilities at 5 years. Underestimation could also result from factors not included in the model. Several aggregate data meta-analyses have identified risk factors for aneurysm growth in addition to the risk factors included in the ELAPSS score, such as sex, hypertension, smoking, and aneurysm multiplicity [13,14]. However, in the development of the ELAPSS score, sex and hypertension were discarded as predictors because of their limited predictive value. Aneurysm multiplicity carries a higher patient risk of growth, but this is probably because of the multiple hit risk [15]. Smoking is an important risk factor for aneurysm growth that should be weighed in clinical decision-making, but could not be included in the present prediction model because of lack of data in the parent cohorts of the development study. Since patients may start or quit smoking during follow-up, or increase or decrease the number of cigarettes per day over time, the exact impact of smoking on aneurysm growth is difficult to assess.
This study has some limitations that need to be addressed. First, because management of the patients was not according to a pre-specified protocol, indication for and timing of aneurysm follow-up were determined by the local treating physicians, and thus varied between and within the cohorts, as it was the case in the ELAPSS development study. Moreover, aneurysms with a very high growth risk were probably treated and therefore did not contribute to follow-up. Similarly, patients with a limited life expectancy due to advanced age or comorbidity often do not undergo follow-up imaging. Therefore, this risk score can only be applied to patients in whom it is decided to perform a wait-and-scan policy. Second, measurements were done by the local investigators of the participating centers, mostly without double data entry or evaluation of interobserver agreement, again similar to how this was done in the study underlying the development of the ELAPSS score. Furthermore, we included different imaging modalities. These are limitations for the outcome measurement, but on the other hand increase the external validity of our results.
It remains to be investigated if the discriminative performance of the model can be improved by including other parameters. Examples may be smoking status, population differences not accounted for by the model, novel imaging parameters like gadolinium enhancement of the aneurysm wall on magnetic resonance imaging [16], or other, still unknown risk factors. Also, in order to formulate recommendations on the ideal time intervals for repeated imaging, a cost-effectiveness analysis should be performed. As shown in a recent study [17], no follow-up imaging may be indicated for small, low-risk aneurysms.

Conclusions

The ELAPSS score showed accurate calibration for 3- and 5-year risks of aneurysm growth and modest discrimination in our external validation cohort. This indicates that the score is externally valid and could assist patients and physicians in predicting growth of unruptured intracranial aneurysms and plan follow-up imaging accordingly.

Supplementary materials

Supplementary materials related to this article can be found online at https://doi.org/10.5853/jos.2019.01277.
Supplementary Table 1.
Patient- and aneurysm characteristics of included cohorts
jos-2019-01277-suppl1.pdf
Supplementary Figure 1.
Cumulative observed growth risks in development and validation cohort according to low-, moderate-, and high-risk categories of the ELAPSS (Earlier subarachnoid hemorrhage, aneurysm Location, Age, Population, aneurysm Size and Shape) score. Cumulative observed growth risk in validation cohort (solid line) and development cohort (dotted line) across risk categories of the ELAPSS score (low, ≤9; moderate, 10 to 19; high ≥20).
jos-2019-01277-suppl2.pdf

Notes

The authors have no financial conflicts of interest.

Figure 1.
Cumulative risk of aneurysm growth in development and validation cohort.
jos-2019-01277f1.jpg
Figure 2.
Calibration plots: predicted and observed (A) 3-year and (B) 5-year growth risk. Dots represent observed probabilities and bars the corresponding 95% confidence intervals, grouped according to quintiles of predicted probabilities.
jos-2019-01277f2.jpg
Figure 3.
Calibration of the ELAPSS (Earlier subarachnoid hemorrhage, aneurysm Location, Age, Population, aneurysm Size and Shape) score according to low-, moderate-, and high-risk categories. Cumulative observed risk (solid line) and predicted risk (dotted line) across risk categories of the ELAPSS score (low, ≤9; moderate, 10 to 19; high ≥20).
jos-2019-01277f3.jpg
Table 1.
The ELAPSS score for prediction of risk of aneurysm growth
Aneurysm growth risk score Points
Earlier subarachnoid hemorrhage
 Yes 0
 No 1
Location of the aneurysm
 ICA/ACA/ACOM 0
 MCA 3
 PCOM/Posterior circulation 5
Age (yr)
 ≤60 0
 61-65 1
 66-70 2
 71-75 3
 76-80 4
 81-85 5
 86-90 6
 91-95 7
 >95 8
Population
 North America, China, Europe (other than Finland) 0
 Japan 1
 Finland 7
Size of the aneurysm (mm)
 1.0-2.9 0
 3.0-4.9 4
 5.0-6.9 10
 7.0-9.9 13
 ≥10.0 22
Shape of the aneurysm
 Regular 0
 Irregular 4
Reprinted from Backes et al.,[8] with permission from Neurology.
ELAPSS, Earlier subarachnoid hemorrhage, Location of the aneurysm, Age, Population, Size and Shape of the aneurysm; ICA, internal carotid artery; ACA, anterior cerebral artery; ACOM, anterior communicating artery; MCA, middle cerebral artery; PCOM, posterior communicating artery.
Table 2.
Patient characteristics at time of aneurysm detection for the previously reported development cohort [8] and the current validation cohort
Patient characteristic Development cohort [8]
Validation cohort
Growth (n=257) No growth (n=1,250) Total (n=1,507) Growth (n=155) No growth (n=917) Total (n=1,072)
Female sex 175 (68) 876 (70) 1,052 (70) 120 (77) 676 (74) 796 (74)
Mean age (yr) 63 (20-97) 60 (18-91) 61 (18-97) 56 (19-89) 56 (19-85) 56 (19-89)
 ≤40 13 (5) 71 (6) 84 (6) 16 (10) 88 (10) 104 (10)
 41-50 31 (12) 199 (16) 230 (15) 34 (22) 203 (22) 237 (22)
 51-60 60 (23) 350 (28) 410 (27) 44 (28) 279 (30) 323 (30)
 61-70 66 (26) 348 (28) 414 (28) 40 (26) 248 (27) 288 (27)
 71-80 66 (26) 219 (18) 285 (19) 18 (12) 89 (10) 107 (10)
 >80 21 (8) 63 (5) 84 (6) 3 (2) 10 (1) 13 (1)
Earlier subarachnoid hemorrhage 21 (8) 232 (19) 253 (17) 19 (12) 213 (23) 232 (22)
Multiple unruptured aneurysms 81 (32) 232 (19) 313 (21) 42 (27) 243 (27) 285 (27)
Geographical region
 Finland 38 (15) 112 (9) 150 (10) 21 (14) 188 (21) 209 (20)
 Japan 148 (58) 576 (46) 724 (48) 6 (4) 78 (9) 84 (8)
 Other 71 (28) 562 (45) 633 (42) 128 (83) 651 (71) 779 (73)
Values are presented as number (%) or mean (range). Means were chosen to facilitate comparison with the ELAPSS (Earlier subarachnoid hemorrhage, aneurysm Location, Age, Population, aneurysm Size and Shape) development article.
Table 3.
Aneurysm characteristics at time of aneurysm detection for the previously reported development cohort [8] and the current validation cohort
Aneurysm characteristic Development cohort [8]
Validation cohort
Growth (n=267) No growth (n=1,642) Total (n=1,909) Growth (n=199) No growth (n=1,253) Total (n=1,452)
Mean size (mm) 5.5 (1.0-31.4) 3.8 (1.0-32.9) 4.0 (1.0-32.9) 5.8 (1.0-26.6) 4.1 (1.0-50.0) 4.3 (1.0-50.0)
 1.0-2.9 61 (23) 687 (42) 748 (39) 50 (25) 469 (37) 519 (36)
 3.0-4.9 95 (36) 641 (39) 736 (39) 48 (24) 447 (36) 495 (34)
 5.0-6.9 48 (18) 203 (12) 251 (13) 43 (22) 194 (16) 237 (16)
 7.0-9.9 32 (12) 82 (5) 114 (6) 32 (16) 104 (8) 136 (9)
 ≥10.0 31 (12) 29 (2) 60 (3) 26 (13) 39 (3) 65 (5)
Location
 ACA/ACOM 39 (15) 306 (19) 345 (18) 49 (25) 245 (20) 294 (20)
 PCOM 36 (14) 142 (9) 178 (9) 16 (8) 77 (6) 93 (6)
 ICA 59 (22) 490 (30) 549 (29) 49 (25) 313 (25) 362 (25)
 MCA 90 (34) 548 (33) 638 (33) 64 (32) 516 (41) 580 (40)
 Posterior circulation 43 (16) 156 (10) 199 (10) 21 (11) 102 (8) 123 (9)
Irregular shape 64 (24) 192 (12) 256 (13) 61 (31) 180 (14) 241 (17)
Values are presented as mean (range) or number (%). Means were chosen to facilitate comparison with the ELAPSS (Earlier subarachnoid hemorrhage, aneurysm Location, Age, Population, aneurysm Size and Shape) development article.
ACA, anterior cerebral artery; ACOM, anterior communicating artery; PCOM, posterior communicating artery; ICA, internal carotid artery; MCA, middle cerebral artery.

References

1. Vlak MH, Algra A, Brandenburg R, Rinkel GJ. Prevalence of unruptured intracranial aneurysms, with emphasis on sex, age, comorbidity, country, and time period: a systematic review and meta-analysis. Lancet Neurol 2011;10:626-636.
crossref pmid
2. Gabriel RA, Kim H, Sidney S, McCulloch CE, Singh V, Johnston SC, et al. Ten-year detection rate of brain arteriovenous malformations in a large, multiethnic, defined population. Stroke 2010;41:21-26.
crossref pmid
3. Brown RD Jr, Broderick JP. Unruptured intracranial aneurysms: epidemiology, natural history, management options, and familial screening. Lancet Neurol 2014;13:393-404.
crossref pmid
4. Mehan WA Jr, Romero JM, Hirsch JA, Sabbag DJ, Gonzalez RG, Heit JJ, et al. Unruptured intracranial aneurysms conservatively followed with serial CT angiography: could morphology and growth predict rupture? J Neurointerv Surg 2014;6:761-766.
crossref pmid
5. Villablanca JP, Duckwiler GR, Jahan R, Tateshima S, Martin NA, Frazee J, et al. Natural history of asymptomatic unruptured cerebral aneurysms evaluated at CT angiography: growth and rupture incidence and correlation with epidemiologic risk factors. Radiology 2013;269:258-265.
crossref pmid
6. Thompson BG, Brown RD Jr, Amin-Hanjani S, Broderick JP, Cockroft KM, Connolly ES Jr, et al. Guidelines for the management of patients with unruptured intracranial aneurysms: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2015;46:2368-2400.
crossref pmid
7. Steiner T, Juvela S, Unterberg A, Jung C, Forsting M, Rinkel G, et al. European Stroke Organization guidelines for the management of intracranial aneurysms and subarachnoid haemorrhage. Cerebrovasc Dis 2013;35:93-112.
crossref pmid
8. Backes D, Rinkel GJE, Greving JP, Velthuis BK, Murayama Y, Takao H, et al. ELAPSS score for prediction of risk of growth of unruptured intracranial aneurysms. Neurology 2017;88:1600-1606.
crossref pmid
9. Bleeker SE, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol 2003;56:826-832.
crossref pmid
10. Koffijberg H, Buskens E, Algra A, Wermer MJ, Rinkel GJ. Growth rates of intracranial aneurysms: exploring constancy. J Neurosurg 2008;109:176-185.
crossref pmid
11. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD). Ann Intern Med 2015;162:735-736.
crossref
12. Brinjikji W, Pereira VM, Khumtong R, Kostensky A, Tymianski M, Krings T, et al. PHASES and ELAPSS scores are associated with aneurysm growth: a study of 431 unruptured intracranial aneurysms. World Neurosurg 2018;114:e425-e432.
crossref pmid
13. Brinjikji W, Zhu YQ, Lanzino G, Cloft HJ, Murad MH, Wang Z, et al. Risk factors for growth of intracranial aneurysms: a systematic review and meta-analysis. AJNR Am J Neuroradiol 2016;37:615-620.
crossref pmid pmc
14. Backes D, Rinkel GJ, Laban KG, Algra A, Vergouwen MD. Patient- and aneurysm-specific risk factors for intracranial aneurysm growth: a systematic review and meta-analysis. Stroke 2016;47:951-957.
crossref pmid
15. Tominari S, Morita A, Ishibashi T, Yamazaki T, Takao H, Murayama Y, et al. Prediction model for 3-year rupture risk of unruptured cerebral aneurysms in Japanese patients. Ann Neurol 2015;77:1050-1059.
crossref pmid
16. Backes D, Hendrikse J, van der Schaaf I, Algra A, Lindgren AE, Verweij BH, et al. Determinants of gadolinium-enhancement of the aneurysm wall in unruptured intracranial aneurysms. Neurosurgery 2018;83:719-725.
crossref pmid pdf
17. Malhotra A, Wu X, Forman HP, Matouk CC, Gandhi D, Sanelli P. Management of tiny unruptured intracranial aneurysms: a comparative effectiveness analysis. JAMA Neurol 2018;75:27-34.
crossref pmid


ABOUT JoS
AUTHOR INFORMATION
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
Editorial Office
Department of Neurology, Asan Medical Center,Ulsan University College of Medicine
88, Olympic-ro 43-gil, Songpa-gu, Seoul 05505, Korea
Submission, status and progress, etc ⟫ E-mail: editor@j-stroke.org
Website and system ⟫ E-mail: journal@m2community.co.kr
Publishing company ⟫ E-mail: ka72sus@smileml.com
Developed in M2PI
Copyright © 2024 by Korean Stroke Society.
Close layer
prev next