Estimation of Acute Infarct Volume with Reference Maps: A Simple Visual Tool for Decision Making in Thrombectomy Cases
Article information
Abstract
Background and Purpose
Thrombectomy within 24 hours can improve outcomes in selected patients with a clinical-infarct mismatch. We devised an easy-to-use visual estimation tool that allows infarct volume estimation in centers with limited resources.
Methods
We identified 1,031 patients with cardioembolic or large-artery atherosclerosis infarction on diffusion-weighted images (DWIs) obtained before recanalization therapy and within 24 hours of onset, and occlusion of the internal carotid or middle cerebral artery. Acute DWIs were mapped onto a standard template and used to create visual reference maps with known lesion volumes, which were then used in a validation study (with 130 cases) against software estimates of infarct volume.
Results
The DWI reference map chart comprises 144 maps corresponding to 12 different infarct volumes (0.5, 1, 2, 3, 5, 7, 9, 11, 13, 15, 17, and 19 mL) in each of 12 template slices (Montreal Neurological Institute z-axis –15 to 51 mm). Infarct volume in a patient is estimated by selecting a slice with a similar infarct size at the corresponding z-axis level on the reference maps and then adding up over all slices. The method yielded good correlations to software volumetrics and was easily learned by both experienced and junior physicians, with approximately 1 to 2 minutes spent per case. The sensitivity, specificity, and accuracy for detecting threshold infarct volumes (<21, <31, and <51 mL) were very high (all about >90%).
Conclusions
We developed easy-to-use reference maps that allow prompt and reliable visual estimation of infarct volumes for triaging patients to thrombectomy in acute stroke.
Introduction
The Diffusion-weighted Imaging or Computed Tomography Perfusion Assessment with Clinical Mismatch in the Triage of Wake Up and Late Presenting Strokes Undergoing Neurointervention with Trevor (DAWN) trial demonstrated the efficacy and safety of endovascular thrombectomy performed 6 to 24 hours after the onset of stroke [1,2]. The trial included patients with occlusion of the intracranial internal carotid artery or proximal middle cerebral aretery, who had a mismatch between the severity of the clinical deficit and the infarct volume. Infarct volume was assessed by processing diffusionweighted magnetic resonance images (diffusion-weighted images [DWIs]) or perfusion computed tomography images with automated software. The cutoff infarct volumes in the definition of mismatch were 21, 31, and 51 mL, depending on the National Institutes of Health Stroke Scale scores (≥10 or ≥20) and age (<80 or ≥80 years) [1].
However, software packages that quantify infarct volume may not be affordable to every stroke center worldwide. Appropriately triaging a patient to the correct group for thrombectomy does not require an absolute infarct volume estimate; it only requires choosing the correct volume category. Thus, we hypothesized that an easy-to-use visual estimation tool that allows for the categorization of infarct volumes to clinically useful ranges could be helpful for triaging patients with acute stroke, depending on the (clinical) practice environment of an individual hospital (or stroke center).
We previously provided an objective visual grading system for white matter hyperintensity (WMH) [3]. To facilitate the use of our data as the reference standard for grading WMH, we plotted the data as frequency-volume (FV) maps that demonstrate the statistical volume and spatial distribution in a single format. The system (Kim statistical WMH scoring system) reflects the prevalence of WMH, enabling physicians to estimate the degree of brain WMH volume as the percentile rank in the stroke population [3-5].
In the present study, we aimed to develop an easy-to-use reference map system that allows a prompt visual estimation of DWI lesion volumes, with practically acceptable levels of accuracy and precision, for the clinical management of acute stroke patients who can be candidates for mechanical thrombectomy at 6 to 24 hours after onset, based on the results of the DAWN trial.
Methods
Study population
This is a multicenter study that involved 11 academic stroke centers participating in the Korean Nationwide Image-based Stroke Database Project [6]. From May 2011 to February 2014, we consecutively enrolled 1,031 acute stroke patients (Supplementary Figure 1) with (1) cardioembolic or large-artery atherosclerosis infarction on DWIs that were obtained before recanalization therapy and within 24 hours after symptom onset, and (2) occlusion of the internal carotid artery or middle cerebral artery. All patients or their legally authorized representatives provided written informed consent. The Institutional Review Board of Dongguk University Ilsan Hospital approved the project.
Brain magnetic resonance imaging and quantitative image registration
Brain magnetic resonance imaging (MRI) was performed on a 1.5-T (n=745) or 3.0-T (n=286) MRI system. The diffusion-weighted MRI protocols were as follows: b-values 0 and 1,000 s/mm2, repetition time 2,400 to 9,000 ms, echo time 50 to 99 ms, voxel size 1×1×3 to 1×1×5 mm3, interslice gap 0 to 2 mm, and thickness 3 to 7 mm. All images were transferred to the Korean Brain MRI Data Center for central data storage and quantitative analysis. As previously reported [3,4,6], DWIs were converted to a patient-independent quantitative visual format. Briefly, brain template images (1×1×1 mm3 voxels) were chosen from the Montreal Neurological Institute (MNI) template within the range of –63.5 to 74.5 mm in the z-axis of the Talairach space. After the normalization of images, each patient’s high-signal-intensity lesions on DWIs were semiautomatically segmented and registered onto the brain templates, under meticulous supervision by a vascular neurologist (W.S.R.). The infarct percentage (per brain parenchymal volume) on DWIs was calculated as a percentage of brain volume by dividing the number of voxels in the lesions over the total number of parenchymal brain voxels, with corrections applied to account for the differences in scan slice thicknesses by adjusting the denominators [3,4,6]. Considering that the reported mean brain volume of an elderly Korean population was 1,170 mL [7], the total infarct volume in each patient’s registration data was approximated as follows: infarct percentage × 1,170 / 100.
Statistical anatomic maps: DWI lesion FV maps
To display our data as visual maps that could be used as a reference for estimating DWI lesion volume, we used 12 MNI template slices (centered on –15, –9, –3, 3, 9, 15, 21, 27, 33, 39, 45, and 51 mm on the z-axis) covering the levels of the striatocapsular, corona radiata, and centrum semiovale regions. These slices were chosen, on the basis of clinical experience, as being useful in the estimation of anterior circulation infarct volume, and are representative visual images of a larger data set comprising the whole brain [3]. First, we plotted the frequency (incidence) of DWI lesions at every voxel coordinate of each slice as a heat map. Then, we plotted the data as FV maps to show the statistical volume and spatial distribution in a single format, arranged to reflect the distribution of disease in our study population, as previously reported [3], with some modifications. The 256 color-coded FV maps were processed to generate the final red color-overlaid “reference maps.”
Visual estimation of infarct volume with the reference maps
By using the reference maps that correspond to 12 different infarct volumes (0.5, 1, 2, 3, 5, 7, 9, 11, 13, 15, 17, and 19 mL) in each of the 12 template slices, an experienced vascular neurologist (W.S.R.) and a first-year neurology resident (P.K.) independently estimated infarct volumes in 130 patients who were randomly chosen from the Korean Nationwide Image-based Stroke Database Project. For DWIs with 5 to 7 mm thickness (total number of slices approximately 24), all 12 supratentorial slices were used. For DWIs with 3 mm thickness, half of all supratentorial slices (n=12) were used, based on the proximity to the template slices. In the selection of the closest reference image, the first step was to find the DWI slice with the first appearance of the lateral ventricle, corresponding to slice #5 in the reference maps. Once the first matching was done, the other matchings were subsequently determined by the DWI slice thickness itself; that is, every consecutive reference image for 5 to 7 mm thickness versus every other consecutive reference image for 2 to 4 mm thickness. For each DWI slice, infarct volume was estimated by selecting a slice with a similar infarct size at the corresponding z-axis level in the reference maps. If the total infarct volume in a slice was <0.5 mL, the estimated infarct volume in the slice was assigned as 0.3 mL. All infarct slice volumes were then summed to obtain the “estimated infarct volume” in each patient. To generate a learning curve for the estimation of infarct volume based on the reference maps, we measured the time spent on estimating the infarct volume.
Calculation of infarct volumes with the original DWIs
A vascular neurologist (H.G.J.) who was blinded to the study hypothesis computed the “measured infarct volume” in each patient by using Analyze 12.0, a software package for biomedical image analysis (Biomedical Imaging Resource, New York, NY, USA). The regions of interest were segmented using the Region Grow in the Volume Edit module, with manual removal of artifacts when necessary. The total volume was reported in mL.
Statistical analysis
We compared the “estimated infarct volume” and the “measured infarct volume” by using Pearson correlation analysis and Bland-Altman plots. We also calculated the sensitivity, specificity, and accuracy of the reference map-based estimation of infarct volumes in terms of the classification of the infarct volumes as <21, <31, and <51 mL. In addition, we included the cutoff point of 70 mL to also reflect the eligibility criteria of the Endovascular Therapy following Imaging Evaluation for Ischemic Stroke (DEFUSE 3) trial [8]. A P<0.05 was considered statistically significant. All analyses were performed using STATA version 14.0 (STATA Corp., College Station, TX, USA).
Results
Patients
In this study on 1,031 patients with acute (<24 hours) cerebral infarction due to the occlusion of the anterior circulation large arteries, the mean age was 70.4±12.4 years and 55% of the patients were men (Table 1). The occluded vessel was the middle cerebral artery in 573 patients (56%) and the internal carotid artery in 458 patients (44%). The stroke subtype was large-artery atherosclerosis in 579 patients (56%) and cardioembolism in 452 patients (44%). The median onset-to-imaging time was 14 hours (interquartile range, 6 to 21). The median infarct volume was 0.93% (interquartile range, 0.29% to 4.18%) of the total brain volume, which corresponds to 10.9 mL (interquartile range, 3.4 to 48.9).
Baseline characteristics of 1,031 patients with acute stroke due to anterior circulation large-artery occlusion
Reference maps
Supplementary Figure 2 shows the results of a combined mapping of both spatial extent and frequency of occurrence of acute DWI lesions, and Figure 1 shows the final reference maps. The number above each brain image indicates the corresponding infarct volume in the “hemisphere” (or one-half) of the brain. As aforementioned, the estimation of infarct volume on a clinical DWI section could be performed by selecting a slice with a similar infarct size at the corresponding z-axis level in the reference maps. If a DWI section has multiple infarcts with different volumes, the total lesion volume in the section may have to be calculated by finding multiple slices that represent the different volumes at the same z-axis level.
Reference maps for visual estimation of acute infarct volume. The topographic frequency-volume maps of acute diffusion-weighted imaging (DWI) lesions (Supplementary Figure 2) were processed to generate these final reference maps. The number above each brain image indicates the corresponding infarct volume (mL) in one “hemisphere” of the brain; the lesion volumes of both hemispheres on each brain image are the same. The thickness of the map slices is 6 mm. The visual estimation of infarct volume on a clinical DWI section is performed by selecting a slice with a similar infarct size at the corresponding z-axis level in the reference maps. After all infarct volumes in every slice are summed, the formula at the bottom of this figure is used for adjusting the sum value to improve the accuracy
The color coding within each infarcted volume in Supplementary Figure 2 represents the hemisphere-wise infarct frequency at each voxel, reflecting that a red (vs. blue) voxel had a higher (vs. lower) number of patients with a lesion in the voxel.
Visual estimation of infarct volume with the reference maps vs. computer-assisted volume measurement with an image analyzer
The “estimated infarct volumes” that were obtained by a vascular neurologist using reference maps correlated well to the “measured infarct volumes” that were obtained by another vascular neurologist using the Analyze software (r=0.977, P<0.001) (Figure 2A). A Bland-Altman plot showed good agreement between the two methods; only six of the 130 cases (4.6%) were outside the limits of agreement (Figure 2B). The disagreement was more prominent when infarct volumes were larger; if the infarct volumes measured using the Analyzebased method were >50 mL, the reference map-based method estimated them to be about 40% smaller.
Correlations between estimated vs. measured infarct volumes. (A) Each dot indicates one patient’s datum (n=130), representing a single reader estimate of volume (mL) versus the actual volume (mL). The clustering close to the cutoffs suggests a “categorical” effect. Blue, green, and red dotted lines indicate infarct volume thresholds of 21, 31, and 51 mL. The red dots indicate the data that were misclassified (for the classification of volumes of >21, >31, and >51 mL) by the visual estimation based on the reference maps (Figure 1) versus the measurement with an image analyzer software package (Analyze 12.0). The gray dotted line indicates a perfect estimation slope (x=y). Owing to a skewed distribution of infarct volumes, the x-axis and y-axis were logarithmically transformed for better visualization. (B) A Bland-Altman plot showing that the error in estimation increases with infarct volume, and that estimates for larger volumes are biased to underestimation. The dotted line indicates the mean difference between the estimated infarct volumes and the measured infarct volumes. A gray area indicates 95% limits of agreement. A logarithmic scale is used for the x-axis of the Bland-Altman plot.
When we divided the patients according to the DWI slice thickness (3, 5, 6, and 7 mm), a Bland-Altman plot again showed good agreement between the two methods, although the reference map-based method again tended to underestimate infarct volumes when they were large (Supplementary Figure 3).
The sensitivity, specificity, and accuracy of the reference map-based estimation of infarct volumes in terms of the classification of the infarct volumes as <21, <31, and <51 mL were high (88.9%, 100%, and 95.4%; 85.1%, 100%, and 94.6%; and 86.5%, 100%, and 96.2%, respectively) (Table 2 and Figure 2A). The kappa value between the classification by the two methods was 0.75 (P<0.001), indicating a good agreement. In addition, the sensitivity, specificity, and accuracy of the reference map-based estimation of infarct volumes in terms of the classification of the infarct volumes as <70 mL versus ≥70 mL were also high (77.8%, 100%, and 95.4%, respectively).
Categorization of infarct volumes after visual estimation with reference maps vs. measurement with an image analyzer
As described before, we observed that infarct volumes estimated using the reference maps tended to be smaller than infarct volumes measured using an image analyzer. To improve the performance of the infarct volume estimation and classification based on the reference maps, we generated the following simple adjustment formula after considering the aforementioned infarct volume-related non-linear pattern of the errors as well as the cutoff DWI infarct volumes (21, 31, and 51 mL): multiply the sum of all estimated infarct volumes by 1.1, 1.2, 1.3, or 1.4 for the sum value of <21, 21 to <31, 31 to <51, or ≥51 mL, respectively. The adjusted estimated infarct volumes better approximated to the measured infarct volumes (Figure 3A). The mean difference in the infarct volumes that were provided by the reference map-based method versus the Analyzebased method decreased from 14.5 to 2.6 mL. Only five cases (3.8 %) were outside the limits of agreement (Figure 3B). Accordingly, the sensitivity, specificity, and accuracy of the reference map-based estimation of infarct volumes in terms of the classification of the infarct volumes as <21, <31, and <51 mL became higher (90.7%, 100%, and 96.2%; 93.6%, 98.8%, and 96.9%; and 100%, 100%, and 100%, respectively) (Figure 3A and Supplementary Table 1). The kappa value between the classification by the two methods was 0.89 (P<0.001), indicating an excellent agreement. In addition, the sensitivity, specificity, and accuracy of the reference map-based estimation of infarct volumes in terms of the classification of the infarct volumes as <70 mL versus ≥70 mL again became higher (92.3%, 94.2%, and 93.9%, respectively).
Closer correlations between “adjusted” estimated infarct volumes versus measured infarct volumes. (A) Each dot indicates one patient’s datum. Blue, green, and red dotted lines indicate infarct volume thresholds of 21, 31, and 51 mL. The red dots indicate the data that were misclassified (for the classification of volumes of >21, >31, and >51 mL) by the visual estimation based on the reference maps and the post hoc adjustment (Figure 1) versus the measurement with an image analyzer software package (Analyze 12.0). The gray dotted line indicates a perfect estimation slope (x=y). Owing to a skewed distribution of infarct volumes, the x-axis and y-axis were logarithmically transformed for better visualization. (B) A Bland-Altman plot. The dotted line indicates the mean difference between the adjusted estimated infarct volumes and the measured infarct volumes. A gray area indicates 95% limits of agreement. A logarithmic scale is used for the x-axis of the Bland-Altman plot.
The post hoc adjustment-related improvement of the infarct volume estimation was proved in a validation study using a different set of data (n=30). The adjusted estimated infarct volumes again closely correlated with the measured infarct volumes (r=0.979, P<0.001) (Supplementary Figure 4). A Bland-Altman plot again showed a high level of agreement between the two methods; only three of the 30 cases (10%) were outside the limits of agreement (Supplementary Figure 5). The sensitivity, specificity, and accuracy of the reference map-based estimation of infarct volumes in terms of the classification of the infarct volumes as <21, <31, and <51 mL were also high (93.8%, 92.9%, and 93.3%; 91.7%, 94.4%, and 93.3%; and 100%, 100%, and 100%, respectively) (Supplementary Table 2 and Supplementary Figure 4). The kappa value between the classification by the two methods was 0.80 (P<0.001). In addition, the sensitivity, specificity, and accuracy of the reference map-based estimation of infarct volumes in terms of the classification of the infarct volumes as <70 mL versus ≥70 mL were all 100%.
Visual estimation of infarct volumes by a vascular neurologist vs. that by a first-year resident
The infarct volumes estimated by a vascular neurologist and a first-year resident using the reference maps were reliable when assessed using a Bland-Altman plot; 10 of 130 (7.7%) patients were outside the limits of agreement, and the mean difference between the two raters was 0.8 mL (Supplementary Figure 6). In addition, the correlation plot between the estimated volumes by the first-year resident and the measured volumes appeared similar to that by the vascular neurologist (Supplementary Figure 7). The times taken for estimating infarct volumes were short (1 to 2 min/case) and showed a decreasing trend over experiences: 42, 41, 36, 30, 31, and 29 minuts per 20 patients for the vascular neurologist versus 29, 25, 26, 24, 20, and 16 minutes per 20 patients for the first-year resident (Supplementary Figure 8).
Discussion
In the present study, we developed reference maps and showed the potential clinical utility of the map chart that enables the visual estimation of infarct volumes (Supplementary Video 1) in patients with acute supratentorial ischemic stroke due to large-artery atherosclerotic or cardioembolic occlusion of the middle cerebral artery or internal carotid artery. There was a close correlation between the reference map-based visual estimation of infarct size and the image analyzer-based volume measurement, particularly after applying the post hoc adjustment formula to our map-based method. The estimated infarct volumes correlated well between a vascular neurologist and a first-year resident, and it took only 1 to 2 minutes for the estimation of one patient’s DWIs after a short learning period. In addition, the threshold infarct volumes used in previous randomized clinical trials for image-based thrombectomy were well differentiated by using the reference maps.
This study has limitations. First, we did not compare the estimation of infarct volume by using the reference maps with the volume measurement by using the identical software packages used in clinical trials for image-based thrombectomy [1]. However, the software used (Analyze) has a long history in multiple publications [9-11], and would be expected to perform similarly to that used in the DAWN trial.
Second, our findings are limited to DWI infarct estimation; further investigations are required to study whether the reference maps could be used for the measurement of perfusion CT-derived infarct core volumes.
Third, the reference map-based method tended to underestimate infarct volumes. The reference maps were generated using the registered images on the MNI template set after normalization of the original DWIs and warping of segmented infarct lesions. The use of 1,170 mL as the brain volume as well as the image registration-related process may be partly responsible for the disagreement; we speculate that the average brain volume in our study population may be >1,170 mL. Hence, we devised an adjustment formula, which improved the performance of the visual estimation of infarct volume with our map-based method. The post hoc adjustment seems rather arbitrary; however, the purpose of this study was to develop an easy-to-use reference map system that allows a prompt estimation of DWI lesion volumes with practically acceptable levels of accuracy and precision. Thus, it is notable that after the adjustment, the sensitivity, specificity, and accuracy of the map-based visual estimation of infarct volumes in terms of the classification of the infarct volumes as <21, <31, and <51 mL (i.e., the cutoff infarct volumes; see also Figure 4: maps that are representative of 21, 31, and 51 mL infarct volumes) were almost 90% to 100%. In addition, the DEFUSE criteria (70 mL cutoff) could also be easily accommodated. The post hoc adjustment still cannot overcome the inherent limitation that is related to our approach based on the estimation of an absolute lesion volume by multiplying a lesion-to-brain ratio with a fixed number: the average brain volume. However, choosing the correct volume category, which is currently required for triaging patients to the correct group for thrombectomy, may be less stringent than securing an absolute infarct volume estimate.
Reference maps that are representative of the infarct volumes of 21, 31, and 51 mL. Representative reference maps that correspond to 21 mL (upper), 31 mL (middle), and 51 mL (lower) infarct volumes (adjusted values; see the main text for the adjustment formula). The maps reflect not only volumetric information but also the spatial distribution that represents the voxel-wise lesion frequency in the stroke population of this study.
Fourth, the results of this study may not be directly applicable to other centers or ethnic groups. The thickness of DWIs in our multicenter cohort was mostly 6 or 3 mm. Thus, the clinically acceptable validity of our method might possibly be impeded when a very different slice thickness is used as a scanning protocol of DWIs.
Despite the limitations, we believe that the reference mapbased estimation of DWI lesion volumes has practically acceptable levels of accuracy and precision, allowing for thrombectomy-related clinical management of patients with acute stroke in centers with limited resources.
Supplementary materials
Supplementary materials related to this article can be found online at https://doi.org/10.5853/2018.03202.
Supplementary Video 1.
Illustration of the reference map-based visual estimation of infarct volume.
Improved categorization of infarct volumes after adjustment* of the reference map-based volume estimation versus volume measurement by using an image analyzer
A validation study (n=30) to test the improved categorization of infarct volumes by using the reference map-based volume estimation with adjustment* versus volume measurement by using an image analyzer
Study flowchart. MRI, magnetic resonance imaging.
Topographic frequency-volume maps to generate the final reference maps (Figure 1) for visual estimation of acute infarct volumes. Combined mapping of both the spatial extent and the frequency of occurrence of acute diffusion-weighted magnetic resonance imaging lesions, with the number above each brain image indicating the corresponding infarct volume (mL) in the “hemisphere” of the brain; the lesion volumes of both hemispheres on each brain image are the same. The color coding represents the hemisphere-wise infarct frequency at each voxel. The thickness of the map slices is 6 mm. Max=maximum frequency. Min=minimum frequency higher than zero.
Bland-Altman plots, with stratification by the slice thickness of diffusion-weighted magnetic resonance images (A, 3 mm; B, 5 mm; C, 6 mm; D, 7 mm). The dotted line indicates the mean difference between estimated infarct volumes (obtained using the reference maps) and measured infarct volumes (obtained using Analyze 12.0, an image analysis software package). The gray area indicates 95% limits of agreement. A logarithmic scale is used for the x-axis. The number of patients in each group (3, 5, 6, and 7 mm) was 23, 24, 59, and, 24, respectively. The number of patients outside the limits of agreement is 1 (4.4%), 2 (8.3%), 5 (8.5%), and 2 (8.3%), respectively.
Correlations between "adjusted" estimated infarct volumes versus measured infarct volumes. Each dot indicates one patient’s datum. Blue, green, and red dotted lines indicate infarct volume thresholds of 21, 31, and 51 mL. The red dots indicate the data that were misclassified (for the classification of volumes of >21, >31, and >51 mL) by the visual estimation based on the reference maps and the post hoc adjustment (Figure 1) versus the measurement using an image analyzer (Analyze 12.0). The gray dotted line indicates perfect estimation slope (x=y). Owing to a skewed distribution of infarct volumes, the x-axis and y-axis were logarithmically transformed for better visualization.
A Bland-Altman plot of "adjusted" estimated infarct volumes vs. measured infarct volumes in a validation study (n=30). The dotted line indicates the mean difference between the estimated infarct volumes (obtained using the reference maps with the post hoc adjustment in Figure 1) and the measured infarct volumes (obtained using Analyze 12.0, an image analysis software package). The gray area indicates 95% limits of agreement. A logarithmicscale is used for the x-axis. Three patients (10%) are outside the limits of agreement.
A Bland-Altman plot for the reference map-based estimation of infarct volumes by a vascular neurologist versus a first-year neurology resident. The dotted line indicates the mean difference between the estimated infarct volumes by an experienced vascular neurologist and those by a first-year neurology resident. The gray area indicates 95% limits of agreement. A logarithmic scale is used for the x-axis. Ten patients (7.7%) are outside the limits of agreement.
A correlation plot of the infarct volumes estimated by a first-year resident using the reference maps vs. the infarct volumes measured by a vascular neurologist using an image analysis software package. Each dot indicates one patient’s datum. Blue, green, and red dotted lines indicate infarct volume thresholds of 21, 31, and 51 mL. The colored dots indicate the data that were misclassified for the classification of volumes of >21, >31, and > 51 mL. The gray dotted line indicates a perfect estimation slope (x=y). Owing to a skewed distribution of infarct volumes, the x-axis and y-axis were logarithmically transformed for better visualization.
Learning curve for the infarct volume estimation with the reference maps: vascular neurologist versus first-year resident. Each round is for 20 patients’ diffusion-weighted magnetic resonance images (cases).
Notes
Disclosure
The authors have no financial conflicts of interest.
Acknowledgements
This study was supported by grants from the National Center for Standard Reference Data, Korean Agency for Technology and Standards, Ministry of Trade, Industry & Energy, Ministry of Health & Welfare (HI12C1847; Korea Healthcare Technology R&D Project), and the Global Research Lab program (NRF2015K1A1A2028228) of the National Research Foundation, funded by the Korean government, Republic of Korea.
We appreciate the contributions of all members of the Clinical Research Center for Stroke in Korea (CRCS-K) to this study.