Article Text
PDF
Abstract
Objectives To evaluate a predictive model for robust estimation of daily out-of-hospital cardiac arrest (OHCA) incidence using a suite of machine learning (ML) approaches and high-resolution meteorological and chronological data.
Methods In this population-based study, we combined an OHCA nationwide registry and high-resolution meteorological and chronological datasets from Japan. We developed a model to predict daily OHCA incidence with a training dataset for 2005–2013 using the eXtreme Gradient Boosting algorithm. A dataset for 2014–2015 was used to test the predictive model. The main outcome was the accuracy of the predictive model for the number of daily OHCA events, based on mean absolute error (MAE) and mean absolute percentage error (MAPE). In general, a model with MAPE less than 10% is considered highly accurate.
Results Among the 1 299 784 OHCA cases, 661 052 OHCA cases of cardiac origin (525 374 cases in the training dataset on which fourfold cross-validation was performed and 135 678 cases in the testing dataset) were included in the analysis. Compared with the ML models using meteorological or chronological variables alone, the ML model with combined meteorological and chronological variables had the highest predictive accuracy in the training (MAE 1.314 and MAPE 7.007%) and testing datasets (MAE 1.547 and MAPE 7.788%). Sunday, Monday, holiday, winter, low ambient temperature and large interday or intraday temperature difference were more strongly associated with OHCA incidence than other the meteorological and chronological variables.
Conclusions A ML predictive model using comprehensive daily meteorological and chronological data allows for highly precise estimates of OHCA incidence.
- cardiac arrest
Data availability statement
No data are available. The All-Japan Utstein Registry of the FDMA is a publicly accessible open database. The availability of the Kobe Municipal Fire Department database, which includes detailed information on the location of cardiac arrest used with permission for this study, is restricted. The Weather Company Data Packages from the Weather Company is subject to a full license agreement, which does not permit data sharing outside of the research team.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Out-of-hospital cardiac arrest (OHCA) is becoming a substantial worldwide health burden.1 2 A systematic review of the international epidemiology of OHCA from 1991 to 2007 reported that the estimated incidence of emergency medical services (EMS)-attended OHCA per 100 000 person-years was 86.4 in Europe, 98.1 in North America, 52.5 in Asia and 112.9 in Australia. The percentage of patients with survival to discharge is extremely low: 9.4% in Europe, 6.3% in North America, 2.2% in Asia and 10.7% in Australia.2 Accurately predicting the daily incidence of OHCA may provide a significant public benefit. Since the incidence of OHCA is affected by meteorological conditions,3–10 the application of high-resolution meteorological data to medicine might provide ways to improve predictions of the daily incidence of OHCA.
Machine learning (ML) has recently emerged as a novel approach to integrate multiple quantitative variables to improve diagnosis and accuracy of incidence predictions in cardiovascular medicine.11–13 Since meteorological data are very extensive and complex, ML can help identify associations not identified by conventional one-dimensional statistical approaches. By combining OHCA data with high-resolution meteorological data, such as daily forecasts, ML could use advanced analytics to build a warning system for individuals potentially at risk for OHCA of cardiac origin through internet of things (IoT) devices.
This study presents a predictive model for robust estimation of daily OHCA incidence of cardiac origin using a suite of ML approaches. This model was evaluated using a nationwide database of OHCA, as well as comprehensive meteorological data and chronological data.
Methods
Study design and setting
We matched two datasets between 1 January 2005 and 31 December 2015 at the hour level based on the time of the emergency call: the All-Japan Utstein Registry of the Fire and Disaster Management Agency (FDMA) dataset on patients with OHCA of cardiac origin and a meteorological dataset from the Weather Company, an IBM Business (Atlanta, Georgia, USA). We classified data from 1 January 2005 to 31 December 2013 in this merged dataset as the training dataset for developing the predictive models and data from 1 January 2014 to 31 December 2015 as the testing dataset for assessing whether the predictive models can work in other years. Japan has an area of approximately 378 000 km2 and its population was approximately 127 million in 2005.14 Online supplemental figure 1 shows three representative cities located at different latitudes in Japan: Sapporo at N43°, Kobe at N34° and Naha at N26°.
Supplemental material
We performed a four-step analysis. First, we elucidated the association between the incidence of OHCA and daily meteorological and chronological variables. Second, we developed an ML predictive models for OHCA incidence based on daily meteorological data, chronological data and combined meteorological and chronological data from the training dataset.15 16 Third, we examined the concordance between the predicted incidence of OHCA based on the ML model and the observed incidence of OHCA in a testing dataset. To further examine concordance at the district level after the time period covered by the original dataset, we performed heatmap analysis using another dataset on the location of OHCA in Kobe city between 1 January 2016 and 31 December 2018. The Kobe Municipal Fire Department has detailed information about where OHCA events occurred in certain districts. The population of Kobe city is more than 1.5 million. Its age distribution (population pyramid) is similar to that of Japan overall. Finally, we investigated the relative strength of the associations between meteorological variables and the incidence of OHCA in each predictive model. The main outcome was the daily incidence of OHCA.
A subcommittee for resuscitation science in the Japanese Circulation Society was provided with registry data following the prescribed governmental procedures.
OHCA dataset
Patients with OHCA of cardiac origin in the FDMA’s All-Japan Utstein Registry were included. The All-Japan Utstein Registry is a prospective, population-based, nationwide registry of patients who have had an OHCA event. Data were prospectively recorded using the internationally standardised Utstein template.17 The following patient information was collected and analysed: age, sex, aetiology of arrest (ie, cardiac or non-cardiac) and time of the emergency call. All event times were synchronised with the dispatch centre clock. In Japan, all patients with OHCA who received prehospital resuscitation efforts by EMS personnel are transported to a hospital because they are not permitted to terminate resuscitation in the field. Data were stored on the FDMA registry database server and checked for missing values. If a data form was incomplete, the FDMA returned it to the respective fire station for completion. The registry has yielded some findings about patients with OHCA.14 18–21 Details of the registry are described in the online supplemental materials.
Meteorological and chronological dataset
We analysed meteorological data from the Weather Company (https://www.ibm.com/weather) that operates a weather forecasting service platform (online supplemental materials). Between 1 January 2005 and 31 December 2015, the resolution of the meteorological data was 30 km gridded points (Weather Company Data Packages). In 2016, the resolution improved to 4 km gridded points. Meteorological variables included ambient temperature (°C), relative humidity (%), precipitation during the previous hour (mm), snowfall (mm), cloud coverage (%), wind speed (kph) and atmospheric pressure (hPa). Chronological variables included year (2005 was considered year 0), season (spring: March–May; summer: June–August; autumn: September–November; winter: December–February, with winter coded as the reference value), day of the week (with Sunday coded as the reference value), holidays and Japanese holiday season from 29 December to 6 January (categorical variable with a value of 0 or 1).
Development of predictive models
To develop predictive models for the daily incidence of OHCA, we used the the eXtreme Gradient Boosting (XGBoost) algorithm, which is an optimised distributed gradient boosting library widely used by data scientists for many ML challenges.11–13 Hyperparameters of the XGBoost algorithm were chosen to maximise the predictive ability of the model using fourfold cross-validation. In fourfold cross-validation, we classified our dataset into four groups, and the XGBoost algorithm fitted decision trees to three groups and used the remaining group for validation. This procedure was performed four times with a different validation group each time. Population size for each prefecture was included in the XGBoost algorithm as an offset term. We did not use ambient air pressure to avoid possible multicollinearity.
Statistical analysis
The characteristics of present dataset were summarised with medians and IQRs for continuous variables and numbers and percentages for categorical variables by prefecture and day in the training and testing datasets. The generalised linear models (GLMs) based on the Poisson distribution investigated the associations between meteorological variables and daily OHCA incidence by prefecture using all data in univariable models and a multivariable model. We exponentiated regression coefficients and 95% CIs to present incidence rate ratios (IRRs) for estimated OHCA incidence with each 1-unit increase in a meteorological variable. A p value of less than 0.05 was considered to indicate a significant difference.
We evaluated the predictive accuracy of the predictive models based on mean absolute error (MAE) and mean absolute percentage error (MAPE) between predicted values calculated by the predictive models and observed daily OHCA incidence by prefecture. MAE reflects the average magnitude of differences between predicted values and observed values. MAE ranges from zero to infinity. Lower MAE values indicate higher predictive performance. MAPE is generally used as a measure of the predictive accuracy of a forecasting method. It is an average of the absolute values of errors divided by observed values. MAPE ranges from 0% to 100%. Lower MAPE values indicate higher model predictive performance. In general, MAPE less than 10% is considered highly accurate predicting.22 We also calculated correlation coefficients, which can range from −1.00 to 1.00. Higher absolute values indicate higher model predictive performance.
We investigated the relative strength of the associations between each meteorological variable and OHCA incidence in the ML predictive model using a Shapley Additive Explanations (SHAP) algorithm.23 For a given set of feature values, a SHAP value reflects how much a single variable, in the context of its interaction with other variables, contributes to the difference between the actual prediction and the mean prediction.
All statistical analyses were performed with R statistical software V.3.5.0 (https://www.R-project.org/) and the xgboost package for R V.0.71.2 (https://CRAN.R-project.org/package=xgboost). Full analysis dataset had no missing information in any key variables.
Results
Characteristics of the training and testing datasets
Among the 1 299 784 OHCA cases in the All-Japan Utstein Registry between 2005 and 2015, there were 661 052 OHCA cases of cardiac origin matched with meteorological data; 525 374 cases were in the training dataset on which fourfold cross-validation was performed, and 135 678 cases were in the testing dataset. The characteristics of the datasets are summarised in table 1. Between 2005 and 2015, the median daily incidence of OHCA increased from 133 (IQR 109–167) to 173 cases (IQR 146–216) and the median annual incidence of OHCA increased from 44.5 to 59.7 per 100 000 person-years. The median age of OHCA onset increased from 77 (IQR 66–85) to 80 years (IQR 70–87), and the proportion of males decreased from 59% to 57%. Various meteorological changes were observed in representative prefectures located at different latitudes over the study period. The median of the mean ambient temperature within a day increased from 5.6°C (IQR −2.8 to 14.8) to 7.1°C (IQR −1.5 to 14.6) in Sapporo at N43°. This trend was not observed in Kobe at N34° or Naha at N26°. Differences between maximum and minimum ambient temperatures within a day decreased from 7.0°C (4.1–10.2) to 6.4°C (4.1–9.2) in Sapporo, but increased from 5.0°C (3.5–6.4) to 5.4°C (4.0–6.9) in Kobe.
Characteristics of daily data in the training (2005–2013) and testing (2014–2015) datasets
Association between meteorological and chronological variables and OHCA incidence
Online supplemental figure 2 shows the incidence of OHCA by each meteorological variable. The association between OHCA incidence of cardiac origin and mean ambient temperature was U-shaped, meaning that the incidence of OHCA was lowest at approximately 25°C and higher at temperatures above and below.
Exponentiated regression coefficients (ie, IRRs) of the GLM are shown in table 2. In univariable models, conventional ambient temperature, relative humidity, precipitation during the previous hour, snowfall, cloud coverage and wind speed were statistically associated with OHCA incidence (p<0.05, respectively). In the multivariable model, similar statistical associations were observed, except for mean precipitation during the previous hour, mean snowfall and mean wind speed for each day.
Incidence rate ratios (IRRs) obtained with a GLM based on the Poisson distribution for both datasets
Predictive accuracy of the models
Predicted and observed OHCA incidence of cardiac origin are plotted for each model in figure 1. Initially, we developed the predictive models based on comprehensive meteorological variables and chronological variables, respectively, using ML. The predicted values fitted the observed values well. ML predictive models with comprehensive meteorological variables were able to predict the daily change in OHCA incidence but were not able to predict a large increase in OHCA incidence accurately during the winter (figure 1A). ML predictive models with chronological variables were able to predict a large increase in OHCA incidence during the winter (figure 1B). By combining meteorological and chronological variables in a single ML predictive model, the concordance of the predicted values and the observed values improved (figure 1C). Predictive accuracy of the predictive models is shown in table 3. Among all predictive models, the predictive model with combined meteorological and chronological variables had the highest predictive accuracy in the training (MAE 1.314 and MAPE 7.007%) and testing datasets (MAE 1.547 and MAPE 7.788%). The predictive model with combined meteorological and chronological variables also had the highest correlations between observed and predicted values in the training (r=0.880, 95% CI 0.880 to 0.880) and testing datasets (r=0.870, 95% CI 0.860 to 0.870) (online supplemental figure 3).
Observed versus predicted incidence of OHCA. The blue dots indicate the observed total number of OHCAs per day in Japan. The red dots indicate the predicted number based on the following predictive models: (A) ML model with comprehensive meteorological variables, (B) ML model with chronological variables and (C) ML model with combined meteorological and chronological variables. ML, machine learning; OHCA, out-of-hospital cardiac arrest.
Accuracy of the predictive models for out-of-hospital cardiac arrest based on meteorological data, chronological data and combined meteorological and chronological data
Moreover, using the predictive model with combined meteorological and chronological variables, we predicted the incidence of OHCA at a district level in Kobe city during a 1-week period after 2016. Figure 2 shows the heatmap of observed vs predicted numbers of OHCA incidence between 28 January and 3 February 2018. During this week, 24 OCHA events were predicted for Kobe city, while 27 OCHA events were observed. The heatmap showed that zero to four OHCA events occurred in each district during this week. Among nine districts, the predicted OHCA incidence matched the observed OHCA incidence in four districts (districts A, B, E and G). One fewer OHCA event was predicted than observed in three districts (districts C, F and I).
Heatmap of observed and predicted OHCA incidence at the district level in Kobe city during a 1-week period. The blue graduation indicates the number of OHCA events, ranging from 1 to 4. The population for each district is provided. OHCA, out-of-hospital cardiac arrest.
Predictive importance
The predictive importance of meteorological and chronological variables in the ML predictive model is shown in figure 3. With regard to meteorological variables, lower mean ambient temperature within a day was the most strongly associated with the incidence of OHCA. In addition, larger difference in mean ambient temperature from the previous day and larger difference between maximum and minimum ambient temperatures within a day were also more strongly associated with the incidence of OHCA than other variables. Among chronological variables, more recent year, winter, Sunday, Monday and holiday were more strongly associated with the incidence of OHCA.
Importance of meteorological and chronological variables in a machine learning predictive model. This figure shows variable importance plots for meteorological (A) and chronological variables (B) in the machine learning predictive model using XGBoost. The yellow to purple dots in each row represent low to high values of the number of OHCA events corresponding to that meteorological or chronological variable. The x-axis shows the Shapley value, indicating the variable’s impact on the model. Positive SHAP values tend to drive predictions towards an OHCA event, and negative SHAP values tend to drive the prediction towards no OHCA events. *In the model, 2005 was considered year 0. OHCA,out-of-hospital cardiac arrest; SHAP, Shapley Additive Explanations; XGBoost, eXtreme Gradient Boosting.
Discussion
In this study, using the predictive model developed with combined meteorological and chronological variables, we succeeded in predicting OHCA incidence of cardiac origin with high precision. Our study is the first to predict daily OHCA incidence based on both meteorological and chronological variables using ML.
Previous studies that investigated meteorological variables associated with the incidence of cardiovascular events used ambient temperature alone or seasons8–10 24 and were limited to one city or region.8 9 Thus, these studies did not take diversity in geography and climate into account. Indeed, Japan is located in a temperate zone with four distinct seasons and its climate varies from cool temperate in the north to subtropical in the south. Latitude ranges from N45° to N20°. In this study, we used ML to process complex data that included a nationwide registry of OHCA events and a comprehensive meteorological dataset. If climate change becomes more intense, the relationship between OHCA incidence and comprehensive meteorological data may become all the more crucial.
Meaning of study
Our predictive model for the daily incidence of OHCA had high predictive accuracy. In particular, a larger difference in mean ambient temperature from the previous day and a larger range in ambient temperature within a day, in addition to mean ambient temperature lower or higher than 25°C within a day (a U-shaped distribution), were associated with OHCA incidence of cardiac origin. We speculated that a sudden change in ambient temperature on days with extreme cold or heat plays a key role in increasing the risk of OHCA of cardiac origin; this might be related to increased sympathetic tone and blood viscosity.25 26 However, in this study, we could not investigate how the location of OHCA affects the association between meteorological condition and the incidence of OHCA because detailed information about whether OHCA events occurred indoors or outdoors was not available. If this information is available in future research, one prevention method might be to advise individuals to stay home using an IoT device warning system on high-risk days. We found important chronological variables that affect the incidence of OHCA such as season, day of the week and holiday. Combining meteorological and chronological variables further improved the predictive accuracy of the ML predictive model. Importantly, at the local level, a heatmap showed that predicted OHCA incidence based on the ML predictive model fitted observed OHCA incidence well. Although our model was developed based on meteorological data with a resolution of 30 km gridded points, it could be applied even at a district level within one city. The model could be more practically useful if it could be further improved to predict OHCA incidence within a medical catchment area.
Implications
One advantage of using meteorological data to make predictions of OHCA incidence is that weather forecasts can predict meteorological conditions 2 weeks ahead. Our predictive model for daily incidence of OHCA is widely generalisable for the general population in developed countries because this study had a large sample size and used comprehensive meteorological data. Many developed countries are located in a similar latitude range as Japan. The methods developed in this study serve as an example of a new model for predictive analytics that could be applied to other clinical outcomes of interest related to life-threatening acute cardiovascular disease. It could also provide more opportunities to support self-management in high-risk individuals through IoT devices.27 28 Moreover, we expect to use our predictive model to provide warnings to EMS personnel, in addition to citizens, on high-risk days. As a result, it may lead to shorter transport time from onset to hospital arrival and rapid start of advanced resuscitation care after hospital arrival. Future research should prospectively evaluate the effectiveness of this approach and whether it translates into improved clinical outcomes.
Strengths and limitations
Our study has several strengths. First, the All-Japan Utstein Registry included all patients with OHCA who received prehospital resuscitation efforts by EMS personnel because they are not permitted to terminate resuscitation in the field. Moreover, uniform data collection, a large sample size and a population-based design covering all known OHCA events in Japan minimise potential sources of bias. These features contribute to the representativeness of the present predictive models.
This study has several inherent limitations. First, we did not have detailed information about where OHCA events occurred in various districts except in Kobe city; information was generally only available on the prefecture level. Second, our data did not address the potential variability in patients’ preexisting medical conditions. Third, the predictability of future OHCA events will depend on the accuracy of meteorological data. Finally, external testing in other developed countries was not performed.
Conclusion
An ML predictive model using combined multiple meteorological and chronological variables could predict OHCA incidence of cardiac origin with high precision. Furthermore, this predictive model may be useful for preventing OHCA and improving the prognosis of patients with OHCA via a warning system for citizens and EMS on high-risk days in the future.
Key messages
What is already known on this subject?
Previous studies have shown an association between lower ambient temperature and the incidence of cardiovascular events.
What might this study add?
This study using Japanese out-of-hospital cardiac arrest (OHCA) registry data combined with high-resolution meteorological and chronological data demonstrated that various meteorological and chronological variables are significantly associated with the incidence of OHCA of cardiac origin. A machine learning predictive model developed with comprehensive meteorological and chronological variables predicted the daily incidence of OHCA with high precision. Sunday, Monday, holiday, winter, low ambient temperature, and large interday or intraday temperature difference were strongly associated with OHCA incidence.
How might this impact on clinical practice?
This predictive model may be useful for preventing OHCA and improving the prognosis of patients with OHCA via a warning system for citizens and emergency medical services on high-risk days, thereby in the future.
Data availability statement
No data are available. The All-Japan Utstein Registry of the FDMA is a publicly accessible open database. The availability of the Kobe Municipal Fire Department database, which includes detailed information on the location of cardiac arrest used with permission for this study, is restricted. The Weather Company Data Packages from the Weather Company is subject to a full license agreement, which does not permit data sharing outside of the research team.
Ethics statements
Ethics approval
A subcommittee for resuscitation science in the Japanese Circulation Society was provided with registry data following prescribed governmental procedures. The ethics committee of the National Cerebral and Cardiovascular Centre approved the study (M30-055-2). In this registry study, the requirement of written informed consent was waived because the researchers only analysed deidentified (anonymised) data.
Acknowledgments
We would like to thank all the emergency medical services (EMS) personnel and staff members of the Fire and Disaster Management Agency (FDMA) and Institute for Fire Safety and Disaster Preparedness of Japan for their cooperation in establishing and maintaining the Utstein database.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors TN contributed to formulation of the study design, interpretation of the results and contributed to the final report. TN, YY, SY, KI and RN contributed to the final report. KN, SO, SK and DO contributed to data analysis. YT, SK, TI, KN, TS and HN contributed to formulation of the concept of the All-Japan Utstein Registry, data collection and data management. All authors approved the final version.
Funding This research was supported by the Environment Research and Technology Development Fund (1–1905) of the Environmental Restoration and Conservation Agency of Japan, a Grant-in-Aid for Young Scientists (A) (20K17914) of the Japan Society for the Promotion of Science, and the Intramural Research Fund of Cardiovascular Disease of the National Cerebral and Cardiovascular Centre (30-6-15). All authors had full access to all datasets. The OHCA dataset from the All-Japan Utstein Registry of the FDMA is a publicly accessible, open database. The meteorological dataset from the Weather Company, an IBM Business, was available by license agreement. The corresponding author had the ultimate responsibility for the decision to submit for publication.
Disclaimer The funders of the study, FDMA, Kobe Municipal Fire Department and IBM had no role in study design, data, collection, data analysis, data interpretation or writing of the report.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.