Article Text
PDF
Abstract
Objective To evaluate the performance of the non-laboratory INTERHEART risk score (NL-IHRS) to predict incident cardiovascular disease (CVD) across seven major geographic regions of the world. The secondary objective was to evaluate the performance of the fasting cholesterol-based IHRS (FC-IHRS).
Methods Using measures of discrimination and calibration, we tested the performance of the NL-IHRS (n=100 475) and FC-IHRS (n=107 863) for predicting incident CVD in a community-based, prospective study across seven geographic regions: South Asia, China, Southeast Asia, Middle East, Europe/North America, South America and Africa. CVD was defined as the composite of cardiovascular death, myocardial infarction, stroke, heart failure or coronary revascularisation.
Results Mean age of the study population was 50.53 (SD 9.79) years and mean follow-up was 4.89 (SD 2.24) years. The NL-IHRS had moderate to good discrimination for incident CVD across geographic regions (concordance statistic (C-statistic) ranging from 0.64 to 0.74), although recalibration was necessary in all regions, which improved its performance in the overall cohort (increase in C-statistic from 0.69 to 0.72, p<0.001). Regional recalibration was also necessary for the FC-IHRS, which also improved its overall discrimination (increase in C-statistic from 0.71 to 0.74, p<0.001). In 85 078 participants with complete data for both scores, discrimination was only modestly better with the FC-IHRS compared with the NL-IHRS (0.74 vs 0.73, p<0.001).
Conclusions External validations of the NL-IHRS and FC-IHRS suggest that regionally recalibrated versions of both can be useful for estimating CVD risk across a diverse range of community-based populations. CVD prediction using a non-laboratory score can provide similar accuracy to laboratory-based methods.
- cardiovascular disease
- risk prediction
- INTERHEART risk score
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Introduction
Approximately 80% of cardiovascular disease (CVD) related deaths occur in middle-income or low-income countries (MICs or LICs) where health resources are limited.1 In developing regions of the world, the World Health Organization (WHO) recommends incorporating overall CVD risk in the decision to initiate primary prevention treatments, with specific treatment thresholds based on the resources that are available.2 A major barrier to this approach is that in most regions of the world, there are few validated risk prediction tools, and those validated in regions outside of North America and Europe (eg, Framingham and Globorisk) mostly rely on laboratory measures (eg, lipid values), limiting their use in communities where laboratory facilities are not readily available.3–6
Using the multinational INTERHEART case–control study, four risk prediction tools have been developed and validated to predict the risk of myocardial infarction (MI) and coronary artery disease (CAD) development. Of these, the non-laboratory INTERHEART risk score (NL-IHRS) was shown to predict CAD risk based solely on clinical history and simple physical measurements, making it well suited for use across a wide range of geographic regions and resource settings.7 However, since the score was developed in a case–control study of MI, to better evaluate its applicability, it is necessary to examine the tool’s performance across a broader range of major CVD outcomes (eg, stroke, fatal CVD and heart failure) in a prospective cohort study. The Prospective Urban Rural Epidemiology (PURE) study is a large, community-based cohort study that has collected data on CVD risk factors and outcomes across a wide range of resource settings in 17 countries. The primary objective of this study was to evaluate the performance of the NL-IHRS as a CVD risk prediction tool in PURE across seven distinct geographic regions. We also examined the performance of the laboratory-based, fasting cholesterol IHRS (FC-IHRS) in the PURE study.
Methods
Study population
The NL-IHRS and FC-IHRS were previously developed using the INTERHEART case–control study, which examined risk factors for incident MI in 27 043 participants (14 605 cases and 12 438 controls) across 52 countries (online supplementary figures 1 and 2).7 The methodology of the PURE study has been previously described.8 PURE enrolled community-dwelling participants between 35 and 70 years of age across 628 communities from 17 countries.8Follow-up occurred between 2008 and 2016. Cardiovascular events were obtained from participants or their family members and reviewed centrally within each country with available supporting documentation (eg, verbal autopsy and medical records) using standardised definitions. Demographics of the study population have been shown to be broadly consistent with national data.9 Variables from PURE were chosen to reflect those from the original NL-IHRS as closely as possible (table 1 and online supplementary table 1). To examine the performance of the NL-IHRS across distinct populations, countries were grouped into seven distinct regions: Europe/North America (Canada, Poland, Sweden and Turkey), South America (Argentina, Brazil, Chile and Columbia), South Asia (India, Pakistan and Bangladesh), Middle East (Iran and United Arab Emirates), China, Southeast Asia (Malaysia) and Africa (South Africa and Zimbabwe). The primary outcome was a composite of major cardiovascular events, defined as: death from a cardiovascular cause, MI, stroke, heart failure or revascularisation (by either percutaneous coronary intervention or coronary artery bypass surgery). Ethics approval for PURE was obtained through local ethics boards, and all participants provided informed consent.
Statistical analysis
Assessment of individual predictor variables in the NL-IHRS
Analyses were conducted in participants without a history of CVD and with complete data. Associations between individual predictor variables and incident CVD were calculated using a multivariable logistic regression model comprised of the NL-IHRS predictor variables in addition to geographic region. Results are presented as regression coefficients (β-coefficient) and ORs with 95% CIs, with a two-sided p value of <0.05 considered statistically significant.
Discrimination and calibration of the NL-IHRS
Performance of the NL-IHRS was first evaluated using the original multivariable logistic regression model developed in INTERHEART.7 External validation was performed using the performance measures outlined by Debray et al 10 testing model discrimination and calibration. Discrimination reflects the ability of the logistic regression model to differentiate participants who had a cardiovascular event from those who did not and was measured using the concordance statistic (C-statistic).11 We examined the discrimination of the score in the overall study population and within individual geographic regions. Performance of the score was also examined according to country income status based on World Bank classifications at the initiation of PURE. To determine the impact of missing data, sensitivity analyses were performed following imputation of missing variables using the multiple imputation by chained equation method as described by Vergouwe et al 12 (see supplementary methods for further details).
Calibration measures how closely a model’s estimates of predicted risk agree with observed outcomes. Model calibration was assessed by fitting a simple logistic regression model of the outcome with the prognostic index (PI) as the only covariable.13 14 The PI was the weighted sum of the variables in the model where weights were the original regression coefficients of the INTERHEART model.
PI=−1.45 + 0.25*male ≥55 years or female ≥65 years of age + 0.55*parental history of CAD + 0.93*diabetes + 0.72*hypertension + 0.22*former smoker + 0.30*1–5 cigarettes/day + 0.53*6–10 cigarettes/day + 0.86*11–15 cigarettes/day + 0.98*16–20 cigarettes/day + 1.45*>20 cigarettes/day + 0.28*second-hand smoke exposure + 0.27*low physical activity + 0.39*depression + 0.37*general stress + 0.10*WHR second quartile + 0.17*WHR third quartile + 0.54*WHR upper quartile + 0.12*salty food consumption≥1/day + 0.16*fruit consumption <1/day + 0.21*vegetable consumption <1/day + 0.19*fried food/trans saturated fat consumption≥3/week + 0.23*red meat/poultry consumption≥2/day [1]
For each calibration measure, 95% CIs were calculated, and deviations of the intercept (α) from 0 (ie, a systematic overestimatoin or underestimation of risk) or the slope (β) from one indicated miscalibration.10 In the presence of miscalibration, recalibration was performed within each region by updating the PI using the following equation:
[2]
where α and β were the respective point estimates of the intercept and slope of the recalibrated NL-IHRS measured within each region in PURE (summarised in table 2).
Estimation of CVD risk for each region using the NL-IHRS
To allow for simple estimation of CVD risk using a single point system, the PI was first calculated based on the methods outlined by Sullivan et al, which was used as the basis for deriving the original point system for the NL-IHRS (equation 3).7 14 This allows for estimates the PI for a given point total using the following equation:
[3]
Recalibrated estimates of CVD risk were then calculated for each region using logistic calibration according to the following equation:
[4]
where 
and 
were the respective point estimates of the intercept and slope for the original NL-IHRS measured within each region (which are summarised in table 2).7 13–15
External validation of the FC-IHRS
Validation of the FC-IHRS used the same process as outlined for the NL-IHRS. Using recalibrated models, we calculated region-specific annual predicted risks of CVD for each ‘point’ across the range of the score.14 SAS V.9.4 and STATA V.14.2 were used for the statistical analyses.
Results
Number of participants, incident cardiovascular events and demographics
Overall, 142 531 participants without a history of CVD had follow-up data available, of whom 100 475 (70%) had complete data for all NL-IHRS variables (mean follow-up of 4.89 (SD 2.24 years) and 107 863 (76%) had complete data for the FC-IHRS. Characteristics of the participants on which the performance of the NL-IHRS (primary objective) was tested, and CVD events during follow-up are summarised in online supplementary table 2. The largest number of participants were from China (38 431 (38.2%)) and the smallest number of participants were from Africa (1291 (1.3%)). The mean age of the study population was 50.53 (SD 9.79) years, and 58 995 (58.7%) participants were women. The prevalence of individual risk factors varied considerably by region. During follow-up, 2284 (2.3%) participants developed CVD (event rate=0.46 %/year), with MI in 992 (1.0%) and stroke in 915 (0.9%). The largest number of CVD events occurred in China (n=568), and the fewest in Africa (n=50). Of the 107 863 participants in which the FC-IHRS was studied, 2681 (2.5%) had a major CVD over a mean follow-up of 5.33 (SD 2.63) years (event rate=0.47 %/year).
Associations between individual clinical predictors and incident cardiovascular events
Of the 15 predictors that comprised the NL-IHRS, 11 were significantly associated with increased odds of CVD (table 1). Associations between dietary predictors and CVD risk varied, with high meat consumption and low fruit consumption both associated with a higher odds of CVD; low vegetable consumption and consumption of salty foods ≥1 time per day both associated with a lower odds of CVD; and no significant effect observed with the consumption of ≥3 servings of fried foods/foods containing trans unsaturated fat per week.
Associations with incident cardiovascular disease for each predictor variable included in the non-laboratory INTERHEART risk score
Discrimination of the NL-IHRS
The discrimination of the NL-IHRS for CVD is summarised in table 2. C-statistic=0.69 (0.68, 0.70) in the overall cohort, performing best in the Middle East (C-statistic=0.74 (0.70, 0.78)) and lowest in Africa (C-statistic=0.64 (0.56, 0.72])). The score showed similar discrimination for MI (C-statistic=0.69 (95% CI 0.67 to 0.71)) and stroke (C-statistic=0.67 (95% CI 0.66 to 0.69)). Discrimination was slightly higher in MICs compared with HICs and LICs (table 2). In our sensitivity analysis, which included all 142 531 participants with multiple imputation used to account for missing data, discrimination was similar to that observed in the primary analysis for the overall cohort and regionally (online supplementary table 3). In an exploratory analysis, removing the discordant or non-significant dietary variables (eg, salty foods, vegetables and fried foods) only resulted in a marginal increase in C-statistic (0.72 to 0.73)
Discrimination and calibration of the non-laboratory INTERHEART Risk Score for incident cardiovascular disease
Calibration of the NL-IHRS
Intercept and calibration slopes suggested that in most regions, the NL-IHRS systematically overestimated CVD risk (table 2). Calibration plots (online supplementary figure 3) also suggested that recalibration was necessary in all regions. Simple recalibration of the model slope and intercept was performed in each region (see online supplementary figure 3) and significantly improved discrimination in the overall cohort (C-statistic from 0.69 to 0.72, p<0.001). Annualised estimates of CVD risk following recalibration are provided in online supplementary table 4.
External validation of the FC-IHRS
Measures of discrimination and calibration for the FC-IHRS are summarised in table 3. The score demonstrated good discrimination for CVD in the overall cohort (C-statistic=0.71 (95% CI 0.70 to 0.72)) and moderate to good discrimination in all regions (C-statistic ranging between 0.66 and 0.78). Calibration tests suggested a systematic overestimation of risk in all regions, and therefore regional recalibration was applied, which improved the overall discrimination of the score (C-statistic from 0.71 to 0.74, p<0.001). Annualised estimates of CVD risk following recalibration are provided in online supplementary table 5. For both scores, actual and predicted CVD risk estimates per each tertile of risk (overall and in each region) are provided in online supplementary tables 6a and 6b.
Discrimination and calibration of the fasting cholesterol INTERHEART risk score for incident cardiovascular disease
Performance of the FC-IHRS compared with the NL-IHRS
We compared discrimination of the recalibrated NL-IHRS and FC-IHRS in 85 078 participants with complete data for both scores (ie, complete data on all clinical variables and fasting lipids). The FC-IHRS performed statistically better than the NL-IHRS, although the absolute difference was very small (C-statistic=0.74 (95% CI 0.73 to 0.75) vs 0.73 (95% CI 0.72 to 0.74), p value <0.001). Similar results were observed in most geographic regions, except in North America/Europe (table 4). Cohen’s kappa statistic showed moderate agreement between scores (0.52). Using predicted CVD risk thresholds of 1 %/year and 2 %/year (to identify low-risk, intermediate-risk and high-risk groups), reclassification with the FC-IHRS was also relatively small when compared with the NL-IHRS (Net Reclassification Index (NRI)=0.075 (95% CI 0.06 to 0.10)), event classification=0.095 (95% CI 0.075 to 0.115) and non-event classification=−0.019 (95% CI –0.021 to –0.018)).15 Using a single threshold of 0.46 %/year (representing the incidence of major CVD in the population cohort) NRI was modest (0.02 (95% CI 0.004 to 0.04)).
Comparison of the non-laboratory and fasting cholesterol INTERHEART risk scores for predicting cardiovascular disease
Discussion
This study is the first to evaluate the performance of the NL-IHRS and FC-IHRS to predict incident CVD in a large, multinational, community-based prospective cohort study, providing two methods to estimate CVD risk across seven distinct geographic regions. Our results can be applied to assist CVD risk stratification and prevention strategies in similar populations.
CVD risk prediction tools perform best when used in populations similar to the cohorts from which they were derived. For example, Wu et al have developed a laboratory-based score for use in China based on separate large, community-based population cohorts, but similar well-validated tools have not been developed in most regions of the world outside of North America or Europe.16 Alternatively, externally developed risk prediction tools (eg, Framingham and Globorisk) can adequately estimate CVD risk (usually following recalibration) in regions where locally developed tools are lacking.3 5 17 While some non-laboratory risk scores have been developed in North American cohorts (eg, Framingham and National Health and Nutrition Examination Survey), they have undergone limited external validation in other regions of the world.4 18 19 The WHO/International Society of Hypertension has developed both laboratory and non-laboratory based risk charts to estimate CVD risk in MICs and LICs, but risk predictors and outcomes were modelled using different population cohorts, and this approach has not been adequately externally validated.2 3
This study extends on current global applications of CVD risk assessment tools by providing a non-laboratory (and an alternative laboratory based) method to predict CVD events in several regions of the world where prediction tools are largely lacking. Following recalibration, discrimination of the NL-IHRS was moderate to good in the overall study cohort, and within each region. The score performed equally well for predicting individual CVD outcomes (eg, MI and stroke) as it did for overall CVD events, suggesting that although the score was derived as a prediction tool for MI, it is consistent across a range of clinically important CVD outcomes. Of the individual predictors, age, hypertension, smoking, diabetes and abdominal obesity conferred the greatest risks. Less consistent associations were observed with the score’s dietary predictors. This may have been partly due to differences in how dietary measures were collected between studies. Also, it is possible that variations in food preservation, cooking or processing methods that were not directly captured in the score, but are known to vary considerably across the world, accounted for some of our dietary associations. For example, although we observed that self-reported consumption of salty foods >1 time per day was associated with a lower CVD risk, in further exploratory analyses we also found this question correlated poorly with more accurate estimates of 24 hours sodium excretion based on fasting urine samples from PURE (correlation coefficient=−0.10).20 This discrepancy may be due the contribution of non-discretionary sodium consumption, which is captured poorly through self-reported methods. In the context of a risk score, more consistent effects might be achieved with dietary factors that are adjusted to reflect average consumption patterns within a given region. Despite the inconsistencies observed with three dietary predictors (eg, salty food, fried food and vegetable consumption), their impact on the overall performance of the NL-IHRS was very modest (ie, change in C-statistic of 0.01). Finally, both high red meat and low fruit consumption continued to be independent predictors of major CVD risk, highlighting the added value of these dietary variables as part of the NL-IHRS.
The recalibrated FC-IHRS also had moderate to good discrimination for incident CVD in the overall study cohort and across geographic regions. The performance of the FC-IHRS was comparable with studies of other lab-based CVD risk stratification tools when validated in populations outside of Europe and North America, and similarly we needed to recalibrate the score to accurately estimate CVD risk in each region.5 15 21 Although the FC-IHRS predicted incident CVD better than the NL-IHRS, differences in accuracy were relatively small, suggesting that the NL-IHRS could be used as an alternative method to risk stratify individuals.22 23 One potential concern with the NL-IHRS is that with a greater number of predictors compared with other scores, including waist-to-hip ratio (WHR), calculating risk is more complex than other risk stratification tools. Above its predictive performance, choosing a particular risk score for clinical use will depend on whether it has distinct advantages in a particular health resource setting. The NL-IHRS performed best in MICs, followed by HICs and lowest in LICs, although absolute differences in discrimination were relatively small (eg, C-statistic ranging between 0.66 and 0.72), suggesting that the NL-IHRS can be used with at least moderate discrimination for CVD across a variety of resource settings. We foresee two scenarios where the NL-IHRS can potentially improve on current methods of risk stratification. The first is in communities with limited access to laboratory tests. Second, because the NL-IHRS only requires a tape measure (to measure WHR), it can be easily incorporated into ‘community-based’ CVD screening programmes, which are currently being implemented in several LICs and MICs.24 For example, in the Rural Andhra Pradesh Cardiovascular Prevention Study, a community-based, door-to-door, non-physician health worker-based CVD screening programme resulted in a 12% increase in the detection of high-risk individuals with prior CVD.24 The use of a non-laboratory-based CVD score in this setting could also provide a method of rapidly assessing CVD risk in asymptomatic individuals to guide futher management.
Some limitations of our study warrant consideration. We observed a relatively low incidence of CVD events in PURE, which is expected in a community-based cohort. Although most regions had >100 events to evaluate the performance of the NL-IHRS, the number of events was lower in Africa (online supplementary table 2). This could have reduced our precision to evaluate the score’s performance and partly accounted for the lower discrimination observed in this region when compared with others. Despite this, the ability to provide some estimation of risk in this population is an advance over currently employed methods, and with ongoing expansion of the current cohort and further follow-up, more data will accrue, allowing for future updates of the prediction tool. Furthermore, 30% of participants were excluded in our primary analysis of the NL-IHRS due to the presence of at least one missing variable, which is a common challenge in risk score development and validation studies.12 However, in our sensitivity analysis using imputed data, discrimination of the NL-IHRS was similar in the overall study cohort and within most regions.
Conclusions
The regionally recalibrated NL-IHRS and FC-IHRS can be used to estimate CVD risk across a wide range of communities in different regions of the world. In an international setting, non-laboratory-based tools have similar predictive ability when compared with laboratory-based tools.
Key messages
What is already known on this subject?
Although several risk scores are available to estimate cardiovascular disease (CVD) risk, most are not well validated outside of North America or Europe, and only few can estimate risk without the use of laboratory measurements. This significantly limits their utility in many regions of the world.
What might this study add?
This study examines the performance of a non-laboratory-based and laboratory-based CVD risk stratification tool, and provides calibrated versions for use in seven distinct regions of the world.
How might this impact on clinical practice?
Using these tools, CVD risk can be estimated in several regions where validated measures were lacking, and in settings where limited access to a laboratory hinders the ability to estimate risk.
Acknowledgments
We wish to acknowledge the additional persons provided in the supplementary appendix who have contributed to the PURE study.
References
Footnotes
Contributors PJ, SY, SFL, QI, KT and SSA contributed to the conception and design of the paper; analysis and interpretation of the data; and drafting the article. SR, RG, AR, SL, AA, PL-J, SG, AY, JC, FL, RK, NM, VM, PM, AnK, XL, BG, WZ, YY, RP, RD, AmK, RI, RY and SSA contributed to the acquisition of data and critical revision of the article for intellection content. PJ, SFL and QI are responsible for the overall content of the article and data analysis.
Funding A full description of funding is provided in the supplementary appendix.
Competing interests None declared.
Ethics approval Hamilton Integrated Research Ethics Board, McMaster University.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Unpublished data related to this study are not available for public use as consent for this was not obtained by participants at the time of study enrolment.