In their recent paper[1], the authors describe the performance of
ASSIGN and Framingham algorithms in comparison to the original QRISK
equations.
Readers may be interested that the QRISK2 algorithm was published in
February 2009[2] and made available as free open source software in April
20103. This can be found at http://svn.clinrisk.co.uk/opensource/qrisk2/.
The open source is intended to further increase the reliable and
widespread implementation of QRISK2 into clinical practice.
There are substantial differences between the original QRISK[4] and
QRISK2[5] algorithms which includes additional predictor variables
together with their associated significant age interactions:
- Self assigned ethnicity
- rheumatoid arthritis
- chronic renal disease
- atrial fibrillation.
All of these are independent predictors and improve risk estimates in
individual patients. Both QRISK and QRISK2 have been independently
validated on an external cohort[6,7]. The results for QRISK2 showed an
improvement compared with the original QRISK equation[4,5].
We disagree with the conclusion that "using any of the models for
initial systematic assessment of high or lower CVD risk would result in
the majority of men and women to which the models apply getting very
similar assessment and hence prioritisation for further investigation of
treatment". The key issue is the extent of reallocation. Allocation is
critically dependent on the CVD risk score used and its performance in
contemporaneous, ethnically diverse UK populations. The fact that ASSIGN,
like Framingham, is associated with 20% or more overestimation in men
results from a dependence on historical cohorts from the 1980's when the
vascular epidemic was near its height. Vascular mortality has halved in
succeeding decades and the incorrect allocation of individuals to high
risk categories will increase using ASSIGN and Framingham.
The QRISK2 algorithm is derived from contemporaneous cohorts and is
updated annually to take account of population trends in risk factors and
disease incidence, improvements in data quality and changing requirements
(eg need to incorporate a broader age range as in the GP "QOF" Contract).
QRISK2(2010) has therefore has been refitted to the latest version of the
QResearch database and includes a broader age range of patients aged 30-84
years3. This has resulted in considerable improvements in performance as
can be seen from the table below.
Systematic use of a cardiovascular risk score which doesn't include
ethnicity is likely to under-estimate risk, particularly in South Asians
and also to contribute to widening health inequalities.
The inclusion of ethnicity is especially important given the effect
of ethnicity on cardiovascular risk. For example, Pakistani men have a 97%
increased risk of CVD compared with white men (adjusted HR 1.97, 95% CI
1.70 to 2.29). Using a 20% threshold to define high risk, then 15% of
South Asian men would be identified as high risk using QRISK2 (2010)
compared with 10% using the NICE modified version of Framingham.
Similarly, 8% of South Asian women would be identified as high risk using
QRISK2(2010) compared with 3% based on Framingham.
Table 1: Validation statistics for QRISK2 (2010) on the QResearch
database compared with ASSIGN1 on the THIN database. The table shows
measures of the performance of the scores i.e. how accurate the scores are
at identifying high risk patients and distinguishing them from low risk
patients and how much of the 'variation' in risk is explained by the
scores themselves. High values for R2, D statistic and ROC indicate better
performance than low values. A predicted/observed ratio of 1 indicates
perfect calibration and a ratio greater than one indicates over-
prediction.
QRISK2 (2010) ASSIGN
Mean (95% CI) Mean (95% CI)
women
R2 51.4(50.9-5.19) 37.39 (36.70-37.97)
D statistic 2.11(2.09-2.13) 1.58 (1.56-1.60)
ROC value 0.853(0.851- 0.855) 0.792
Predicted
/observed 0.97 1.20
men
QRISK2 (2010) ASSIGN
Mean (95% CI) Mean (95% CI)
R2 45.9(45.4-46.4) 30.47(29.82-31.16)
D statistic 1.89 (1.87-1.91) 1.35 (1.33-1.37)
ROC value 0.830(0.828-0.833) 0.756
Predicted
/observed 0.95 1.20
References
1. de la Iglesia B, Potter JF, Poulter NR, Robins MM, Skinner J.
Performance of the ASSIGN cardiovascular disease risk score on a UK cohort
of patients from general practice. Heart 2010.
2. Hippisley-Cox J. Publication of the QRISK2 algorithm and release
of software for academics. BMJ. London: BMJ, 2009.
3. Hippisley-Cox J. 2010 update for QRISK2 and release of open source
software. BMJ. London: BMJ, 2010.
4. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M,
Brindle P. Derivation and validation of QRISK, a new cardiovascular
disease risk score for the United Kingdom: prospective open cohort study.
BMJ 2007:bmj.39261.471806.55.
5. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R,
Sheikh A, et al. Predicting cardiovascular risk in England and Wales:
prospective derivation and validation of QRISK2. BMJ
2008:bmj.39609.449676.25.
6. Collins GS, Altman DG. An independent and external validation of
QRISK2 cardiovascular disease risk score: a prospective open cohort study.
BMJ 2010;340(may13_2):c2442-.
7. Collins GS, Altman DG. An independent external validation and
evaluation of QRISK cardiovascular risk prediction: a prospective open
cohort study. BMJ 2009;339(jul07_2):b2584-.
Conflict of Interest:
JHC is professor of clinical epidemiology at the University of Nottingham and co-director of QResearch - a not-for-profit organisation which is a joint partnership between the University of Nottingham and EMIS (leading commercial supplier of IT for 60% of general practices in the UK). JHC is also director of ClinRisk Ltd which produces open and closed source software to ensure the reliable and updatable implementation of clinical risk algorithms within clinical computer systems to help improve patient care. CC is associate professor of Medical Statistics at the University of Nottingham and a consultant statistician for ClinRisk Ltd. JR and PB were previously members of the NICE Guideline Development Group for Lipid Modification of which JR was chair.
In their recent paper[1], the authors describe the performance of ASSIGN and Framingham algorithms in comparison to the original QRISK equations.
Readers may be interested that the QRISK2 algorithm was published in February 2009[2] and made available as free open source software in April 20103. This can be found at http://svn.clinrisk.co.uk/opensource/qrisk2/. The open source is intended to further increase the reliable and widespread implementation of QRISK2 into clinical practice.
There are substantial differences between the original QRISK[4] and QRISK2[5] algorithms which includes additional predictor variables together with their associated significant age interactions:
- Self assigned ethnicity - rheumatoid arthritis - chronic renal disease - atrial fibrillation.
All of these are independent predictors and improve risk estimates in individual patients. Both QRISK and QRISK2 have been independently validated on an external cohort[6,7]. The results for QRISK2 showed an improvement compared with the original QRISK equation[4,5].
We disagree with the conclusion that "using any of the models for initial systematic assessment of high or lower CVD risk would result in the majority of men and women to which the models apply getting very similar assessment and hence prioritisation for further investigation of treatment". The key issue is the extent of reallocation. Allocation is critically dependent on the CVD risk score used and its performance in contemporaneous, ethnically diverse UK populations. The fact that ASSIGN, like Framingham, is associated with 20% or more overestimation in men results from a dependence on historical cohorts from the 1980's when the vascular epidemic was near its height. Vascular mortality has halved in succeeding decades and the incorrect allocation of individuals to high risk categories will increase using ASSIGN and Framingham.
The QRISK2 algorithm is derived from contemporaneous cohorts and is updated annually to take account of population trends in risk factors and disease incidence, improvements in data quality and changing requirements (eg need to incorporate a broader age range as in the GP "QOF" Contract). QRISK2(2010) has therefore has been refitted to the latest version of the QResearch database and includes a broader age range of patients aged 30-84 years3. This has resulted in considerable improvements in performance as can be seen from the table below.
Systematic use of a cardiovascular risk score which doesn't include ethnicity is likely to under-estimate risk, particularly in South Asians and also to contribute to widening health inequalities.
The inclusion of ethnicity is especially important given the effect of ethnicity on cardiovascular risk. For example, Pakistani men have a 97% increased risk of CVD compared with white men (adjusted HR 1.97, 95% CI 1.70 to 2.29). Using a 20% threshold to define high risk, then 15% of South Asian men would be identified as high risk using QRISK2 (2010) compared with 10% using the NICE modified version of Framingham. Similarly, 8% of South Asian women would be identified as high risk using QRISK2(2010) compared with 3% based on Framingham.
Table 1: Validation statistics for QRISK2 (2010) on the QResearch database compared with ASSIGN1 on the THIN database. The table shows measures of the performance of the scores i.e. how accurate the scores are at identifying high risk patients and distinguishing them from low risk patients and how much of the 'variation' in risk is explained by the scores themselves. High values for R2, D statistic and ROC indicate better performance than low values. A predicted/observed ratio of 1 indicates perfect calibration and a ratio greater than one indicates over- prediction.
QRISK2 (2010) ASSIGN
Mean (95% CI) Mean (95% CI)
women R2 51.4(50.9-5.19) 37.39 (36.70-37.97)
D statistic 2.11(2.09-2.13) 1.58 (1.56-1.60)
ROC value 0.853(0.851- 0.855) 0.792
Predicted /observed 0.97 1.20
men
QRISK2 (2010) ASSIGN
Mean (95% CI) Mean (95% CI)
R2 45.9(45.4-46.4) 30.47(29.82-31.16)
D statistic 1.89 (1.87-1.91) 1.35 (1.33-1.37)
ROC value 0.830(0.828-0.833) 0.756
Predicted /observed 0.95 1.20
References
1. de la Iglesia B, Potter JF, Poulter NR, Robins MM, Skinner J. Performance of the ASSIGN cardiovascular disease risk score on a UK cohort of patients from general practice. Heart 2010.
2. Hippisley-Cox J. Publication of the QRISK2 algorithm and release of software for academics. BMJ. London: BMJ, 2009.
3. Hippisley-Cox J. 2010 update for QRISK2 and release of open source software. BMJ. London: BMJ, 2010.
4. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ 2007:bmj.39261.471806.55.
5. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R, Sheikh A, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008:bmj.39609.449676.25.
6. Collins GS, Altman DG. An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ 2010;340(may13_2):c2442-.
7. Collins GS, Altman DG. An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study. BMJ 2009;339(jul07_2):b2584-.
Conflict of Interest:
JHC is professor of clinical epidemiology at the University of Nottingham and co-director of QResearch - a not-for-profit organisation which is a joint partnership between the University of Nottingham and EMIS (leading commercial supplier of IT for 60% of general practices in the UK). JHC is also director of ClinRisk Ltd which produces open and closed source software to ensure the reliable and updatable implementation of clinical risk algorithms within clinical computer systems to help improve patient care. CC is associate professor of Medical Statistics at the University of Nottingham and a consultant statistician for ClinRisk Ltd. JR and PB were previously members of the NICE Guideline Development Group for Lipid Modification of which JR was chair.