Article Text


Original article
Prospective validation and assessment of cardiovascular and offspring risk models for pregnant women with congenital heart disease
  1. Ali Balci1,2,
  2. Krystyna M Sollie-Szarynska3,
  3. Antoinette G L van der Bijl2,
  4. Titia P E Ruys4,
  5. Barbara J M Mulder5,
  6. Jolien W Roos-Hesselink4,
  7. Arie P J van Dijk6,
  8. Elly M C J Wajon7,
  9. Hubert W Vliegen8,
  10. Willem Drenthen2,
  11. Hans L Hillege2,9,
  12. Jan G Aarnoudse4,
  13. Dirk J van Veldhuisen2,
  14. Petronella G Pieper2
  15. On behalf of the ZAHARA-II investigators
  1. 1Department of Cardiology, Isala, Zwolle, The Netherlands
  2. 2Department of Cardiology, University Medical Centre Groningen, University of Groningen, Groningen, The Netherlands
  3. 3Department of Obstetrics, University Medical Centre Groningen, University of Groningen, Groningen, The Netherlands
  4. 4Department of Cardiology, Erasmus Medical Centre, Erasmus University, Rotterdam, The Netherlands
  5. 5Department of Cardiology, Academic Medical Centre, University of Amsterdam, Amsterdam, The Netherlands
  6. 6Department of Cardiology, Radboud University Nijmegen Medical Centre, Radboud University Nijmegen, Nijmegen, The Netherlands
  7. 7Department of Cardiology, Medical Spectrum Twente, Enschede, The Netherlands
  8. 8Department of Cardiology, Leiden University Medical Centre, University of Leiden, Leiden, The Netherlands
  9. 9Department of Epidemiology, University Medical Centre Groningen, University of Groningen, Groningen, The Netherlands
  1. Correspondence to Dr. P.G. Pieper, Department of Cardiology, University Medical Centre Groningen, University of Groningen, Groningen 9700 RB, The Netherlands E-mail: p.g.pieper{at} or Dr Ali Balci, Department of Cardiology, Isala, PO Box 10.400, Zwolle, 8000 GK, The Netherlands


Objectives Adequate prepregnancy prediction of maternal cardiovascular and offspring risk is important for counselling and management of pregnancy in women with congenital heart disease (CHD). Therefore we performed a study to identify the optimal assessment strategy for estimating the risk of pregnancy in women with CHD.

Methods In this prospective study, we determined the outcomes of 213 pregnancies in 203 women with CHD. The ZAHARA I (Zwangerschap bij Aangeboren HARtAfwijkingen I) and CARPREG (CARdiac disease in PREGnancy) risk scores were calculated for each pregnancy, as was the total number of cardiovascular (TPc) or offspring risk predictors (TPo) from these and other studies combined. Pregnancies were also classified according to the modified WHO classification of maternal cardiovascular risk and according to disease complexity (DC).

Results Maternal cardiovascular events occurred during 22 pregnancies (10.3%). Offspring events occurred during 77 pregnancies in 81 children (37.3%). Cardiovascular and offspring event rates increased with higher risk scores, higher TPc or TPo, higher WHO class and greater DC. The highest area under the curve (AUC) for maternal cardiovascular risk was achieved by the WHO class (AUC: 0.77, p<0.0001). AUC for the ZAHARA I risk score was 0.71 (p=0.001), and for the CARPREG risk score 0.57 (p=0.32). All models performed insufficiently in predicting offspring events (AUC≤0.6).

Conclusions The WHO classification is the best available risk assessment model for estimating cardiovascular risk in pregnant women with CHD. None of the offspring prediction models perform adequately in our cohort.

  • Congenital Heart Disease

Statistics from


Pregnancy in women with structural congenital heart disease (CHD) is associated with increased maternal cardiac and offspring risk. Mothers with CHD are mainly at risk of developing arrhythmias and episodes of heart failure, whereas the offspring is mainly at risk of premature birth, small for gestational age and mortality.1–8 The magnitude of cardiac and offspring risk depends on the underlying CHD, and is attributable to the complexity of the heart disease and (residual) lesions such as valvular and ventricular dysfunction.1 ,2 ,7 For the attending cardiologist, adequate risk assessment is essential to optimise prepregnancy counselling and pregnancy management.

Several classifications and risk scores are available to estimate the maternal cardiac and offspring risk associated with pregnancy in women with CHD.2 ,5 ,7 ,9 ,10 Risk assessment models developed by the CARPREG (CARdiac disease in PREGnancy) investigators and by our own ZAHARA (Zwangerschap bij Aangeboren HARtAfwijkingen, pregnancy in CHD) research group provide quantification of maternal cardiovascular and offspring risk of pregnancy.2 ,7 Both identified independent predictors of maternal cardiovascular and offspring events, as described elsewhere in detail.2 ,7 Both models attribute points to each predictor of maternal cardiovascular risk, thus attributing a certain cardiovascular and offspring risk to the pregnancy. Additional predictors were identified by Khairy et al5 The European Society of Cardiology guidelines for the management of heart disease in pregnancy advise to estimate maternal risk according to the modified WHO classification.9 ,11 This classification integrates knowledge from the total body of literature and takes into account the underlying heart disease, ventricular and valvular function, as well as predictors identified by several studies. Patients are classified as low, moderate or high risk, or contraindication for pregnancy.9 ,11 Because risk of pregnancy is associated with disease complexity (DC), risk assessment may also be performed using a generally accepted DC classification.1 ,10 ,12 A prospective external validation and comparison of the abovementioned risk scores and risk assessment models has not been performed.

We therefore aimed in this prospective multicenter study to provide external validation of the CARPREG and ZAHARA I risk scores, and to compare the different risk assessment models in order to identify the optimal assessment strategy for estimating the risk of cardiovascular and offspring events of pregnancy in women with CHD.

Patients and methods

Design and setting

This prospective observational multicenter cohort study was conducted between March 2008 and August 2011. The extensive study design of the ZAHARA II study was published previously and is summarised below.13

Patient selection

All consecutive pregnant women with structural CHD (≥18 years) reporting pregnancy with a duration ≤20 weeks, who provided written informed consent, and who were followed in one of the eight participating hospitals, were eligible for enrolment. Miscarriages or termination before 20 weeks gestation were excluded, as were women with known illicit drug or alcohol abuse.

Baseline characteristics

Medical records were used to collect baseline data at the first prenatal visit at 20 weeks gestation including: maternal age, smoking, alcohol consumption, medications, obstetric history, medical history, presence of cyanosis (oxygen saturation <90%), underlying heart disease, prior interventions, cardiac sequelae, cardiovascular event history, New York Heart Association (NYHA) functional class before and during pregnancy as well as postpartum, prior and present cardiac status including ventricular and valvular function assessed according to the recommendations and guidelines of the European Association of Echocardiography/American Society of Echocardiography.14–17

Risk assessment

Maternal cardiovascular and offspring risk of pregnancy were scored by two investigators who were ignorant of pregnancy outcome according to the aforementioned risk assessment models using the baseline characteristics. The final score was based on consensus. Based on the presence of independent predictors, the ZAHARA I and CARPREG maternal cardiovascular risk scores were calculated (table 1).2 ,7 Although the previously published ZAHARA I and CARPREG studies presented independent predictors for offspring events, both papers lacked a risk score for offspring events. Since we had full access to the ZAHARA I data, we were able to calculate the ZAHARA I offspring risk score using identical methodology as previously described for the maternal cardiac risk score.2 To calculate the ZAHARA I offspring risk score, we used the exponent value of the previously identified independent predictors for the composite offspring endpoint to weigh the risk factors and attribute points per risk factor (see online supplementary appendix 1 for details). Table 1 describes the ZAHARA I offspring risk score and the corresponding offspring risk during pregnancy. We also developed an offspring risk score based on the independent predictors identified in the CARPREG study by using the exponent value of the odds ratios (ORs) from the independent predictors for offspring events published by the CARPREG investigators to weigh the risk factors and attribute points per risk factor7 (table 1). Additionally, the total number of (non-overlapping) predictors of maternal cardiovascular events and offspring events (TPo) of ZAHARA I and CARPREG, as well as the predictors from the study of Khairy et al, were assessed (predictors of the study of Khairy et al for maternal risk were: severe pulmonary regurgitation or subpulmonary ventricular dysfunction and smoking history; for offspring risk: subaortic ventricular outflow tract gradient >30 mm Hg). Patients were also classified according to the modified WHO classification of pregnancy risk (table 1) and according to DC.9–11 For DC, we used the generally accepted three categories of Warnes et al:10 simple CHD (ie, isolated aortic or mitral valve disease, small atrial septal defect, mild pulmonic stenosis, repaired atrial or ventricular septal defect), moderate complex CHD (ie, atrioventricular septal defect, coarctation, Ebstein's anaomaly, tetralogy of Fallot) and complex CHD (ie, cyanotic CHD, transposition of the great arteries, Fontan procedure, truncus arteriosus).

Table 1

CARPREG and ZAHARA maternal cardiovascular and offspring risk scores, and modified WHO classification of maternal risk


We scored maternal cardiovascular and offspring events for each pregnancy according to the definitions used in the CARPREG and ZAHARA I studies.2 ,7 Primary cardiovascular events were: cardiovascular mortality, clinically significant (needing treatment) arrhythmia, clinically significant (needing treatment) heart failure, thromboembolic events (eg, pulmonary embolism, valve thrombosis or deep venous thrombosis), vascular events (eg, stroke, myocardial infarction or dissection), need for urgent or invasive cardiovascular intervention up to 6 months postpartum, and endocarditis.2 ,7 Secondary cardiac events were: NYHA class deterioration ≥2 points compared to baseline. Offspring events were: fetal death, neonatal death, premature birth (delivery <37 weeks gestation), small for gestational age birth weight (<10th percentile), respiratory distress syndrome, infections leading to hospital admission, neonatal intensive care unit admission, cerebral intraventricular haemorrhage, occurrence of CHD and occurrence of other congenital disease.

Statistical analysis

We used SPSS (IBM SPSS Statistics, V.19.0, IBM SPSS Statistics, IBM Corporation, Armonk, New York, USA) and STATA (V.12.0, StatCorp LP, College Station, Texas, USA) for statistical analysis. Descriptive statistics for nominal data were expressed in absolute numbers and percentages. Continuous variables with normal distribution are presented as mean with standard deviation (SD) (±SD), whereas non-normal distributed variables as median with inter quartile range (IQRs), and dichotomous variables are presented as absolute numbers with percentages. All p values are two-sided. External validation of the CARPREG cardiovascular risk score and ZAHARA I cardiovascular and offspring risk scores were performed by plotting the expected versus observed event rates. We also calculated the area under the receiver operating characteristic (ROC) curve (AUC) to compare the discriminative capacity of the different cardiovascular and offspring models.18 ,19 An AUC>0.90 is generally considered perfect, while an AUC<0.70 is considered poor, and an AUC=0.5 is considered a worthless test. The best combination of risk assessment models was assessed by calculating the AUC following logistic models for the different test combinations. The p value for the AUC was calculated using the χ2-test, testing for random guess (AUC=0.5). The calibration of the model was assessed using the Hosmer-Lemeshow Goodness-of-fit test.20

Ethical considerations

The Dutch Heart Foundation had no role in the design, data collection, analysis, interpretation, writing of the manuscript or the decision to submit for publication of this manuscript. The corresponding author has full access to all data and the responsibility for the submission of this manuscript for publication.


We Identified 234 women with structural CHD who were eligible for participation. Twenty-one women were excluded, because of miscarriage (n=11), serious protocol violation (n=6) and withdrawal of informed consent (n=4). A total of 213 pregnancies in 203 women were observed (209 singleton and 4 twin pregnancies). None of the included women had uncorrected cyanotic disease or SpO2<90%, severe pulmonary hypertension or Eisenmenger syndrome, impaired glucose tolerance or hypertensive disorder of pregnancy. Maternal baseline characteristics are shown in table 2.

Table 2

Maternal baseline characteristics (prior to pregnancy)

Maternal cardiovascular events

The distribution of cardiovascular and offspring events by primary type of CHD are shown in table 3. No maternal death occurred. Primary cardiovascular events were observed in 22 pregnancies (10.3%). Most frequent events were clinically significant arrhythmias (n=14), followed by heart failure (n=8) and thromboembolic events (n=4). Women with a history of arrhythmia (n=19) had six cardiovascular events, including four arrhythmias. One woman underwent pacemaker implantation because of atrioventricular block. Women with a mechanical valve prosthesis (n=11) had six cardiovascular events including valvular thrombosis in four pregnancies (36.4%). Deterioration of NYHA functional class ≥2 points (secondary cardiovascular event) occurred in 39 pregnancies (18.3%).

Table 3

Distribution of cardiovascular and offspring events by primary type of congenital heart disease in 213 completed pregnancies

Offspring events

Offspring events occurred in 81 children (37.3%), corresponding to 74 pregnancies. The distribution of offspring events per CHD subtype is shown in table 3. Thirty children (12.2%) were born prematurely (50% due to preterm labour); 34 children (16%) were born small for gestational age; 15 children (6.9%) had respiratory distress syndrome (66.7% were born premature) and three children (1.4%) had a cerebral (intraventricular) haemorrhage. Recurrence of CHD occurred in 12 children (5.5%). Offspring death occurred in six children (2.8%). Four children died in utero (>20 weeks gestation). Two children died within 28 days after birth.

Validation of risk scores and comparison of different risk assessment techniques

Figures 1 and 2 show the risk of primary cardiac events during pregnancy per risk assessment technique. Overestimation of cardiovascular risk (expected events>observed events) was observed in the ZAHARA I and CARPREG cardiovascular risk scores mainly in the mid-segment and/or high-risk segment, where a relatively low number of patients could be included.

Figure 1

Expected maternal cardiovascular risk (%) and observed cardiovascular events (%) for the ‘ZAHARA I’ and ‘CARPREG’ cardiovascular risk scores.

Figure 2

Observed maternal cardiovascular events (%) for the cardiovascular risk assessment models ‘Total number of Predictors’, ‘WHO Classification’ and ‘Disease Complexity’. CHD, congenital heart disease; DC, Disease Complexity; TP, Total number of Predictors.

Figure 3 shows the ROC for cardiovascular events for the different risk assessment models. All ROC curves of cardiovascular events deviate significantly from the diagonal line (no discrimination), with exception of the CARPREG risk score (AUC 0.57; 95% CI 0.43 to 0.70; p=0.32). The AUC for the ZAHARA I cardiovascular risk score was 0.71 (95% CI 0.59 to 0.83; p=0.001). Of the five cardiovascular risk assessment models, WHO classification had the highest AUC for prediction of maternal cardiovascular events (AUC 0.77; 95% CI 0.67 to 0.87; p<0.0001). A combination of WHO classification, TPc and DC had a slightly higher AUC: 0.80; 95% CI 0.71 to 0.90; p<0.0001).

Figure 3

Receiver operating characteristic curves of maternal cardiovascular events for the different cardiovascular risk assessment models. AUC, area under the curve; ROC, receiver operating characteristic. *Composite ROC: optimal combination of risk assessment models (WHO class, total no. cardiovascular predictors and disease complexity).

Figures 4 and 5 show the risk of offspring events in women with CHD per risk assessment technique. All risk assessment techniques, with the exception of DC, show an increase in offspring risk with increasing risk points, number of predictors or class. All models performed badly in predicting offspring events in our cohort, with all AUC≤0.6. A combination of the different risk assessment models provided the highest AUC, but still not sufficiently discriminative (AUC: 0.63).

Figure 4

Expected offspring risk (%) and observed offspring events (%) for the ‘ZAHARA I’ risk score and observed offspring events (%) for CARPREG risk points.

Figure 5

Observed offspring events (%) for ‘number of offspring predictors’, ‘WHO risk class’ and ‘complexity of CHD’. CHD, congenital heart disease; DC, disease complexity.


This study is the first to prospectively validate, compare and integrate the different risk assessment models that are used to predict cardiovascular and offspring risk during pregnancy and puerperium in women with CHD. All risk assessment models are able, to some extent, to identify women with CHD at risk of primary cardiovascular and offspring events. When comparing the five individual risk assessment models, the modified WHO classification provides the most adequate individual assessment of maternal cardiovascular risk in our cohort. For the assessment of offspring events, the difference in AUC is very small between the models. A combination of the assessment models provides the highest AUC, although not sufficiently discriminative.

Maternal cardiovascular events

The cardiac event rate in our cohort is low compared to some other studies.5 ,21–23 The difference in observed cardiovascular events is mainly due to differences in study population and in definition of primary and secondary cardiovascular events. Several other studies found comparable cardiovascular event rates.1 ,2 ,4 ,7 ,24 ,25 Our cohort is a relative low-risk cohort, with 99% of women in NYHA class ≤II prepregnancy and no women with cyanosis or pulmonary arterial hypertension. Well organised prepregnancy counselling in the tertiary centres in The Netherlands prevents most high-risk women from becoming pregnant, which can explain the relatively low event rate in our cohort.

Validation of cardiovascular risk assessment models

The ZAHARA I risk score discriminates the cardiovascular events in pregnancy better in this cohort of women with CHD than the more widely used CARPREG risk score. The AUC for ZAHARA I is higher and it deviates significantly from random guess (AUC=0.5), unlike the CARPREG risk score. The low prevalence of systemic ventricular dysfunction and high NYHA class, as well as the absence of mitral valve stenosis in our cohort is the most likely explanation. The CARPREG risk score overestimates maternal cardiovascular risk in our cohort, in line with other studies.4 ,5 ,24 ,25

Overall, in our cohort of women with CHD, the modified WHO classification discriminates best for cardiovascular events. This is not surprising, since the modified WHO classification integrates all knowledge about maternal risk, including known contraindications for pregnancy which are ignored in the CARPREG and ZAHARA I risk scores, as well as known predictors found by CARPREG and other studies, underlying heart disease and other morphological and clinical variables. We recently confirmed that modified WHO class was associated with maternal complications in this population.26 This is in line with recently published data from the Registry On Pregnancy And Cardiac disease (ROPAC). In this registry, modified WHO class was strongly associated with maternal cardiac events and was an independent predictor of heart failure during pregnancy.27 ,28 A disadvantage of the WHO class may be that expert knowledge is sometimes required, especially when choosing between WHO class II and class III. Whether physicians with less expertise might make a different choice than a more experienced physician was not assessed in our study. Finally, it is important that WHO class I has a negative predictive value of 100% for maternal cardiovascular events, indicating that pregnancy is relatively safe in these women.

A combination of the risk classification systems from the WHO class with total number of cardiovascular predictors and DC provides the most adequate assessment of cardiovascular risk in pregnancy. This illustrates that integration of clinical information and predictors, or population-based information, has additional value on top of individual risk assessment models.

Offspring events

The offspring event rate observed in our cohort is comparable to most other studies in women with CHD.1 ,2 ,4–7 21–23 ,29 ,30 Offspring death occurred in 2.8% of pregnancies. Although offspring mortality in our cohort is in accordance with previous studies in women with CHD, it is much higher than in the general Dutch population.2 ,4–7 ,23 ,31 Also, premature births, small for gestational age and recurrence of CHD occurred more often than would be expected in the general Dutch population. Our previous research indicated a strong association between maternal cardiac events and offspring events.2 We recently found evidence that maternal cardiac function is related to uteroplacental flow parameters, while uteroplacental flow is impaired in women with CHD. Since uteroplacental flow is related to offspring outcome, the high prevalence of offspring complications in women with cardiac disease may be explained by suboptimal placental function related to maternal cardiac dysfunction.32

Validation of offspring risk assessment models

All risk estimation models preformed insufficiently in predicting offspring risk. The risk models predicting offspring events appear to be interchangeable, because the differences in AUC are very small, especially between ZAHARA I, CARPREG and TPo. This is explained by the huge overlap between the risk factors found by ZAHARA I and CARPREG. The ROPAC investigators reported a strong association between modified WHO class and offspring outcome, especially preterm birth and birth weight, which we could not confirm.27 Our results may be explained by the fact that the modified WHO classification was not designed to assess offspring events in women with CHD, and therefore, does not take into account factors such as maternal age, parity, smoking and twin pregnancy, which are known risk factors for offspring events. This is probably also the main reason why DC alone is not an accurate predictor of offspring events.

Strengths and limitations

The participation rate was excellent with 98% of women providing written informed consent and only two women lost to follow-up. Although inclusion rate is high, some limitations need to be addressed. Some inclusion bias might have been introduced, since only pregnancies of ≥20 weeks were taken into account. The risk estimation models that we tested may have been used in clinical practice in our study population, which may explain why no patients with a high risk of maternal death, such as Eisenmenger syndrome, could be included, and which may have improved management of pregnancies. Nevertheless, the distribution of the CHD subtypes adequately represents a tertiary hospital pregnant CHD cohort. Though women in the study mainly received standard care during their pregnancies, it cannot be excluded that their participation in the study influenced clinical management. The available risk prediction systems that we validated did not allow prediction of more threatening events such as heart failure separately from more innocent events such as supraventricular arrhythmias. The risk assessment systems that we validated have their own limitations. They do not take into account some important predictors of pregnancy outcome such as pulmonary hypertension.28 NT-proBNP at 20 weeks gestation has recently been validated as a valuable parameter of pregnancy outcome but is not included in present risk scores.26 Despite the limitations, our study is the first prospective study to validate, compare and integrate the available risk estimation models to predict the cardiovascular and offspring risks during pregnancy in women with CHD.

Key messages

What is known on this subject?

  • Adequacy of prepregnancy prediction of maternal cardiovascular and offspring risk associated with pregnancy in women with congenital heart disease determines, for a large part, the efficacy of counselling prior to, and management during, pregnancy. Various studies previously validated the CARPREG risk score. The ZAHARA risk score, disease complexity, a total of all risk factors and the modified WHO classification, however, have not been validated and compared systematically before in a prospective study.

What might this study add?

  • This study is the first to prospectively compare and validate the various prediction models. It shows that the modified WHO classification is the most favourable risk estimation model for cardiac risk in pregnancy, and that there are no effective risk models for the prediction of offspring events yet.

How might this impact on clinical practice?

  • This study shows that the most widely used CARPREG risk score is far less accurate than the modified WHO classification. This finding will bring about a shift in use of risk score from CARPREG to the modified WHO classification.


View Abstract
  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:


  • Contributors All authors have contributed significantly to the submitted manuscript: AB, JGA, KMS-S, BJMM, JWR-H, APJvD, HWV, WD and PGP were responsible for the study design. Data collection and processing was performed by AB, TPER, AGLvdB, APJvD and EMCJW. Analysis and interpretation of the data was done by AB, HLH, PGP, JWR-H, BJMM, JGA and DJvV. The drafting of the article was performed by AB, HLH, PGP, AGLvdB, TPER, BJMM, WD and DJvV. Critical revision of the manuscript was performed by KMS-S, JWR-H, APJvD, HWV, EMCJW, WD and JGA. AB, PGP, BJMM, JWR-H and DJvV were responsible for the final approval of the version to be published.

  • Funding This work is supported by a grant from the Netherlands Heart Foundation (2007B75); DJvV is a clinically established investigator of The Netherlands Heart Foundation (D97-017).

  • Competing interests None.

  • Ethics approval The study was conducted according to the principles of the Declaration of Helsinki and in accordance with the Medical Research Involving Human Subjects Act (WMO). The Medical Ethics Committee of all participating hospitals approved the study.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles