Article Text

PDF

Validation of the EuroQol questionnaire in cardiac rehabilitation
  1. B Schweikert1,*,
  2. H Hahmann2,
  3. R Leidl1,*
  1. 1Ludwig-Maximilians University, Munich School of Management, Health Economics and Management, Munich, Germany
  2. 2Schwabenlandklinik, Isny-Neutrauchburg, Germany
  1. Correspondence to:
    Bernd Schweikert
    MSc, Ludwig-Maximilians University, Munich School of Management, Health Economics and Management, Ludwigstraße 28 RG, D-80539 Munich, Germany; schweikert{at}bwl.uni-muenchen.de

Abstract

Objective: To analyse the psychometric properties of the EuroQol questionnaire (EQ-5D) applied to patients with acute coronary syndromes (ACS).

Setting: Rehabilitation hospital.

Patients and design: 106 consecutive patients with ACS (51% myocardial infarction, 42% coronary artery bypass grafting, 7% angina) completed the EQ-5D, the 36 item short form health survey (SF-36), and the MacNew questionnaire at admission, at discharge, and three months after inpatient cardiac rehabilitation. Acceptance, validity, reliability, and responsiveness of the EQ-5D were tested.

Results: The EQ-5D was highly accepted. The EQ-5D index showed substantial ceiling effects after rehabilitation. As expected the EQ-5D visual analogue scale (VAS) score (70.3 v 57.1) and EQ-5D index (77.8 v 64.5) were significantly better for patients with myocardial infarction than for patients who underwent surgery (both p ⩽ 0.001). Significant correlations were found between the EQ-5D VAS score, EQ-5D index, and domains of the SF-36 (r  =  0.21 to r  =  0.74). The correlation with the MacNew subscores and with the global score ranged between 0.55 and 0.78. With repeated measurement the EQ-5D showed reasonable reliability in stable patients with intraclass correlation ranging between 0.91 and 0.54. EQ-5D was responsive in patients who indicated improvement in health states between admission and discharge (effect size 0.74–0.82).

Conclusion: The psychometric properties of the EQ-5D were satisfying. It is a reasonably valid, reliable, and responsive instrument for patients with ACS. It may be useful in clinical research and epidemiological studies to generate preference based valuations of health related quality life.

  • ACS, acute coronary syndromes
  • EQ-5D, EuroQol questionnaire
  • HRQoL, health related quality of life
  • MI, myocardial infarction
  • NYHA, New York Heart Association
  • SF-36, 36 item short form health survey
  • VAS, visual analogue scale
  • cardiac rehabilitation
  • quality of life
  • EuroQol questionnaire
  • EQ-5D

Statistics from Altmetric.com

Acute coronary syndromes (ACS) are highly prevalent in western countries. The diagnosis of ACS is associated with a high mortality and leads to increased morbidity, including chronic physical impairments and functional limitations, all of which have an impact on health related quality of life (HRQoL). HRQoL is a subjective measure of well being comprising social, mental, and physical dimensions.1

Instruments used to measure HRQoL can be divided into generic and disease specific instruments. Generic instruments assess the overall health state and allow comparison of HRQoL across different health problems. They are therefore suited to support decision making at the population level.2 In comparison, disease specific instruments focus on certain aspects of the disease and are considered to be more sensitive to health changes.3

Preference based summary scores of HRQoL are increasingly used in medical research and accepted by physicians as a complementary measure of the medical effectiveness of interventions.4,5 They are also recognised as a major outcome in the evaluation of new health technologies, which has a growing influence on allocation decisions in the health care sector.6,7

The EuroQol questionnaire (EQ-5D) was developed by an international task force aiming at creating a simple generic measure that aggregated HRQoL into a single index.8 In contrast to summary scores of other HRQoL instruments, the EQ-5D allows preference based valuation of the health state with the visual analogue scale (VAS) used for a patient’s subjective valuation9 and a survey based index reflecting community preferences.10

The EQ-5D has been used in numerous studies for various indications, including coronary heart disease.11,12

Validity and reliability of the questionnaire have been tested for various diseases and for the general population.9 However, to our knowledge, the psychometric properties of the EQ-5D have not been previously assessed for patients with ACS. Thus, the goal of the study was to analyse the psychometric properties (validity, reliability, and responsiveness) of the EQ-5D for testing patients after ACS.

METHODS

Patients

For the study, 114 consecutive patients starting inpatient rehabilitation after an acute cardiac event were recruited at admission to the Schwabenlandklinik, a rehabilitation hospital in Southern Germany. Exclusion criteria were age above 65 years, insufficient command of the German language, and an acute cardiac event occurring more than three months before admission to the rehabilitation hospital.

Questionnaires

Patients received a questionnaire at admission, at discharge, and three months after discharge from rehabilitation. It consisted of questions on demographic data as well as the German versions of the EQ-5D, the 36 item short form health survey (SF-36), and the MacNew questionnaire. The clinic staff handed out and collected the questionnaire during the patients’ inpatient stay. The questionnaire three months after discharge was sent by mail.

EQ-5D

The EQ-5D comprises five questions asking for the current health state in the five dimensions: mobility, self care, usual activities, pain/discomfort, and anxiety/depression.8,9,13 The categories of the response offer three levels: no problems (score of 1), moderate problems (2), and extreme problems (3). This part is called “self classifier” and provides a five dimensional description of the health state. Combining the classifications yields 35  =  243 different health states, which can be expressed as a five digit number. For example, 11122 indicates no problem in usual activities, mobility, and self care but moderate problems with pain and anxiety/depression.

In this study the descriptive part of the questionnaire was followed by a VAS, a graph representation similar to a thermometer that ranges from 0 (worst imaginable health state) to 100 (best imaginable health state).

The questionnaire used at clinic discharge and three months after discharge also included a transition question, which asked whether patients felt better, worse, or unchanged compared with their previous situation (that is, at admission and at discharge, respectively).

Different approaches can be used for the preference based valuation of EQ-5D health states. The direct method is to take the VAS value the patient has attributed to his or her current health state. An alternative method is based on the EQ-5D valuation questionnaire. In several countries people in a sample of the general population were asked to value various health states defined by the EQ-5D descriptive system. These preference weights were used to develop national valuation formulas for all 243 EQ-5D health states.13,14 In a recent study by the EuroQol group, data from six countries were pooled and a single European formula was derived that additively aggregates the five dimensions describing the health state.15 This EQ-5D index based on the European formula has been used in this study.

Whereas the subjective rating of their own health state on the VAS reflects the respondents’ personal valuation of their health state, the EQ-5D index reflects health state valuation based on general population preferences. Population preferences become increasingly important in making decisions on the coverage and reimbursement of new health technologies.

SF-36

The SF-36 is a generic HRQoL questionnaire containing 36 items that are grouped into eight domains measured on a scale from 0 to 100.16 In the version used in this study items referred to the week preceding questioning. The SF-36 was used as a reference instrument, since its psychometric properties have been tested in the general population and among patients with ACS and related cardiovascular diseases.17–19 Furthermore, it has been widely used in cardiac studies and its 36 items cover similar aspects of HRQoL in more detail. In contrast to the EQ-5D, the SF-36 provides a purely descriptive measure without a preference based valuation of the health state. The German version of the SF-36 has been tested and validated in a general population sample.20

MacNew HRQoL questionnaire

The MacNew questionnaire is a disease specific instrument originally designed for measuring quality of life after myocardial infarction (MI) and has been refined since.21–23 It contains 27 items referring to the preceding two weeks, which can be combined in one global score. Three subscales (physical functioning, emotional functioning, and social functioning) can also be calculated. Scores are normalised in the range from 1 (poor) to 7 (high). Norm scores for several patient subgroups with cardiac complaints have been published recently.24 The SF-36 has been found to be a sensitive measure compared with other HRQoL instruments for cardiac interventions.25 The German version of the MacNew has been successfully validated in a sample of patients with coronary artery disease.26

Psychometric analysis

Acceptance and feasibility were assessed by the proportion of missing entries and invalid responses. Ambiguous responses to the EQ-5D (self classifier) and EQ-5D VAS were rated invalid.

The ability of the EQ-5D to discriminate between health states was assessed by searching for a tendency towards a single level of response as well as ceiling effects. A ceiling effect means that the instrument is unable to discriminate between comparatively good health states.

To assess construct validity of the EQ-5D several hypothesis were tested. Firstly, we hypothesised that the EQ-5D VAS and EQ-5D index score correlate positively with physical capacity measured by the peak load in ergometric exercise. Secondly, we hypothesised that the level of response to the EQ-5D self classifier and the EQ-5D VAS score and the EQ-5D index indicate better HRQoL for patients after MI than for patients after coronary artery bypass graft surgery (CABG). We intended to compare EQ-5D scores across New York Heart Association (NYHA) classes; however, owing to the unbalanced distribution of patients in NYHA classes, a meaningful comparison was not possible.

Criterion validity was tested by comparing EQ-5D response with the corresponding scores of the SF-36 and MacNew questionnaire. With respect to the EQ-5D self classifier we analysed the association between response level and the median scores of the SF-36 and MacNew. For the EQ-5D VAS score and EQ-5D index, correlation coefficients with the SF-36 subscales and MacNew scores were calculated.

To assess test-retest reliability of the subjective health state, consecutive measurements of EQ-5D self classifier and the EQ-5D VAS score were compared for the subsample of patients who indicated in the transition question that HRQoL remained unchanged.

Sensitivity to change was assessed by analysing the change in EQ-5D VAS scores and EQ-5D index of patients indicating in the transition question improvement in health within period 1 (admission to discharge) and period 2 (discharge to three months’ follow up). In addition, effect sizes were calculated for scores of EQ-5D VAS and EQ-5D index by means of the standard deviation at baseline for normalising the mean difference.27

Calculations and statistical analysis were performed with the software package SAS 8.02 (SAS Institute, Cary, North Carolina, USA).

RESULTS

Of 114 patients who where recruited into the trial, eight were excluded from the analysis: five patients withdrew their informed consent for various reasons and three patients had incomplete documentation because of administrative reasons. Thus, the size of the valid sample was 106. Of these participants, 54 (51%) had an MI and 45 (42%) underwent CABG. With the exception of disease duration, patient characteristics were comparable across diagnosis groups (table 1).

Table 1

 Demographic and disease related characteristics

Acceptance and feasibility

The percentage of missing and invalid responses in the EQ-5D self classifier ranged from 0.6% (admission) to 2.9% (discharge). For the VAS it ranged from 2.2% to 4.8% and for the SF-36, from 1.5% (three months) to 6.5% (discharge). For the MacNew questionnaire, from 1.5% to 2.3% had invalid answers. The majority of the participants rated the EQ-5D as easy or very easy to fill in, ranging from 78% (admission) to 85% (discharge) at all three times of completion.

Discriminative ability

Respondents indicated problems in the dimensions usual activities, pain, and anxiety/depression, whereas the dimensions mobility and self care were less affected (fig 1). At admission, the 10 most frequent combinations of the self classifier accounted for 75% of all answers, although these combinations amounted to just 4.1% of all possible combinations. At discharge and after three months, this percentage increased to around 90% with the best possible health state (11111) accounting for 42% and 43%, respectively, whereas at admission this health state was indicated by 15%. Thus, the ceiling effect for the EQ-5D self classifier was substantial. Since the EQ-5D index is calculated based on the classifier its behaviour in terms of ceiling effect is comparable: at admission 15% had values in the top 10% of the scale scoring within the interval 91 to 100 and 42% at the three months. For the VAS the respective proportions were 4% at admission and just 16% three months after discharge.

Figure 1

 Distribution of responses to items of the EuroQol questionnaire (EQ-5D) self classifier. Data are from questionnaires from 88 patients for whom all three surveys (admission, discharge, three months’ follow up (FU)) were available.

Construct validity

Correlation between the EQ-5D VAS score and peak performance on the ergometer was 0.34 (p < 0.001) and 0.28 (p  =  0.006) for the EQ-5D index. We further hypothesised that because of postsurgical complaints HRQoL at admission for patients after bypass surgery is lower than for patients who have had an MI. Comparison of the EQ-5D self classifier showed that with the exception of the dimension anxiety/depression, the percentage of patients citing no problems was significantly higher among patients with MI than among patients after surgery (table 2).

Table 2

 Construct validity: comparison of EuroQol questionnaire (EQ-5D) results between subgroups at admission

For the EQ-5D VAS and the EQ-5D index, mean scores were significantly better for the MI group than for patients after CABG (p ⩽ 0.001).

Criterion validity

Table 3 shows the median scores of SF-36 and MacNew subscales at admission. For all EQ-5D items the median of the subscales of SF-36 and MacNew were ranked as expected and significantly different between groups. With the exception of the subscale role limitations caused by emotional problems, the correlation coefficients between SF-36 subscales and the EQ-5D index score were > 0.5 at all three time points (range 0.57–0.74) (table 4). The correlation of EQ-5D index with the MacNew global score ranged from 0.69 (admission) to 0.78 (at three months). The pattern of correlation of SF-36 and MacNew with the EQ-5D VAS score was similar but mostly lower, ranging from 0.21–0.72.

Table 3

 Association between the 36 item short form health survey (SF-36) and MacNew subscales and EQ-5D item by response level at admission

Table 4

 Spearman rank correlations of EQ-5D VAS score and EQ-5D index with subscales of SF-36 and MacNew

Reliability

Of 106 patients, 11 reported no change between admission and discharge (period 1). Between discharge and three months’ follow up (period 2), 34 patients indicated that “by and large” their health state had not changed. Table 5 shows the rate of agreement and κ statistics of the responses between admission and discharge and between discharge and three months’ follow up. The rate of agreement in EQ-5D self classifier items ranged from 55% to 100% in the first period and 65% to 97% in the second period. The mean rate of agreement was 82% in period 1 and 78% in period 2. Intraclass correlation for the EQ-5D VAS score was 0.82 in period 1 and 0.80 in period 2 and for the EQ-5D index was 0.91 in period 1 and 0.54 in period 2.

Table 5

 Test-retest reliability of EQ-5D, patients with stable health state

Responsiveness

The effect size was calculated for the group of patients with improved health states, since in each period the number of people experiencing deterioration in health state was too low for analysis. The effect size in both periods was higher for the EQ-5D VAS (0.82 for period 1, 0.4 for period 2) than for the EQ-5D index (0.74 for period 1, 0.09 for period 2).

DISCUSSION

Although the EQ-5D has been previously used for patients with ACS,11,12 to our knowledge its psychometric properties have not yet been tested for ACS patients.

The low proportion of missing and invalid entries for the EQ-5D self classifier as well as for the EQ-5D VAS shows that the instrument is well understood and accepted by patients with ACS. This is supported by the low proportion of patients who found it difficult or very difficult to fill in the questionnaire. Compared with the SF-36, the EQ-5D did not present problems when patients were hospitalised, as is shown by the low missing rates at discharge. This also makes the EQ-5D attractive for use in studies in an inpatient setting.

In contrast with the EQ-5D VAS score, the EQ-5D self classifier and the EQ-5D index showed substantial ceiling effects at discharge and three months after rehabilitation. At these time points, the proportions of patients who described their health state as the best possible (11111) were 42% and 43%, respectively, which equals the result found in a survey of the general population.13 This indicates that the EQ-5D may not discriminate between health states. It may also indicate that the patients after rehabilitation were indeed only minimally affected by ACS. This is supported by the results of the MacNew questionnaire: the global scores at admission and three months after discharge were 5.51 and 5.41, respectively, for the total sample and were therefore higher than the mean values of a reference sample of patients 3–5 months after an MI.24 This result also shows the state dependence of subjective health measures. A healthy person may hesitate to rate any health state perfect given the experience of a life threatening ACS just a few weeks earlier.

Construct validity measures the extent to which the instrument is compatible with prediction deduced from established theories. We assumed that HRQoL is positively correlated with physical ability measured by ergometer exercise. Furthermore, we assumed that owing to postsurgical complaints, patients after CABG report lower HRQoL than do patients after MI. The results confirm both hypotheses. The correlation with ergometer exercise was positive and significantly different from zero. The relatively low value of the correlation may be explained by the fact that physical functioning is just one dimension of HRQoL and ergometer exercise is not expected to correlate strongly with dimensions of social or mental well being.

Comparison of EQ-5D response with comparable items and subscale scores of SF-36 and MacNew showed reasonable criterion validity: the EQ-5D index correlated well with seven of eight of the SF-36 health domains, as well as with the MacNew global score and subscores. Although both EQ-5D scores correlate only moderately with the narrowly defined SF-36 subscale role functioning due to emotional problems, which is based on three items, they correlate well with the broader MacNew subscale emotional functioning, combining 14 items.

Furthermore, whereas the EQ-5D asks about the current health state, SF-36 and MacNew have different reference periods. In a relatively dynamic phase of a disease, as is the case for patients with ACS, this may lead to differences in the comparative assessment of changes.

The test-retest reliability of the EQ-5D self classifier was good. The intraclass correlations for stable patients in period 1 are similar to those found for stable patients with stroke.28 Intraclass correlation for period 2 particularly for the EQ-5D index were lower. This may be due to the relatively long time of three months between test and retest, which is also longer than the time spans that are recommended in general.27,29

The good test-retest reliability, especially of the EQ-5D self classifier and EQ-5D index, may be partly due to the ceiling effect, since for patients indicating a health state of 11111, divergence to a better health state is not possible. Some evidence for this effect is provided by an analysis within period 2 excluding patients indicating 11111 at baseline. This analysis produced almost stable intraclass correlations for the EQ-5D VAS score (0.79) and lower intraclass correlation for the EQ-5D index (0.46).

An assessment of the responsiveness of the EQ-5D for the subsample of 81 patients reporting an improvement between admission and discharge measured an effect size of 0.82 for the EQ-5D VAS and 0.74 for EQ-5D index. According to Cohen,30 these effect sizes can be classified as medium to large effects; they are, however, lower than the respective effect size of the MacNew global score (1.06). The effect sizes for the second period were considerably smaller with the EQ-5D VAS similarly being more responsive than the EQ-5D index.

Conclusion

The results of this study indicate that the EQ-5D has good practicability and satisfactory validity and reliability when applied to patients with ACS. The EQ-5D can be used to generate preference based valuations of HRQoL. Since the EQ-5D, consisting of the self classifier and EQ-5D VAS, is very easy to administer in a very short time, it is reasonable to use the questionnaire whenever the general HRQoL is considered to be an important outcome. It may be useful in clinical research and epidemiological studies to generate preference based valuations of HRQoL. Showing overall reasonable psychometric properties the EQ-5D index based on the European formula may be attractive in multinational studies. However, the nature of the EQ-5D as a generic HRQoL instrument inevitably limits its coverage in more detail of aspects of HRQoL considered relevant to a specific disease such as depression or anxiety after ACS. Therefore, the complementary use of a more detailed instrument should be considered.

Acknowledgments

This study was supported by grants from the German Federal Ministry for Education and Research (01GD9820/0) and from the Federation of German Pension Insurance Institutes (02708). The authors thank Gertrud Feike and Matthias Kurz for their support in collection and handling of the data.

REFERENCES

View Abstract

Footnotes

  • * Also the GSF National Research Centre for Environment and Health, Institute of Health Economics and Health Care Management, Neuherberg, Germany

  • Published Online First 29 March 2005

  • Conflicting interest statement: We hereby declare that there are no conflicting interests for any of the authors of this manuscript.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.