Editor,—It is encouraging to see an increasing recognition of the desirability of quoting confidence intervals (CI) in association with indices of the performance of diagnostic tests. It is unfortunate, therefore, when errors are made in the calculation and interpretation of these, as happened in two recent papers in Heart.

A few errors spoil Collinson's1 otherwise useful introduction to ROC curves. First he has defined the prevalence of disease wrongly. Using the notation in his table 1, the prevalence is not (TP+FN)/(TN+FP) but (TP+FN)/(TP+FN+TN+FP), where the denominator is in fact just the total study size. Brackets are essential here to clarify numerator and denominator—these are omitted from several expressions in Collinson's paper. His expression for the likelihood ratio is also incorrect: it should be [TP/(TP+FN)]/[1−TN/(TN+FN)].

Second, Collinson1 notes that variation in the prevalence of disease “will greatly affect the test performance”. While the sensitivity and specificity may vary according to setting (and hence disease prevalence)2 more often the opposite is true. Indeed it is often noted as a characteristic of these measures that they are not affected by disease prevalence.3 For example, the sensitivity of a test is unaffected by how many disease negative patients are included in the study.

Third, he makes a common error in relation to CIs. He suggests that one should compare two sensitivities by seeing whether their CIs overlap. In fact the difference between two sensitivities may be statistically significant even when the CI overlap. The correct procedure when …