Statistics vs Real Life: a difficult coexistence

Rui A. Providencia, ,

Other Contributors:

September 24, 2013

Messori et al [1] present a very interesting matter (the difference between inconclusive results and demonstrating non-inferiority) concerning our recently published meta-analysis of dabigatran vs. warfarin in the setting of catheter ablation of atrial fibrillation [2] that merits a practical reflection. First, proving that two treatments are equal in performance is impossible with statistical tools; at most, one can show that they are equivalent using statistical test that aims at showing that two treatments are not too different in characteristics, where "not too different" is defined in a clinical manner. In our paper, non-significant results were obtained when comparing dabigatran with warfarin for thromboembolic complications and major bleedings. Thus, we cannot claim that dabigatran is different in performance than warfarin, which is our conclusion. We have evaluated the validity of our meta-analysis and tried to control for heterogeneity of data using several sensitivity analysis. These have shown similar results in the differently tested scenarios [2]. Moreover, we have gathered a great body of evidence and presented reassuring results supporting the absence of marked differences between the two studied anticoagulant options. With negative results, the question is always to know if such finding is due to lack of power or whether the new treatment is likely to be similar to the other. Negative results often occur because the sample size is too small, and thereby the probability of making a type II error is too large. Trial sequential analysis (TSA) is a recently introduced statistical tool that may be "applied to determine the optimal information size required to reach a conclusive result". This tool has suggested in a study that between 80 to 90% of all assessed non-significant meta-analysis might eventually have been underpowered to that purpose [3]. This is such a big issue. According to the results of the Messori et al. [1], the TSA methods applied to our meta-analysis failed in showing conclusive result because the number of events were insufficient to construct the boundaries of futility ('no effect'). Moreover, Messori and colleagues' estimations would suppose to conduct a trial with more than 18.000 patients to compare the two drugs, which is unrealistic and in face of the very low incidence of events. Also, according to their opinion, this problem would occur to any of the other novel anticoagulants to be tested. For example, "A Study Exploring Two Treatment Strategies in Patients With Atrial Fibrillation Who Undergo Catheter Ablation Therapy "(VENTURE-AF trial), comparing rivaroxaban and warfarin, estimates an enrollement of 200 patients [4]. Additionally, the "Catheter Ablation vs Anti-arrhythmic Drug Therapy for Atrial Fibrillation Trial" (CABANA), one of the largest catheter ablation of atrial fibrillation trials to date, will enrol 3000 subjects during more than 6 years [5]. However, based on the event rates observed in our meta-analysis, for a non-inferiority trial intending to show that the effect of a dabigatran is not worse than that of warfarin by more than a specified margins (0.5% for thromboembolic and 0. 7% for major bleedings) the inclusion of 4782 patients, as done in our meta-analysis, allow to achieve around 80% power (beta=0.2) at alpha=0.05. Final and robust answer to this important issue may be given in the next few years, with data available from a larger number of patients included in future trials that will allow us to make a more powerful meta- analysis. In the meantime, with the limitations of meta-analysis methodology previously discussed, our study shows that there is no significant difference between dabigatran and warfarin in patients undergoing catheter ablation of atrial fibrillation.

References: 1. Messori A, Fadda CV, Maratea D, Trippoli S. Comparing dabigatran vs warfarin in patients with atrial fibrillation undergoing catheter ablation: inconclusiveness of the results concerning thromboembolic complications and major bleedings. Heart.2013 Sep 16. [EPub] 2. Providencia R, Albenque JP, Combes S, Bouzeman A, Casteigt B, Combes N et al. Safety and efficacy of dabigatran versus warfarin in patients undergoing catheter ablation of atrial fibrillation: a systematic review and meta-analysis.Heart.2013 Jul 22. [Epub ahead of print] 3. Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses. J ClinEpidemiol. 2008;61(8):763-9. 4. A Study Exploring Two Treatment Strategies in Patients With Atrial Fibrillation Who Undergo Catheter Ablation Therapy (VENTURE-AF) , available at http://clinicaltrials.gov/show/NCT01729871 5. Catheter Ablation vs Anti-arrhythmic Drug Therapy for Atrial Fibrillation Trial (CABANA) , available at http://clinicaltrials.gov/ct2/show/NCT00911508

Conflict of Interest:

RP has received honoraria for serving as a speaker and consultant for Boheringher-Ingelheim and as a co-investigator in the ENGAGE-AF TIMI 48 trial.

Conflict of Interest

None declared