Cost comparison of asthma treatments in 12-week study: caution about matching and short observational follow-up

Abstract

In the absence of randomisation, observational studies must take extra care to create treatment groups that are comparable in terms of key characteristics. Various matching methods exist which can create sound comparisons, minimising confounding where possible. A recent observational study by Dal Negro et al. carried out a cost analysis comparing two asthma medications. They report strong conclusions which favour one treatment over the other, however they include little discussion on the limitations of their study. The purpose of this letter is to comment on the weaknesses of the study design, including the level of matching used, and to urge readers to consider these issues alongside the interpretation of results.

Dear editors,

Multidisciplinary Respiratory Medicine recently published the following study about asthma treatments: “Fluticasone furoate/Vilanterol 92/22 μg once-a-day vs Beclomethasone dipropionate/ Formoterol 100/6 μg b.i.d.: a 12-week cost analysis in mild-to-moderate asthma”, by Dal Negro et al. We are moved to comment on this research as there are several issues that we believe should be brought to the attention of your readers to allow for a more balanced interpretation of its results.

The first issue is that there is potentially serious confounding of the results due to poor study design. In an observational study where no randomisation is carried out, the authors are right to have applied propensity-score matching, and have matched patients using age, sex, FEV₁ and co-morbidities. However, in a study where asthma outcomes are being compared, it is vital that asthma severity and treatment history are used as matching criteria. FEV₁ alone is not sufficient. In this study scenario, it is absolutely key to know how these patients have been treated in the baseline period. We know that they were on a combination of long-acting beta agonist and inhaled corticosteroids (LABA/ICS) for at least 12 weeks during follow-up, but the authors do not make clear when they were stepped-up to this therapy. The potential for confounding is clear, as the outcomes of a patient whose step-up was only recently initiated may be different to those of a patient who has had a longer period of treatment stability. It is also key to know the event history (including exacerbations and hospitalisations) of the study sample. These factors are strongly predictive of future exacerbations [1] and the results of a study which fails to account for these may not be reliable. The reliability is questioned further due to the small sample size of the study (only 40 patients in each group), in which an imbalance between treatment groups could be more problematic. The follow-up period of 12 weeks is far too short to assess asthma outcomes; longer-term observation periods (at least 6–12 months) are needed to take into account variability in outcome (especially given that asthma is a chronic disease) and confounders such as seasonal variability. The authors acknowledge this limitation, but mainly in order to present it as the sole reason for a lack of significance in the difference between hospitalisation costs.

The second issue concerns the relevance and interpretation of the results of the cost analysis. The authors carried out a cost-consequence analysis, where disaggregated changes in clinical and cost outcomes are estimated. Reporting results in a disaggregated way shifts the burden of interpretation on to the consumer of the research or clinical decision-maker [2]. What would be more useful is to present the change in costs relative to change in consequences, i.e. cost-effectiveness analysis. As for the results, the mean reductions in costs for GP visits, specialist visits, and hospitalisations are small and, whilst statistically significant for GP and specialist visits, show very small shifts in costs. For example, as reported by the authors, the cost of an asthma-related hospitalisation with a comorbid disease was 2537 Euros and 1832 Euros without a comorbid disease. The change in cost of hospitalisation (173 Euros) between the groups was not statistically significant and it could hardly be considered a meaningful change for health care providers, as it accounts for roughly 7–9 % of the cost of one hospitalisation. Similarly, costs for GP and specialist visits may have been statistically significant, but lack meaningful changes in costs that would result in more efficient or affordable use of one treatment over another in this analysis. Based on these results, it is misleading to claim in the conclusions that fluticasone fuorate/vilanterol “showed clear potential for enhanced clinical outcomes and reduced costs”. A more likely conclusion would carefully claim that no meaningful changes in costs occurred at 12 weeks and that extended follow-up with larger sample sizes are needed prior to making claims on cost reductions.

There are some items in the paper that deserved further discussion from Dal Negro et al. The first is the relevance of their dose comparison. Although dose choices are largely at the discretion of clinicians, based on the Summary of Product Characteristics (SPC) of fluticasone fuorate/vilanterol and clinical trial data comparing fluticasone fuorate and fluticasone propionate, [3–5] the dose of beclomethasone dipropionate equipotent to fluticasone fuorate should be higher than the dose used in this study. It would be of interest to hear the authors’ perspective on the usefulness of their comparison. The second item is that the rate of observed hospitalisations in this mild to moderate study sample is high in comparison with the literature. In other studies, hospitalisation rates have been up to ten times less over the equivalent follow-up period, even in patients with uncontrolled asthma [6, 7]. This deserves discussion as it raises the question of the representativeness of this sample. Thirdly, the method of adherence measurement is not made clear in the paper. If we assume dose counters on the devices were utilised, there is an issue of comparability as the Nexthaler’s adherence monitor counts inhaled doses, whereas the Ellipta counts loaded doses. It is a relevant matter to clarify as the authors write that they have used adherence data to compute the cost of medications.

Lastly, an important limitation of the study is that emergency department visits were not included in the cost analysis, though this event is considered one of the core recommended cost outcomes for asthma [8].

The points discussed above should be carefully considered when interpreting the results of this paper. We hope that they will assist readers in making a fairer assessment of Dal Negro’s study.

Authors’ response

• Roberto W. Dal Negro &
• Sergio Iannazzo 

We read carefully the letter received, and concerning some criticisms to the paper “Fluticasone furoate/Vilanterol 92/22 μg once-a-day vs Beclomethasone dipropionate/ Formoterol 100/6 μg b.i.d.: a 12-week cost analysis in mild-to-moderate asthma”, recently published in Multidisciplinary Respiratory Medicine (2016;11:20).

We are rather surprised by the unfair and unbalanced description of our study. The Authors of the letter essentially mention that we draw “strong conclusions which favour one treatment over the other” based on a poor study design. We do not really see the strong conclusions in our paper, when the final sentence in the published abstract stands out literarily as: “Even if both ICS/LABA combinations were checked over a limited period of time, they seem characterized by a different profile in terms of their effect on lung function and of their economic impact on mild-to-moderate asthma. The once-daily inhalation of combined Fluticasone furoate/Vilanterol 92/22 μg showed the potential for enhanced clinical outcomes and reduced costs when compared to Beclomethasone dipropionate/Formoterol 100/6 μg b.i.d.” In other terms we report a trend, which is in the numbers, but that has to be confirmed by a longer observation.

Furthermore, we wish to pinpoint that the study is an observational, retrospective study, and that data were obtained from an institutional general database which is collecting files of asthma patients managed in real-life. In other words, as clearly perceived from the paper, the study does not belong to the traditional pharmacological trials where subjects have to be preliminarily randomized into different arms. Actually, this was the main reason for the adoption of the propensity matching score. The Authors of the letter believe that we correctly applied the method but left key information unmanaged, leading to a poor design. Essentially, they mention asthma severity and treatment history (i.e. when patients were stepped up to study therapies) as key characteristics that should have been used as matching criteria. We believe this criticism derives from a misunderstanding of our study. Differently from the opinion of the Authors’ letter, we do not perceive any “serious confounding”, because it appears very clearly that not-severe asthma was the unique target of the survey. Actually, the high mean basal FEV₁ value (>82 % pred.), together with the low clinical impact already registered in basal conditions do represent clear real-life indices from this point of view, independently of previous patients’ therapeutic strategies.

Moreover, all automatically selected patients were equally stepped-up exactly at their recruitment date (such as 12 weeks ± 1 week before the end of the survey): previously, they were regularly treated and were in consolidated stable conditions before their step-up date. They also were well matched at the step-up date and absolutely undistinguishable, both in terms of their lung function and of clinical impact. To underline that basal data were obviously referred to the period immediately preceding the start of the survey, and then what the Authors of the letter were requiring and are worried of was already reported in the paper (Table two, column A for both the patients’ samples).

As concerning the duration of the survey, we only wish to recall that, differently than in COPD, several clinical trials carried out in asthma with the aim to check and compare the effects of different drugs (vs placebo or not) had, and still have, a 12-week duration, and that also international guidelines were affected by some papers of this duration.

As concerning the extent of the clinical impact reported at the step-up, we can only comment that it should be unbelievable if the original exacerbation and the hospitalization rates would have been higher in not-severe asthma patients, already treated with other therapies, as the patients of the study were.

Consequently, the achieved economic outcomes may appear not impressive at glance. Nevertheless, the significant difference registered in n. visits between the treatments should not be neglected because occurring over a 12-week period only, and this particular piece of evidence was still missing in literature, to our knowledge, when comparing these two drugs. Obviously, nobody would have expected highly dramatic changes in hospital admission and n. exacerbation rates in two sample of patients of this severity, who were stepped up for three months only. If occurring, it would have been regarded as a biased outcome, in our opinion.

The only simple message from the present paper is that some significant and unexpected differences in clinical outcomes are occurring when comparing the two treatments, even if within only 12 weeks, and in two small samples of subjects. The differences registered (such as in lung function and in reduction of unscheduled GPs’ and specialists’ visits) are in fact reflecting two interesting aspects of asthma governance which proved in favour of Fluticasone furoate/Vilanterol 92/22 μg once-a-day) rather than of Beclomethasone dipropionate/Formoterol 100/6 μg b.i.d., stepped up for a same period of time. Some suggestions for explaining this difference had been reported in the paper. Once again, this small piece of evidence was not available yet.

The authors of the letter also complain about the relevance of the results of a cost analysis, or better that our cost-analysis left disaggregated changes in cost and clinical outcomes for the decision-makers, instead of cooking for them a full cost-effectiveness analysis. This seems in contradiction with their other criticism of poor study design. We all well know that the cost-effectiveness is the most informative analysis among the health economics armamentarium, however we felt that the results of our study were just preliminary and as a consequence their use in a cost-effectiveness analysis would have been inappropriate. This again should reassure the reader that we are not drawing “strong conclusions” on our data.

Finally, just to reassure the Authors of the letter, we wish to inform that the survey had been originally planned for a 12-month duration, and that results reported in the present, recently published paper only represent the very first cluster of data. The overall results collected over one year are in advanced calculation, and their submission has been already planned for the next fall.