This is an updated version of an article that was first published in July 2021
A recent peer reviewed meta-analysis evaluating ivermectin (Bryant et al) concluded that this cheap antiparasitic drug is an effective treatment for reducing Covid-19 deaths. These conclusions were in stark contrast to those of a later study (Roman et al). Although (Roman et al) applied the same classical statistical approach to meta-analysis, and produced similar results based on a subset of the same randomized controlled trials data used by (Bryant et al), they claimed there was insufficient quality of evidence to support the conclusion Ivermectin was effective. But their conclusion is based on a subjective (and possibly biased) assessment ot the 'quality' of the trials; moreover, they wrongly concluded 'no effect' from what was merely weaker evidence of a positive effect.
Those who are unfamiliar with the way statistics are used to analyse effectiveness in such studies would be very confused by the contradictory conclusions if they looked at the quantified results presented. These are summarised by what is called the risk ratio (RR). The RR is an estimate of the death rate of patients taking ivermectin divided by the death rate of patients not taking ivermectin. For example, if we knew for sure that 2% of patients taking ivermectin died compared to 4% not taking ivermectin then the RR would be 0.5. If the RR is clearly less than one then it is reasonable to conclude the treatment is effective.
Bryant and Roman provide very similar estimates for the RR: Bryant reports 0.38, while Roman reports 0.37, which seems to mean both agree that ivermectin is effective. However, because the statistics can only provide uncertain estimates of the true death rates, the RR is also presented with upper and lower “confidence interval” bounds – typically 95%.
The Roman study uses fewer data and (as is common in such situations) arrives at wider confidence bounds: 0.12 to 1.13 compared to bounds of 0.19 to 0.73 in Bryant.
Most people assume this means there is a 95% chance the RR lies between the reported upper and lower bounds. But it does not. It relies on a complex notion of what would be observed in multiple theoretical repeated trials.
In classical statistical hypothesis testing, if the upper 95% confidence interval bound is greater than 1, the hypothesis that the RR is greater than 1 “cannot be rejected with sufficient confidence”.
A new analysis applies an alternative to the classical approach - namely a Bayesian approach - to a subset of the same trial data used in the studies (a summary version of this analysis is to appear in the September issue of American Journal of Therapeutics). It tests several causal hypotheses linking Covid-19 severity and ivermectin to mortality. Applying diverse alternative analysis methods which reach the same conclusions should increase overall confidence in the result.
The paper show that there is strong evidence to support a causal link between ivermectin, Covid-19 severity and mortality, and:
i) for severe Covid-19 there is a 90.7% probability the risk ratio favours ivermectin;
ii) for mild/moderate Covid-19 there is an 84.1% probability the risk ratio favours ivermectin.
Also, from the Bayesian meta-analysis for patients with severe Covid-19, the mean probability of death without ivermectin treatment is 22.9%, whilst with the application of ivermectin treatment it is 11.7%.
Since the first version of the Bayesian analysis was reported in early July, some concerns have been raised about the veracity of some of the studies, notably that of Elgazzar. While some have noted that these concerns may be based on Western elitism (the studies criticized all come from Africa and Asia), the revised version of the paper nevertheless addresses the concerns. Specifically, it evaluates the sensitivity of the conclusions to any single study by removing one study at a time. In the worst case, where Elgazzar is removed, the results remain robust, for both severe and mild to moderate Covid-19.
The paper also highlights advantages of using Bayesian methods over classical statistical methods for meta-analysis. But it should be noted that all studies included in the analysis were prior to data on the delta variant.
18 August UPDATED paper: Martin Neil and Norman Fenton (2021) "Bayesian Hypothesis testing and hierarchical modelling of Ivermectin Effectiveness in Treating Covid-19 Disease http://dx.doi.org/10.13140/RG.2.2.19703.75680
Original paper: Martin Neil and Norman Fenton (2021) "Bayesian Meta Analysis of Ivermectin Effectiveness in Treating Covid-19 Disease" https://doi.org/10.13140/RG.2.2.31800.88323
Is this meta-analysis going to be undated taking into account subsequent large well conducted placebo controlled trials showing minimal efficacy for ivermectin?
The problem with your Bayesian analysis is that some of the data analysed is probably fraudulent. However good the math is if the data being analysed is garbage then the conclusions you come to are garbage. Perhaps the best conclusion is that engineers and mathematicians give poor quality advice on treating life threatening infections. Experts on analysing medical trials, such as the Cochrane collaborative have very different conclusions on the evidence of benefit of ivermectin for covid 19. High quality RCT of ivermectin consistently show little or no benefit.