Undermining a cornerstone of medical research – examining a biased commentary on animal studies

Medical sociologist, Pandora Pound, and epidemiologist, Michael Bracken, recently wrote an opinion piece entitled “Is animal research sufficiently evidence based to be a cornerstone of biomedical research?” for the British Medical Journal. The article was chosen as the editor’s choice, leading to an editorial by the editor in chief, Fiona Godlee.

Pound and Bracken criticise the poor quality and reporting of many animal studies, asserting that this is leading to ineffective drugs going on to clinical trials before failing.

Pound and Bracken make some suggestions for improvement, concluding:

In addition to intensifying the systematic review effort, providing training in experimental design and adhering to higher standards of research conduct and reporting, prospective registration of preclinical studies, and the public deposition of (both positive and negative) findings would be steps in the right direction. Greater public accountability might be provided by including lay people in some of the processes of preclinical research such as ethical review bodies and setting research priorities. However, if animal researchers continue to fail to conduct rigorous studies and synthesise and report them accurately, and if research conducted on animals continues to be unable to reasonably predict what can be expected in humans, the public’s continuing endorsement and funding of preclinical animal research seems misplaced.”

While some aspects of the article are reasonable, the overall impression the reader is left with is that animal research doesn’t work and can’t work in its current form. Their bias is obvious to those who are familiar with the arguments of those who argue against animal research. When they’re not incorrectly conflating basic science* with animal research (most basic biomedical research does not involve animals, e.g. human genetic research), Pound and Bracken argue that “lack of translation” is (apparently) not just from poor research practises, but also due to fundamental differences between humans and other animals, writing:

Even if the research was conducted faultlessly, animal models might still have limited success in predicting human responses to drugs and disease because of inherent inter-species differences in molecular and metabolic pathways.”

However, the bulk of the supporting literature they present to support this statement is – unlike most of the claims made in their commentary – not in the form of peer reviewed scientific research papers or meta-analyses but rather commentaries and books written by (other) opponents of animal research, including a certain Dr Greek whose misleading claims we have discussed several times on this blog (most recently here). For a commentary that sets great store by its evidence-based credentials this is, to say the least, disappointing.

Indeed, in their 2004 publication on whose anniversary this commentary was published, Pound, Bracken and their co-authors found that in all 5 cases where a therapy appeared to be successful in pre-clinical animal studies but later failed in human studies, more rigorous meta-analysis of the pooled pre-clinical animal studies showed that the treatment was not in fact successful in them, and that for one therapy (thrombolysis for stroke) such rigorous analysis would have enabled a serious side effect observed in clinical trials to be identified in the pre-clinical animal studies. In short, their own work shows that animal studies can predict the human outcome when their results are analyzed properly..

Other investigators who have examined failed therapies in cancer, ALS and stroke, have come to the same conclusion that too many therapies in some areas of research have failed in clinical trials not because of species differences, but because they never actually succeeded in animal studies, with most of the apparent successes being false-positive results due to flaws in experimental design and biases in reporting and publication. The authors all agree on a number of steps that need to be taken to avoid false-positive results being taken through to clinical trials, including better study design, requirement for independent replication of results in several animal models of the condition in question, publication of negative results (where the candidate therapy doesn’t work), meta-analyses of animal studies before beginning human trials.

An excellent analysis of animal models of stroke by van der Worp et al (2010) covers many of these issues, but also advises that to avoid false negative results in the clinical trials – where poor trial design leads to the erroneous conclusion that a therapy doesn’t work when in fact it does – human trials should match as closely as possible the conditions e.g. time to drug administration, dose, type of injury) of the successful animal studies.

The “rapid responses” to Pound and Bracken’s piece shows that many scientists who specialize in translating research from bench to bedside are alert to the flaws in their analysis.

To quote the response by Andrew Whitelaw and Marianne Thoresen, Professors of Neonatal Neuroscience at the University of Bristol:

The reader was left with impression that there were no examples in recent years of animal research leading directly to major advances in human health.

Three life-saving treatments in neonatal medicine would never have been given ethical approval for clinical trial if there had not been high quality animal models showing efficacy.

Rather than unselectively condemning the whole of biomedical animal research, we suggest that a more critical approach by funding bodies and journal editors could reduce bad research while supporting the good.

They ought to know, as basic and applied research in animals was crucial to the development of techniques that use cooling and xenon gas to protect babies from brain damage following oxygen starvation during birth.

Dr Thomas Wood, is more succinct:

[T]he overriding message of the article is somewhat confusing – demanding that we optimise and streamline animal research is very different from suggesting that it is useless, but both of these ideas are presented side-by-side.”

Prof Malcolm Macleod, a neurologist at the University of Edinburgh, and a frequent critic of poor design in some animal studies, agrees with many of Pound and Bracken’s criticisms, but in a more balanced manner, noting:

When conducted to the highest standards, animal research can indeed inform the development of human medicines. Given that there are many diseases for which we do now have treatments, it is perhaps self evident that the diseases which remain are more challenging, probably requiring research that is done to a higher standard – there is less signal, and more noise.”

Professor Macleod is one of Europe’s leading experts on the development of therapies for stroke, and is one of the leaders of the EuroHYP-1 trial of therapeutic hypothermia in adult patients with acute ischaemic stroke, a trial he advocated after undertaking a rigorous meta-analysis of studies on this therapy in animal models of ischaemic stroke.

Dr Charles M Pearman discussed how basic science makes up the building blocks that lead to human medicine:

Much clinical research is performed by standing on the shoulders of giants. A phase III drug trial comparing two antihypertensives will have much greater direct impact on clinical decision making than any individual animal model based basic science study. However, hundreds or thousands of such “low impact” works are needed to develop the drugs in questions. The authors reference Wooding et al. who themselves acknowledge this and conclude that clinically motivated basic biomedical research should be encouraged.

Basic biomedical research may try and may fail. Without it, however, there will be no successes to base clinical triumphs upon.

There have been many other comments, Prof Fernando Martins do Vale discusses why some of Pound and Bracken’s criticisms may not have much of an impact on results. Prof Robert Perlman argues that evolutionary differences between species can inform animal research. And Dr Vanitha A J explains that much cancer research has been effectively translated from animals to humans, noting in particular recent progress in cancer immunotherapy.

Another, separate, but strong response to Pound and Bracken’s paper was from Dr Liz Harley at Understanding Animal Research. Harley notes that many of the criticisms made in the original opinion piece are already being addressed by the industry. The UK Government’s delivery plan, “Working to Reduce the Use of Animals in Scientific Research”, explicitly mentioned the problems of poor experimental design and outlined several initiatives aimed to improve current practices. While Pound and Bracken call for a lay person to sit on ethical review bodies, they fail to note this is standard practice in the UK, while US regulations demand a lay person unaffiliated with the university stand on their Institutional Animal Care and Use Committees. Clearly Pound and Bracket do not do their homework sufficiently.

We finish with a quote from Prof Martins do Vale:

But the existence of bias and errors does not invalidate Science; on the contrary, as Karl Popper said, the awareness of errors is the first step for their correction and scientific progress.”

Pound and Bracken’s article opens up some important questions, but their biased interpretation risks throwing out the baby with the bathwater as they use flaws in experimental design to try and argue for a fundamental flaw in animal research. Their attempts to use legitimate concerns over experimental design to attack animal research are in fact a dangerous distraction from ongoing efforts to address problems that affect all areas of biomedical research (and indeed any areas of research where scientists have looked for them) from the most fundamental in vitro molecular biology studies right through to clinical trails.

Speaking of Research

* Confusion over what is meant by basic research is a theme throughout Pound and Bracken’s piece, it’s notable that many of the examples of “basic” research they mention are in fact applied or translational research, and that they focus on a paper on translation of basic research published by Contopoulos-Ioannidis et al. in 2003, a paper whose serious flaws in both design and conclusion we have discussed previously.

To learn more about the role of animal research in advancing human and veterinary medicine, and the threat posed to this progress by the animal rights lobby, follow us on Facebook or Twitter.