Undermining a cornerstone of medical research – examining a biased commentary on animal studies

Medical sociologist, Pandora Pound, and epidemiologist, Michael Bracken, recently wrote an opinion piece entitled “Is animal research sufficiently evidence based to be a cornerstone of biomedical research?” for the British Medical Journal. The article was chosen as the editor’s choice, leading to an editorial by the editor in chief, Fiona Godlee.

BMJ Pound and Bracken

Pound and Bracken criticise the poor quality and reporting of many animal studies, asserting that this is leading to ineffective drugs going on to clinical trials before failing.

Pound and Bracken make some suggestions for improvement, concluding:

In addition to intensifying the systematic review effort, providing training in experimental design and adhering to higher standards of research conduct and reporting, prospective registration of preclinical studies, and the public deposition of (both positive and negative) findings would be steps in the right direction. Greater public accountability might be provided by including lay people in some of the processes of preclinical research such as ethical review bodies and setting research priorities. However, if animal researchers continue to fail to conduct rigorous studies and synthesise and report them accurately, and if research conducted on animals continues to be unable to reasonably predict what can be expected in humans, the public’s continuing endorsement and funding of preclinical animal research seems misplaced.”

While some aspects of the article are reasonable, the overall impression the reader is left with is that animal research doesn’t work and can’t work in its current form. Their bias is obvious to those who are familiar with the arguments of those who argue against animal research. When they’re not incorrectly conflating basic science* with animal research (most basic biomedical research does not involve animals, e.g. human genetic research), Pound and Bracken argue that “lack of translation” is (apparently) not just from poor research practises, but also due to fundamental differences between humans and other animals, writing:

Even if the research was conducted faultlessly, animal models might still have limited success in predicting human responses to drugs and disease because of inherent inter-species differences in molecular and metabolic pathways.”

However, the bulk of the supporting literature they present to support this statement is – unlike most of the claims made in their commentary – not in the form of peer reviewed scientific research papers or meta-analyses but rather commentaries and books written by (other) opponents of animal research, including a certain Dr Greek whose misleading claims we have discussed several times on this blog (most recently here). For a commentary that sets great store by its evidence-based credentials this is, to say the least, disappointing.

Indeed, in their 2004 publication on whose anniversary this commentary was published, Pound, Bracken and their co-authors found that in all 5 cases where a therapy appeared to be successful in pre-clinical animal studies but later failed in human studies, more rigorous meta-analysis of the pooled pre-clinical animal studies showed that the treatment was not in fact successful in them, and that for one therapy (thrombolysis for stroke) such rigorous analysis would have enabled a serious side effect observed in clinical trials to be identified in the pre-clinical animal studies. In short, their own work shows that animal studies can predict the human outcome when their results are analyzed properly..

Other investigators who have examined failed therapies in cancer, ALS and stroke, have come to the same conclusion that too many therapies in some areas of research have failed in clinical trials not because of species differences, but because they never actually succeeded in animal studies, with most of the apparent successes being false-positive results due to flaws in experimental design and biases in reporting and publication. The authors all agree on a number of steps that need to be taken to avoid false-positive results being taken through to clinical trials, including better study design, requirement for independent replication of results in several animal models of the condition in question, publication of negative results (where the candidate therapy doesn’t work), meta-analyses of animal studies before beginning human trials.

An excellent analysis of animal models of stroke by van der Worp et al (2010) covers many of these issues, but also advises that to avoid false negative results in the clinical trials – where poor trial design leads to the erroneous conclusion that a therapy doesn’t work when in fact it does – human trials should match as closely as possible the conditions e.g. time to drug administration, dose, type of injury) of the successful animal studies.

The “rapid responses” to Pound and Bracken’s piece shows that many scientists who specialize in translating research from bench to bedside are alert to the flaws in their analysis.

To quote the response by Andrew Whitelaw and Marianne Thoresen, Professors of Neonatal Neuroscience at the University of Bristol:

The reader was left with impression that there were no examples in recent years of animal research leading directly to major advances in human health.

Three life-saving treatments in neonatal medicine would never have been given ethical approval for clinical trial if there had not been high quality animal models showing efficacy.

Rather than unselectively condemning the whole of biomedical animal research, we suggest that a more critical approach by funding bodies and journal editors could reduce bad research while supporting the good.

They ought to know, as basic and applied research in animals was crucial to the development of techniques that use cooling and xenon gas to protect babies from brain damage following oxygen starvation during birth.

Dr Thomas Wood, is more succinct:

[T]he overriding message of the article is somewhat confusing – demanding that we optimise and streamline animal research is very different from suggesting that it is useless, but both of these ideas are presented side-by-side.”

Prof Malcolm Macleod, a neurologist at the University of Edinburgh, and a frequent critic of poor design in some animal studies, agrees with many of Pound and Bracken’s criticisms, but in a more balanced manner, noting:

When conducted to the highest standards, animal research can indeed inform the development of human medicines. Given that there are many diseases for which we do now have treatments, it is perhaps self evident that the diseases which remain are more challenging, probably requiring research that is done to a higher standard – there is less signal, and more noise.”

Professor Macleod is one of Europe’s leading experts on the development of therapies for stroke, and is one of the leaders of the EuroHYP-1 trial of therapeutic hypothermia in adult patients with acute ischaemic stroke, a trial he advocated after undertaking a rigorous meta-analysis of studies on this therapy in animal models of ischaemic stroke.

Dr Charles M Pearman discussed how basic science makes up the building blocks that lead to human medicine:

Much clinical research is performed by standing on the shoulders of giants. A phase III drug trial comparing two antihypertensives will have much greater direct impact on clinical decision making than any individual animal model based basic science study. However, hundreds or thousands of such “low impact” works are needed to develop the drugs in questions. The authors reference Wooding et al. who themselves acknowledge this and conclude that clinically motivated basic biomedical research should be encouraged.

Basic biomedical research may try and may fail. Without it, however, there will be no successes to base clinical triumphs upon.

There have been many other comments, Prof Fernando Martins do Vale discusses why some of Pound and Bracken’s criticisms may not have much of an impact on results. Prof Robert Perlman argues that evolutionary differences between species can inform animal research. And Dr Vanitha A J explains that much cancer research has been effectively translated from animals to humans, noting in particular recent progress in cancer immunotherapy.

Another, separate, but strong response to Pound and Bracken’s paper was from Dr Liz Harley at Understanding Animal Research. Harley notes that many of the criticisms made in the original opinion piece are already being addressed by the industry. The UK Government’s delivery plan, “Working to Reduce the Use of Animals in Scientific Research”, explicitly mentioned the problems of poor experimental design and outlined several initiatives aimed to improve current practices. While Pound and Bracken call for a lay person to sit on ethical review bodies, they fail to note this is standard practice in the UK, while US regulations demand a lay person unaffiliated with the university stand on their Institutional Animal Care and Use Committees. Clearly Pound and Bracket do not do their homework sufficiently.

We finish with a quote from Prof Martins do Vale:

But the existence of bias and errors does not invalidate Science; on the contrary, as Karl Popper said, the awareness of errors is the first step for their correction and scientific progress.”

Pound and Bracken’s article opens up some important questions, but their biased interpretation risks throwing out the baby with the bathwater as they use flaws in experimental design to try and argue for a fundamental flaw in animal research. Their attempts to use legitimate concerns over experimental design to attack animal research are in fact a dangerous distraction from ongoing efforts to address problems that affect all areas of biomedical research (and indeed any areas of research where scientists have looked for them) from the most fundamental in vitro molecular biology studies right through to clinical trails.

Speaking of Research

* Confusion over what is meant by basic research is a theme throughout Pound and Bracken’s piece, it’s notable that many of the examples of “basic” research they mention are in fact applied or translational research, and that they focus on a paper on translation of basic research published by Contopoulos-Ioannidis et al. in 2003, a paper whose serious flaws in both design and conclusion we have discussed previously.

To learn more about the role of animal research in advancing human and veterinary medicine, and the threat posed to this progress by the animal rights lobby, follow us on Facebook or Twitter.

13 responses to “Undermining a cornerstone of medical research – examining a biased commentary on animal studies

  1. It’s going to be ending of mine day, except before finish I
    am reading this fantastic piece of writing to improve my knowledge.

  2. Do you have any video of that? I’d want to find out more details.

  3. One of the major problems is that scientists continue to use animal models that have been demonstrated to be worthless, and funding agencies continue to fund research based on animal models known to be worthless. When scientists will stop covering for each other and say what needs to be said about the flaws of their studies, the use of animals in research will improve considerably.

  4. James Keele

    More twisting of words to suit your argument. More excuses. Researchers are conducting experiments of such poor methodology that their results are meaningless. The study shows this. Animals are dying and suffering as a result, and all you have are vague appeals to the past?

    The fact is fundamental research is routinely justified on the basis that it is required to be able to initiate the research process into finding cures for diseases, and is therefore life saving. But you want to twist words to make a cheap point. Whatever.

    Poor methodology as the study shows can completely invalidate some research. To not follow good methodology results in at least some meaningless animal deaths and suffering. Agreed?

    • Pound and Bracken wrote an opinion piece – they did not conduct any studies. There is no sign that all results are meaningless (especially considering Pound and Bracken seem to repeatedly mix up translational and basic research).

      We continue to learn what good methodology is – it’s not a static concept and has been improving for centuries.

      I’m bored of the cheap comments and you will argue this wherever it goes. If you wish to continue – use the contact button: https://speakingofresearch.com/contact-us/

  5. James Keele

    Lets take this step by step since all you wish to engage in is rhetoric.

    If the justification for animal experimentation that involves some animal suffering is that it can save lives, then research that has no chance of saving lives because of poor methodology is merely animal cruelty. Agreed?

    • No. Because not all research is life-saving. In order to develop life-saving research you must conduct the fundamental research which underpins it – much of this is done without knowing it will lead to life-saving research (and as a result much doesn’t – but we don’t know which experiments will or will not lead to life-saving breakthroughs before we start).

      Now poor methodology harms research, but doesn’t necessarily invalidate it. Research without, say, randomization might be less solid than research with randomization, but that doesn’t mean you can’t learn from both. There is decades of important research that doesn’t follow every modern scientific methodology, but was still of huge value (Jenner’s initial human small-pox trials would fail most scientific tests of rigour – but saved countless lives).

      Science is a process – we learn how to improve it all the time. It’s not an instant process, but better methods propagate and worse methods are discarded. That is science.

      I recommend read this post:
      https://speakingofresearch.com/2012/02/03/pop-quiz/

  6. James Keele

    More excuses. More rhetoric. If someone doesn’t adhere to ARRIVE guidelines what are the consequences? How many papers are published that (as a %) actually adhere to the guidelines? Why are you not calling for those who don’t adhere to the guidelines to be prosecuted? You can make all the guidelines you want, but it doesn’t change the fact that animals continue to be abused because of poor methodology.

    Do you think that animals deserve the right not to be subject to cruelty? Do you think that those who commit cruelty to animals should be prosecuted? I would say if animal researchers value consciousness, and the ability to suffer, then they should also believe animals deserve some rights. If they believe that people who abuse animals should be punished, then they already believe animals have rights, whether they admit it or not.

    In the UK and the US, people are prosecuted for animal cruelty, so if you think researchers merely believe that subjecting animals to cruelty is merely something we ought to not to do rather than being required by law not to do, then they are completely out of step with reality.

    • “More excuses. More rhetoric”
      I’ve provided no excuses and used no rhetoric, so stop knocking down a straw man.

      ” If someone doesn’t adhere to ARRIVE guidelines what are the consequences? How many papers are published that (as a %) actually adhere to the guidelines? Why are you not calling for those who don’t adhere to the guidelines to be prosecuted? ”
      1) You actually don’t care about the answers to any of these questions because we “can make all the guidelines you want, but it doesn’t change the fact that animals continue to be abused because of poor methodology.”
      2) Nonetheless, if you don’t follow ARRIVE then you won’t be published in any journal that requires ARRIVE (http://www.nc3rs.org.uk/page.asp?id=1357). They are not the only guideline set aimed at improving research.
      3) Why should people be prosecuted for not following guidelines only required by certain journals? That makes no sense.

      In terms of animal rights. Read this for my views (plenty of other people have offered their views on SR as well):
      https://speakingofresearch.com/extremism-undone/ar-beliefs/

      “In the UK and the US, people are prosecuted for animal cruelty, so if you think researchers merely believe that subjecting animals to cruelty is merely something we ought to not to do rather than being required by law not to do, then they are completely out of step with reality.”

      What goes on in labs is not animal cruelty. Minimising suffering is a rule within labs, but that does not mean there is no suffering, because a no suffering rule would limit science in such a manner that people (and animals) would suffer as a result.
      The rules in labs and home are different, just as a doctor can do things in a surgery that would be illegal outside of it.

  7. James Keele

    “Their biased interpretation risks throwing out the baby with the bathwater as they use flaws in experimental design to try and argue for a fundamental flaw in animal research.”

    The fact is that these methodological inadequacies have been known about for years and nothing has been done to improve the situation. I think we should all ask ourselves why this this is. In my opinion scientists lack the will to really value the lives of the animals they experiment on. They live in a culture where killing animals for ones culinary enjoyment is accepted without a second thought, so why should they spare much time to think about the animals they kill as part of how they earn a living? Why challenge norms of poor methodology that results in animal cruelty for no reason when animals don’t really matter that much anyway? If an animal is killed in a poorly designed experiment that could never have produced useful results then surely this is no different from a random act of animal cruelty?

    Where are the researchers calling for those responsible to be held to account? Either the researchers themselves or the relevant licence holders are to blame, so where are the researchers calling for justice? All I read on this site are excuses.

    Until researchers start to actually campaign for animal rights more widely, I don’t see how anyone can trust them to make decisions about whether harming an animal in an experiment is justified by its potential outcomes or not.

    • Except we have been doing things about them. ARRIVE guidelines, and other similar publishing guidelines are a good example of a way in which animal research reporting has improved in order to put greater focus on things like randomization and blinding. The recent government report recommends ways in which some of those points can be addressed, and there has been a greater focus on the 3Rs in the UK with increased funding to the NC3Rs year-on-year.

      The difference with food is that there is no ethical review process before we kill a chicken, but a whole ethical review process whenever an animal is used (and afterwards humanely killed) in medical research.

      “Why challenge norms of poor methodology that results in animal cruelty for no reason when animals don’t really matter that much anyway?”

      Because bad methodology is bad science which doesn’t help your career or the science you care about.

      Why would researchers campaign for animal rights? Few believe in animal rights – many believe we have a responsibility towards them, and to minimise their suffering, but not necessarily that they have rights. And many, many scientists spend an awful lot of time working on animal welfare (which is a whole field of science). Your ethical review committees work every day “campaigning” for improved animal welfare in individual experiments – it’s built into the system.

  8. I find it very telling that the BMJ refuses to print research conducted in animals, but periodically and gleefully prints studies that condemn it.

  9. On a related topic, we recently discussed the Kimmerman 2014 paper on exploratory vs confirmatory preclinical studies – got myself and my colleagues a bit hot under the collar as it seems to be undermining attempts to show animal research is being conducted in a respectful and rigorous manner – would love it if you had a look at the blog I’ve just done on the topic! http://wp.me/p2zisd-ee