Guest Post: Predictability and Utility of Animal Models

This is a guest post on the utility of animal models in drug development, misconceptions about animal models, and alternative methods of drug development, by Dale M. Cooper, DVM, MS, Diplomate, American College of Laboratory Animal Medicine. Dr. Cooper has over 20 years of veterinary experience in private practice, and as a laboratory animal veterinarian in academic and pharmaceutical research.  He is committed to the welfare of research animals.

Part I  Predictability of Animal Models for Effects of Drugs in Humans

Mark Twain popularized an aphorism that “there are three kinds of lies: lies, damn lies, and statistics.”  While all of these strategies have been employed by animal activist groups to discredit animal based research, the misuse of statistics has the most significant impact, as the most believable lies have a kernel of truth to them. Such is the case with the intense efforts by animal activist groups to discredit the use of dogs and other animal models in biomedical research. The various statistics that are cited are that only 0.0002% of studies using animals result in an approved drug, over 90% of drugs shown to be safe or effective in animals fail to make it through human clinical trials, that animal models have a poor predictive value for the effects of drugs in humans, and that the a flip of a coin gives as good of a chance of predicting success in drug development as an animal model. Like all good lies, these statements have a kernel of truth, but are extremely misleading and demonstrate a significant lack of understanding of science and a general lack of critical thinking skills.

Graphs like this are misused to suggest that animal studies result in failures in human trials.

Experts in drug development understand the limitations of animal models, but they also understand their applications. There have been several publications that have retrospectively evaluated the value of animal models in predicting human safety across a variety of therapeutic areas and the overall percentage of human toxicities predicted by animal models is around 70% with variability between different species and body systems.  Predictability for some therapeutic areas are over 90%. When different models are used in combination, the predictability increases. It is also an established fact that only about 1 in 10,000 drugs tested make it to the market and that there is over a 90% attrition rate of drugs in human clinical trials.  How does this happen if animal models are predictive?

Studies that have evaluated the ability of animal models to predict clinical results in humans show very similar results, but the interpretation of the results is varied.  Authors who are acknowledged animal activists claim the results show poor correlation between results in animals and outcomes in humans (e.g ‘no better than a flip of a coin’). However, most scientists (and the FDA and NIH) assert that animal models predict outcomes in humans with good reliability. What is the disconnect? This is where statistics come in. The different papers argue over the use of a calculation for predictive value versus likelihood ratio. The results come out slightly different.  Predictive value calculations show better results for animal models than do likelihood ratio calculations, so animal activists tend to cite the likelihood ratios while overlooking the predictive ratios (see table below). I am not a statistician and therefore won’t weigh in on this point and will use the term ‘predictive value’ to refer to both terms.

What I feel is a far more relevant discussion is how we interpret positive versus negative predictive values.  In general, the positive predictive values of animal models are higher than the negative predictive values. What this means is that the presence of an effect in an animal model is a good indicator that the same effect will be seen in humans (positive predictive value).  However, if an effect is not seen in the animal that does not mean there won’t be an effect in humans (negative predictive value). Activists have focused on the negative predictive value, taking the position that because animal models don’t predict all effects in humans, they are not reliable and therefore, the use of animals in research is not scientifically justified. Is this a valid conclusion?  Let’s apply it to another risk assessment situation and see if this makes sense. Say I want to cross a road but don’t want to get hit by a car. My Mom taught me to look both ways and if I see a car coming I don’t cross the road. There is a positive predictive value to look before I cross. However, if I don’t see a car, that doesn’t always mean one isn’t coming. Depending on its speed or visibility there is still some risk when I cross a road. The negative predictive value of looking both ways isn’t as high as the positive predictive value. So do I bother to look both ways knowing it’s not 100% reliable?  Of course I do.  But I also do other things. I listen, I assess for visibility, I may look for a crosswalk or an overpass. Just like in drug development I understand the predictive value of my risk assessment and run more than one assessment.

The ignored statistics in Bailey’s papers on the prediction rates of animal research. PPV: positive predictive value, NPV: negative predictive value.

Unlike animal activists, biomedical scientists don’t have an agenda to limit the scope of research. We use the experimental systems that allow us to address the questions at hand to develop treatments that improve the lives of both humans and animals.  The models we are using are the ones that are the most successful. Animals are one type of model employed in biomedical research, but are by no means the only models.  Animal models are expensive and time consuming, and scientists recognize the emotional and ethical issues associated with animal research.  We are human and many of us have our own pets.  We bond with the animals we work with.  There is no incentive for us to employ an animal model that may negatively impact animal well-being if another model that does not negatively impact well-being works just as well.  The FDA requires animal data prior to clinical trials in humans, because they are also scientists and have come to the same conclusion — animal models provide essential data in predicting safety and efficacy of new therapies.

Part II Drug Development Without Animals

As discussed in a previous post, animal models are an important component of the development process for drugs and other medical treatments. But they are not the only method of research that is used.  Drug development is an iterative process.  Each study builds on data from other studies.  The process involves computer modeling, benchtop chemistry, a wide range of in vitro models to evaluate absorption, metabolism, distribution, receptor binding, gene expression, and even some aspects of toxicity. We use non-animal methods so much that over 90% of drug candidates are eliminated from consideration using in vitro assays before they even reach the phase of pre-clinical animal studies.

If animal activists groups or well-meaning scientists want to see more non-animal research methods developed and put into use, there is a process for this. The Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM) under the National Toxicology Program evaluates data to validate non-animal models for drug development in the US. There is a list of approved methods on their website. All of these methods were funded by the same research institutions that use animal models.  If activist groups wanted to make a significant impact on animal use in research, they might consider funding alternative research.  It is usually more effective to work on a problem rather than just talk about it.  The fact that we are still using animals isn’t because of a lack of trying, it is the limitation in our scientific knowledge. To build a model, you have to know a lot about the system you are modeling. In fact, it is possible there will never be a complete replacement of animal models in research. By the time we know everything to create the perfect model, we will have answered all of the research questions that can be asked.

The final stage of drug development involves clinical testing in human patients.  In vivo studies in animal models eliminate 90% of drug candidates from the development process before beginning clinical trials in humans, demonstrating their utility in de-risking the testing process in humans.. Over 1 million patients to enroll in clinical trials each year, yet there are few reports of serious adverse events relative to the number of patients in these trials.  When drugs fail during the clinical trial phase, the reasons are more often related to economics or strategy than they are to safety.  The testing in animals served an important purpose.

If society chooses to take more risk in the patient population, it has the power to do so. However, based on the data regarding predictive value of non animal models, this would mean that 70% of the time someone in a clinical trial would be likely to experience toxicity from a drug about which little is known because the nature of it was not first characterized in animal models. This means the physicians would not know how to treat it or the prognosis. It is already a challenge to enroll the number of patients needed for clinical trials even when providing them a significant amount of information so they can make an informed decision to consent to enroll. Having even less information would not likely help with this.

Laboratory mouse.

It is also not clear that humans are a better model for testing drugs than are animal models. It is extremely difficult to control variability in a human test population, due to diet, lifestyle, and genetics, which reduces the statistical power of a given study population compared to a well-controlled animal study. Clinical trials in humans enroll thousands of patients, whereas animal studies use fewer than 100 animals in many studies to achieve similar statistical power.  Humans also have a long lifespan and studying the chronic effects of a drug is difficult in a clinical trial.  In contrast, in 2 years, a rodent undergoes all life stages, allowing assessment of the effects of chronic drug administration. Finally, it would be very unlikely that humans would consent to participate in studies evaluating fetal toxicity, and even if they did, the long duration of human pregnancy and low reproductive rate (1 offspring every 9 months) reduces the power of detection relative to a rodent model that produces 10 offspring in 3 weeks.

It is true that all of these issues will be considerations in patients after a drug is in general use. There is some inherent level of risk in medicine. However, the development and approval process using both animal and non-animal studies is the best that science can currently offer.

Part III The Big Picture of Animal Use

We can argue over the scientific merits of the use of laboratory vs animal vs human testing in drug development, and we can pontificate about the ethics of the decisions we make, but ultimately, we humans have choices to make.  Are we content with our level of health and well-being?  What sacrifices are we willing to make to consider the needs of animals?  The impacts we have on animals go far beyond what level of medical care we choose.  Animals serve as food sources, they work for and with us, and they provide for aesthetics and companionship.  How much of that will we give up to reduce our impacts on them?  If we go that far, we are still competing with them for food and shelter.  Is it possible to not impact animals?  I contend that the ethics of our society show a considerable level of care for animals.  The fact that we worry at all about their welfare is something that to the best of my knowledge, no other species on the planet would do given the same choices we have.

Working with animals in research is not a one-way street.  Animals benefit from veterinary treatments developed through the same research process as for human treatments, and the animals we work with in the research environment benefit from the high level of care and attention to their well-being that is provided to them. People care about them and for them. They experience medicine as part of their lives as do we all. If they were living outside of the research environment they would still experience medical issues as a normal part of life, but particularly in the case of non-companion species, would not necessarily have anyone to care for them.

I believe that the arguments proffered to discredit work with animals in research are largely based on biased and misleading interpretation of data.  Where there are valid data to use alternatives, these alternatives are already being used.  It is appropriate scientifically and ethically to continue to develop and validate new approaches for predicting drug safety and efficacy in the patient population, both animal and non-animal.  Animals are used humanely and also receive benefits from biomedical research.  Ultimately, there is a balance being struck between the needs of humans and of animals.  There is room for constructive dialog on where this balance should be, but I personally do not believe that this should occur using the regulatory and legal systems as a venue.  Science is too intricate and complex to be able to effectively address in this way.  There is a process in place at all research institutions (the IACUC) to ensure ethical and scientific review occurs on each experiment.  Those seeking to drive alternatives would do better to develop the science to validate these alternatives rather than manipulate public emotion and ultimately public policy or law.

~Dale M. Cooper, DVM, MS, Diplomate, American College of Laboratory Animal Medicine.

 

 

11 responses to “Guest Post: Predictability and Utility of Animal Models

  1. A few things stick out to me. The author seems to use the good ol USDA definition of “animal” when he says “animal studies use fewer than 100 animals in many studies to achieve similar statistical power” Mice are not under that definition of “animal” and even IF that was true, and less than 100 mice were used in many studies, less than 100 mice are used for EVERY study, every tiny study adds up to millions, maybe more (no one can say since mice are not regulated) animals used to finally come up with a drug or vaccine by the time all the research is said and done (and for a 70% rate).

    “There is no incentive for us to employ an animal model that may negatively impact animal well-being if another model that does not negatively impact well-being works just as well.” No, that is not true because then he states: “The FDA requires animal data prior to clinical trials in humans, because they are also scientists and have come to the same conclusion” No, the FDA requires all animal data (a rodent and a non-rodent usually a dog, or NHP) because the FDA is outdated and anyone to say differently is labeled an “animal activist” as this author consistently does.

    The funniest part: “It is also not clear that humans are a better model for testing drugs than are animal models. It is extremely difficult to control variability in a human test population”, There are so many issues with that statement I don’t even know where to begin. If it is not clear that humans are a better model, than why is it necessary that there are always clinical trials before the completion of a drug!? Plus, like it or not, the drug or vaccine or whatever, will inevitably go into these humans with variables that are uncontrollable. Which is why animal models can be so unpredictable, we take these animals with very controlled variables when the end goal is to use the same thing for a human with very little control over variables (including genes)! It is asinine if you think about it.

    • Yes, he is talking about all animals – including mice and rats (and fish and birds). There are generally not going to be hundreds of trials of a compound before it moves into humans (there may be hundreds of studies involved in the knowledge behind the drug and the understanding of the disease). In short, millions of animals are not used per drug (for evidence, note the UK uses around 500,000 animals in regulatory studies each year (inc. mice, rats, birds, fish) and produce many drugs each year.

      FDA mandated studies are just part of the drug discovery process – there are many other non-mandated studies, which would not be done if there was a better method.

      Finally, I don’t think anyone debates that human studies are essential, but there are some things which they are poorly suited to. You can’t do controlled studies of, say, teratogencity, on humans, because it would be unethical. You need to use a number of animals under controlled conditions to see what happens.

  2. John Pippin, MD

    There is much falsehood and selective interpretation in this article, and I believe that just like “animal activists” Dr. Cooper has an agenda. As an aside, it is not true that most drugs fail clinical trials because of commercial reasons. The main reasons drugs fail clinical trials are (1) lack of efficacy, and (2) toxicities undetected or substantially worse than seen in animal studies. The correct stats for clinical trial attrition of drugs that receive an IND are up to 56% for phase 1, 82% for phase 2, and 50% for phase 3…a cumulative 96% failure rate. [Arrowsmith; Lovell-Badge]

    • You point out attrition rates – they clearly show drugs passing early human trials and then failing late human trials – yet you blame this on animal studies? By your logic you may as well blame the pre-clinical non-animal tests that come first for every failure.

      • John Pippin, MD

        The issue, Mr. Editor, is that the animal preclinical studies are unreliable and cannot predict which 4-5% of successful animal-tested drugs will gain FDA approval. So the question is: Why go through this when it is so unreliable?

        • The role of animal studies is not to predict which drugs will gain FDA approval (any more than non-animal pre-clinical tests are). They exist to find promising drug candidates and, most importantly, ensure those candidates are safe enough to enter human trials). The rarity of Phase I clinical trial disasters should be evidence of its effectiveness in this arena.

          • John Pippin, MD

            Disagree. The litany of false positive and false negative preclinical animal studies of drugs belies your claim. Whether we look at drugs that “passed” animal testing and then failed in clinical trials, or at drugs that “failed” preclinical drug testing and turned out to be good drugs, those animal studies were not reliable for identifying safe and effective drugs. You claim that the main reason for animal testing is to identify safe drugs, but toxicity undetected in animal studies is the second most common reason clinical drug trials fail…after lack of efficacy.

          • The PPV and NPV data is there for you to see in the post.

            Efficacy becomes more important in later stage clinical trials. At Phase I, safety remains the primary concern. You have consistently failed to:
            (a) Explain why you are blaming animal studies but not preclinical non-animal models that are used
            (b) Explain why late stage failures of drugs are the fault of animal studies but not the earlier human trials which they passed

          • John, you should probably also note your position in the antivivisection group, PCRM, where you’ve worked since 2005.

          • John Pippin, MD

            I’m proud of that. I came to it honestly, as a preclinical animal researcher who saw through the futility of my work and that of colleagues. How about your position as a cheerleader for animal research? We have polar opposite positions…doesn’t mean we’re bad guys, just different viewpoints.

  3. Juan Carlos Marvizon

    What a great article! I hope a lot of people take the time to read it, because it contains lots of valuable information.

    I would like to emphasize a point that is often overlooked: “It is also an established fact that only about 1 in 10,000 drugs tested make it to the market.” And: “When drugs fail during the clinical trial phase, the reasons are more often related to economics or strategy than they are to safety.”

    We often forget that drug development takes place in the context of fierce capitalist competition and that drug companies were created to make a profit, no to increase human health. In our litigious society, many drugs are withdrawn from clinical trials because the drug company perceives a liability or realizes that it won’t make much of a profit if sold. This has nothing to do with whether the drug is effective or safe. Animal models cannot predict how profitable a drug would be, so if should not be blamed if a drug doesn’t make it to the market, as opposed to being safe and efficacious.

Join the conversation

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s