Daily Archives: February 3, 2012

Pop Quiz!

Take out a piece of paper and a sharpened #2 pencil.

Please read carefully the following story and answer all the questions.

You have 15 min.

One Saturday morning Dr. X was walking her dog thinking about some recent results in her field when it dawned on her that she might actually have the key to explaining all those findings.  If she was correct, she could go on to develop a new therapy for a terrible disease.

Being a scientist, Dr. X rapidly turned that idea into a specific hypothesis with testable predictions.  She ran back to her laboratory, gathered her students, told them the idea, and got to work.  They were excited when their first test (T1) yielded a positive result.  This simply meant that the implications of her hypothesis were corroborated by the experiment.  Good job everyone!

The next day her students were up all night running the second test (T2).  Dr. X arrived at the laboratory after dropping her kids in school to find very tired students, but with big smiles on their faces.  The second test, she correctly guessed, gave them another positive result.  Hurrah!

That night, at the dinner table, she shared the excitement with her family. Even the dog appeared to notice something important was going on. Next morning, one of her postdoctoral students came up with, what appeared to be, a direct test of the central idea.  It was agreed at the Lab meeting that this would be the next experiment (T3).

It was a difficult experiment.  Dr. X’s husband agreed to pick up the kids instead and let her finish her work.  Close to midnight the results came in.  Everyone in the lab ran to see the results.   They stared at each other in disappointment.  The result was clearly negative — what this meant is that the outcome contradicted a key prediction of the hypothesis.

Dr. X’s Lab had a difficult month.  They went over the data over and over again — nothing was obviously wrong; but they decided not to give up.  Instead, they brainstormed about how they could come up with a new hypothesis that may explain the data they had collected so far.  And yes, Dr. X explained, this must include a reason for the outcome of the negative experiment as well.

One night, Dr. X was awoken by the sound of the phone. She was startled, it was unusual that anyone would call at 3 am to her home. Understandably, Dr. X answered the phone with some apprehension.  She was relieved to hear one of her students, which after calming himself down and apologizing for the time, described to her a new idea that, he said, came to him out of nowhere in the middle of his sleep.  She grumbles, but listened…  her sleepy eyes slowly widening as the student went on.  When he was done Dr. X immediately knew that there was no doubt her student could explain the diverse findings.

Everyone gathered in the laboratory next morning and started to test again based on the new concept over the week.  T4… positive!  T5….positive!  T6… negative…  Negative?!  Oh no…  Again?!

Yes, again.  But Dr. X gathered her students and explain to them that this is how science works.  New ideas emerge from old ones in an effort to account for all the data their community gathered so far.  And that negative findings were important for science too. They all felt a bit better as they went home… just a little bit.  But more than Dr. X’s words, it was a group feeling that they were getting closer to the truth.

It took her Lab a few more iterations of this difficult game called science, but one day they knew they had nailed it.  They had a new idea that not only explained all past results but stood many additional tests, including replications by her colleagues.  Their work delivered a medical breakthrough that allowed them to develop a new medical treatment that saved uncountable human lives.

Questions:

Assume that in this story, from beginning to end, including her experiments those of her colleagues, scientists performed 20 experimental tests that yielded positive results, 15 experimental tests that yielded negative results, and that each test required the use of exactly one mouse.

Q1. How many mice were scientifically necessary to develop this medical breakthrough?

Q2. Which experimental tests were more important in developing this breakthrough?  The tests yielding positive results or the ones yielding negative results?  Explain.

Q3. Given the end result was that uncountable human lives are being saved.  Which test was morally justifiable and which was not?  Were positive tests in any way more justifiable than negative ones?  Were experiments used in replicating Dr. X’s findings necessary and justified?  Or is it only the final experiment directly preceding the development of the new therapy that was justified?

Q4. Five years after her discovery, and with the new knowledge acquired, one of Dr. X’s colleagues comments that it was obvious some of the ideas she had tried could not have worked.  With 20/20 vision, Dr. X agrees.  Does her admission mean the experiments testing those ideas were scientifically unnecessary or ethically indefensible?

Submit your answers in the comments section below!