Wednesday, February 25, 2009
The Significance of Insignificant Results
What happens when a scientific experiment results in failure to falsify the null hypothesis? Typically negative results get filed away in a cabinet, never to be published or shared with the rest of the research committee. But what is the cost to science of this system in terms of duplicated effort? The article "Null and Void" explores some of these issues, mostly in the biological sciences. What are some of the suggested solutions posed by the author, and do you think this issue has any relevance for anthropologists?
Subscribe to:
Post Comments (Atom)
I agree about the issue concerning repeating previously failed testing that went unpublished, especially when it involves animal or even human testing. It does give a biased and skewed view if we are only getting information that people want us to see. I think that a site where scientists can briefly explain their results would be help, like the article stated. The problem here is how to remove the stigma of negative results. It seems like scientists want the discarded data, but are shy about admitting their failure. But I don’t think negative results are failures. We need to see negative results just merely as what they are: Not the answer we were looking for.
ReplyDeleteEdison once said he didn't fail 1000 times to invent the light bulb he discovered 1000 ways not to invent the light bulb. If someone was reading about the thousand ways not to do it I bet they would have found the right way to do it first.
ReplyDeleteI found it interesting that there is an actual Journal of Negative Results. I can see why some people don't want to be published in this journal and be labled a failure. However, isn't this a text book example of why these scientists should use an internet forum to out their bad results? This way the next researcher won't have to pan through the same bad associations. Maybe I missed the point of this article.
ReplyDeleteI for one am all for publishing negative results. I feel like it would enhance the quality of the positive results to come out of all the fields of anthropology. As the article states the high cost of re-doing failed experiments is crippling many scientific fields. I think its high-time that scientists own up to their failings while learning not to take them personally.
ReplyDeleteLet's be clear in our use of language...failing to falsify a null hypothesis is not really the same as the vernacular meaning of "to fail". One can successfully test the efficacy of a certain drug against a disease with 2 possible outcomes: either it has a statistically significant relationship (i.e., it makes the individual healthier or sicker), or it has no effect on the individual's health. The latter case involves the "failure" to falsify the null hypothesis of no effect of the drug on the disease, but it was a successful experiment that tells us something useful, namely that this drug does not cure this disease. This is valuable information, and we shouldn't label the experiment a "failure".
ReplyDeleteI think Anemone's last line is the most important line of reasoning for us to pull from this topic. Valuable information is valuable information all day long. The reasoning in the article that "no one can read 150 papers and remember what they read" completely misses the point. If I am doing research I do not need to read all those papers, it would just be helpful to search a journal or a database for keywords on articles specific to my project. By having access to results of similar research I would be able to change my project according to previous results, therefore avoiding a tremendous waste of time and funding.
ReplyDeleteI agree with bob, by pointing out the difference between failing and faliure to folsify the null hypothesis, you began to understand more about the research. Just becasue the data collected is a "failure" doesn't mean it cannot be useful to others, even if all it does is stop the experiement from being done twice.
ReplyDeleteNo one really wants to admit that they were wrong, so they won't publish negative results. I don't think that negative results are necessarily "failures", because you performed the experiment and got results, anyway. The only thing is that they aren't exactly what you wanted. But that doesn't mean that these results will negatively impact scientific research. Remember, some of the great scientific discoveries were by accident. Look at peanut butter! Just because negative results don't benefit your research, they may benefit someone's research.
ReplyDeleteJason, refresh our memories about the discovery of peanut butter (and its relevance to the issues at hand).
ReplyDelete