My Actual Homepage - Go here for more info.


I plan to put a graphical banner here eventually...

16 November 2008

Extraordinary Claims Require Extraordinary Evidence

So said Carl Sagan. But why? And what is an extraordinary claim?

The origins of the saying can perhaps be found in Hume’s Maxim:

No testimony is sufficient to establish a miracle, unless the testimony be of such a kind that its falsehood would be more miraculous than the fact which it endeavors to establish…

Replace “miracle” with “extraordinary claim”, and you have the basis of the quote that Carl Sagan popularized. And intuitively, most people would agree with it in principle. For example, if I told you I had cereal for breakfast, you would probably believe me. You know cereal exists and that people eat it for breakfast. Of course, I could be lying, but even if I were, I have not asked you to accept some new and extraordinary idea. (The fact that I lied wouldn’t mean that cereal somehow doesn’t exist any more.) However, if I told you that the cereal I eat every day will guarantee that I will never get sick and will live to be 100, you would probably want some evidence of that, and some pretty good evidence too.

Strictly speaking, all claims require exactly the same amount of evidence, it’s just that most "ordinary" claims are already backed by extraordinary evidence that you don’t think about. When we say “extraordinary claims”, what we actually mean are claims that do not already have evidence supporting them, or sometimes claims that have extraordinary evidence against them. Extraordinary claims require extraordinary evidence because they usually contradict claims that are backed by extraordinary evidence. The evidence for the extraordinary claim must support the new claim as well as explain why the old claims that are now being abandoned, previously appeared to be correct. The extraordinary evidence must account for the abandoned claim, while also explaining the new one.

Most people are probably unaware of the amount of extraordinary evidence required for most scientific claims. Not only must the experiments be written up in such a way that others can challenge the assumptions and be able to spot errors, but they must also be independently replicated. In addition, most scientific discoveries have provenance – that is, we know how and why we decided to test this claim in the first place. For example, a new drug may have a theoretical rationally as well as positive in vitro and animal testing before it is even tested on humans. Consequently, we already have reasons to suppose it might work. Compare that with much of alternative medicine, where we have no basis to suppose it works, and whose tenets we are pretty sure were just made up. In this case by “extraordinary evidence,” all we really mean is the same level of evidence that supports real medicine.

You can see than my claim I had cereal for breakfast is not extraordinary. We know cereal exists and people eat it. There are no other accepted or “proven” claims that you have to abandon to accept that I ate cereal for breakfast. The claim that my cereal will guarantee I will live to be 100 is an extraordinary claim. It is counter to all the other evidence we have that there is no one simple thing you can eat that will guarantee no illness and such a long life.

Examples of Extraordinary Claims

Part of the difficulty in defining what makes an extraordinary claim is this: claims that skeptics consider extraordinary, woos consider quite normal. Woos often consider that (for example) it is already a given that psychics exist, therefore anecdotal evidence is good enough for them. But psychics are scientifically implausible and have not been shown to be real. That doesn’t mean they aren’t real; it does mean we need extraordinary evidence to suppose they are. Woos start from the place that these things are already supported by evidence, and that’s where they go wrong. I’ve tried below to explain what is extraordinary about the following claims – what other claims, and what other implicit evidence, they contradict.

Homeopathy

Homeopathy is the definition of an extraordinary claim, It is initially extraordinary because it does not have provenance – that is, we know that Hahnemann didn’t derive the laws of homeopathy by experiment, he just made them up ad-hoc. Hahnemann made up the Law of Similars based on an observation of one thing (quinine / malaria symptoms), and this “Law” has not been replicated or confirmed. In fact, we now know the Law of Similars is false. We also know from every other piece of evidence we have, that when you dilute something it gets weaker, not stronger. Because of these two basic flaws, homeopathy requires stronger evidence than we would ask from other therapies. And yet with homeopathy we are expected to accept weaker evidence – anecdotes and non-blinded studies written by homeopaths. All well run double-blind tests show homeopathy is no more than placebo.

Incidentally, most alternative medical therapies suffer the same lack of provenance flaw – ie they were mostly just made up by ancient peoples with no knowledge of how the body actually works or of what makes us sick. Similarly, we are mostly offered anecdotes in place of evidence.

Astrology

Astrology is similar to homeopathy in that we know it was not derived by experiment, but was most likely just made up by people who saw pictures in the sky. (At least, no one has ever been able to show this explanation is wrong.) In addition, there is no plausible explanation for how astrology might work – ie what forces could alter a newborn’s personality in the precise ways claimed. Despite this lack of provenance and plausibility, we are still offered only anecdotes and appeals to science doesn’t know everything.

Jesus’ resurrection after 2 days

This goes against all the evidence that people do not come back to life, spontaneously, after two days. Modern medicine can bring people back from what would have been considered in earlier years to be “dead”, but not after 2 days of being dead with no modern life support to keep the vital organs working. In fact, it is probably reasonably safe to say it has been proven beyond a reasonable doubt that people cannot come back to life after being dead for two days. The evidence we are offered are accounts written decades after the event, by people who were not there when the events described were purported to have occurred. We are offered nothing but hearsay anecdotes from superstitious people with a clear reason for wanting others to think the story true. This is hardly acceptable evidence to counteract the fact that this never happens. Christians might ask, what evidence would a skeptic accept for such an extraordinary claim. The fact that even in principle you are unlikely to find extraordinary evidence 2000 years later, is hardly the non-believer’s fault.

Psychics

Psychic powers are extraordinary initially because of lack of scientific plausibility: that is, we have no known way for psychic signals to be sent and received. Lack of plausibility doesn’t mean something isn’t true, but it does make it extraordinary. The continued lack of good evidence for psychic powers, despite 125 years of looking, means that even more extraordinary evidence is now required to explain why the previous 125 years of looking were unsuccessful. For example, attempts to prove psychic powers with Zener cards were abandoned when the few positive results that were obtained were shown to have been achieved by cheating. Subsequent tests of psychics have resorted to tests that are easier to fudge – tests requiring judging to determine if the psychic got it right or not. The wiggle room this introduced results in less extraordinary evidence for this extraordinary claim that has been strangely shy to appear when properly tested (in a way that would truly be extraordinary evidence, if it worked). Instead we are left with lame guesses by the likes of Sylvia Browne and Allison Dubois, that are anything but extraordinary except in the sense that they are extraordinary bad.

Alien Visitation

Strictly speaking, alien visitation does not contradict other claims that are known to be true. It is theoretically possible that aliens exist and may hold advanced technology that enables them to travel across the galaxy. But the claim is extraordinary in that there is zero evidence alien visitation has actually occurred, despite at least sixty years of looking. In addition, there are rational explanations for many claims of alien visitation. There is no hard evidence of alien visitation, such as a crashed spacecraft with technology far advanced of our own.

No comments: