02 June 2007

Bad Science

Has anyone ever done a study on the persistence of bad science in the population? How long after a bad study is debunked do people still believe in it? What are the factors that affect the half-life of a misunderstanding? How many people are still trying to avoid all salt in their diet in order to live longer, but are actually killing themselves? How many people are gobbling down antioxidants for no particular purpose?

I sometimes try to run my own experiment in this area by trying to convince people that the germ theory of disease is pretty well-established and that getting rained on, or getting cold, or getting cold while getting rained on won't actually make you catch your death of pneumonia. I rarely succeed and, sometimes, people violently reject the very idea. (No, being wet and cold doesn't lower your resistance or affect your immune system.)


Harry Eagar said...

Geez, kind of hard to quantify, what's your yardstick?

But there are places that track various threads of idiocy, such as CSICOP and its Skeptical Inquirer magazine, to which I am proud to contribute now and then.

Dr. Stephen Barrett is another.

What strikes me is how some nutty ideas (that unless he takes drastic measures an adult's colon becomes impacted with 17 pounds -- how's that for precision? -- of death-dealing poison) have persisted in America for over a century; while other nutty ideas (oat bran will save you from heart attacks) fade quickly.

If you get your way on immigration, we will face a health crisis over fallen fontanelle.

Mike Beversluis said...

Well, Thomas Kuhn's The Structure of Scientific Revolutions is the classic work on that. To wit:

"...a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it."

And that's even true when you can just go out and make a measurement and get a yes/no answer on some prediction.

PS. affect.

David said...

Harry: It will make a nice change from mercury poisoning.

David said...

Mike: Thanks.

Hey Skipper said...

It must be noted that the half-life of bad science is a lot shorter than that of religion.

Hey Skipper said...

oops, I meant " ... bad religion."

Harry Eagar said...

Kuhn was wrong.

Peter Burnet said...

...getting rained on, or getting cold, or getting cold while getting rained on won't actually make you catch your death of pneumonia.

Of course. Old wives' tales. Yet every teacher will tell you that kids who dress properly in bad weather are sick less than those who don't. Also, wander through an old graveyard and you will be struck by the number of November/December deaths. Pre-antibiotic society was full of tales about old Uncle Seth who "went for a walk, took a chill and died two days later."

Maybe it had something to do with their astrological signs.

Bobby C said...

An excellent post, but it made use of an erroneous example. When you are cold blood flow to your extremities is reduced and your body gets stressed. Among other things, this reduces your ability to resist the virus that causes colds. You still need to be exposed to the ‘germ’, but we are constantly exposed to all sorts of germs. We only get sick when they overwhelm our bodies defences. Getting cold makes this more likely.

Peter Burnet said...

BTW Skipper, thanks for that correction. I feel the same way about secularism. I can't abide bad secularism, but the good kind is, well, really good.

David said...

Skipper: Which religions did that amendment exclude, exactly?

Duck said...

OK, so where do I go to find the real truth on all of these myths? I thought oat bran was good for you. Claims come and go so quickly. The claim always gets more publicity than the debunking. And it's not like you should blindly believe the debunkers.

Michael Fumento "debunked" the Adkins Diet, but I've been on it for 8 weeks and I've lost 23 pounds. So who do you think I'm going to believe?

David said...

I should make explicit that I'm not talking about the various varieties of "denial" here. People who disbelieve things that are (a) well-established and (b) generally accepted in their home cultures have a whole 'nother problem.

Duck: That's the big question. Here are some rules of thumb I use:

Correlations don't even imply causalities.

There needs to be at least a colorable theory about the mechanism at work.

As much as you can, read the original study rather than the media reports.

I find that most good scientific papers are largely comprehensible to the layperson.

A result hasn't been established until it's been reproduced.

Meta-studies are not necessarily wrong, but are always junk.

When it comes to our bodies, results in a test tube are almost as meaningless as correlations.

Be especially skeptical of statements, especially by laymen, that begin "It is well-established," "Everyone knows," or "Obviously."

Always look for an opposing viewpoint.

Does falsifying a major tenet of a theory make a difference to the proponents.

There's a nice example of this happening right before our eyes with respect to global warming. The advocates of agw (most famously, Al Gore) put up charts showing a strong correlation between temperature and atmospheric concentrations of CO2 to prove that CO2 causes warming. After the chart was created, the science advanced sufficiently for us to determine the age of ice core samples (from which the date for the chart came) with greater precisions. It turns out that warming precedes the increase of atmospheric CO2 by a surprisingly consistent 800 years. The advocates' response? That makes absolutely no difference and its simply blind denial to suggest otherwise.

A mechanism without data is as useless as data without a mechanism.

There's always something you're not considering.

The salt and antioxidant episodes are good examples of all this. Both theories (that salt increases blood pressure and antioxidants neutralize free radicals) had good laboratory support (i.e., the theory seemed to work in the test tubes) and decent proposed mechanisms. It's just that neither works in the actual body. The body eliminates antioxidants before they get into the blood stream and has a perfectly good salt-regulation mechanism that works fine so long as we ingest a minimal amount of salt. We also see this in the nonsense about hydration. The healthy body is very good at using thirst and urination to maintain hydration within a surprisingly narrow range. If you're not thirsty, you don't need to drink.

pj said...

duck -

When did Michael Fumento debunk the Atkins diet? I saw a 2003 Reason article (http://www.reason.com/news/show/28714.html), but it hardly debunks Atkins. It offers a great deal of skepticism toward it, but without engaging the arguments and evidence for it. Instead he replies only to an article in the NY Times mag, which was lacking in detailed scientific arguments. But why would we look to a newspaper article for the ultimate in scientific analysis? He was knocking down a straw man.

I believe the scientific evidence supports eating approximately equal masses of fruits and vegetables, on the one hand, and meat, fish, and eggs on the other, with some oil for flavor. This naturally leads to a low-carb, high-fat, moderate-protein macronutrient ratio which is much closer to Atkins than to the conventional American diet.

Susan's Husband said...

There is a correlation between weather that requires heavier clothing and colds, which explains the death date clustering which Mr. Burnet brings up, and that is simply that when the weather gets cold, people spend a lot more time in enclosed spaces where it's easier to transmit viruses. This was even more so back in the pre-biotic agriculture societies.

David said...

Right. And no one's arguing that age doesn't correlate with worse outcomes from colds.

Harry Eagar said...

duck, I didn't say oat bran was bad, or wasn't good, or even that it has been debunked.

I'm just pointing out that for a while every prepared food item was marked 'oat bran' and now you never hear about it.

Why does garlic and vinegar have legs, but oat bran doesn't?

Why does feng shui wax while geomancy wanes?

I was glad to see David knock meta-analysis.