Now We Go Ahunting

As a secret blog, Instapundit is our natural enemy and we wouldn't ordinarily give it even more traffic, but today Instapundit points to a fascinating story in the Economist about why so much medical research is rot. The lead in particularly well-done:

PEOPLE born under the astrological sign of Leo are 15% more likely to be admitted to hospital with gastric bleeding than those born under the other 11 signs. Sagittarians are 38% more likely than others to land up there because of a broken arm. Those are the conclusions that many medical researchers would be forced to make from a set of data presented to the American Association for the Advancement of Science by Peter Austin of the Institute for Clinical Evaluative Sciences in Toronto. At least, they would be forced to draw them if they applied the lax statistical methods of their own work to the records of hospital admissions in Ontario, Canada, used by Dr Austin.

It turns out that, when seeking more than one effect from a database of medical records, scientists forget to adjust their calculations and find causal connections where there aren't any.

The Economist points out that this means we should be suspicious of studies based on databases. But it doesn't point out that we need to be even more skeptical of so-called meta-studies that gather together many studies and tries to tease new relationships out of the agglomerated data. Meta-studies are almost entirely a means of finding a politically correct result. It wouldn't be at all surprising if the designers were also just doing their math wrong.