As I was reading the linked article from yesterday's posts (see link below) on the widespread practice of sweeping public policy decisions made on the basis of faulty research, I checked out some of the references and I was reminded of one of my favorite topics, ecological correlation.
So, what's ecological correlation? No, it's not about recycling or reusing your data, and its not all about Al Gore, either. But, it IS a bit like reducing your data, as in reducing it into a more manageable form so as to make it pretty much meaningless or misleading. In a nutshell, ecological correlation is a statistical fallacy.
Many reading this probably remember John Robbins' "Diet For a New America". In his famous book extolling the virtues of vegetarianism, he showed a chart with breast cancer rates plotted against dietary fat intake, with each point coming from a different country. It had a great caption: See The Pattern?
Of course, we saw the pattern. The pattern was clear. There was impeccable correlation between breast cancer and dietary fat intake. Both the researchers and Robbins concluded that dietary fat intake causes breast cancer.
But, when researchers actually started to do some of these experiments to test what should have been only theory, they couldn't find such a clear relationship.
What went wrong? First, researchers confused a correlation with a cause. Second, they suffered from a bad case of ecological correlation. I am going to cut some slack for John Robbins. He is a writer, not a researcher. His book is peppered, even littered with convincing ecological correlation graphs. He was just using data that had been published in the "peer reviewed" journals, from people like Carroll and Keys. (You might remember Keys. He's the guy who did the starvation experiments with conscientious objectors, and also that bobble-headed cartooned researcher whose data points came crashing down in the movie, Fat Head.) Data trends that show promise for entire groups oftentimes don't apply to individuals. The sad part is that the public policy experts and nutritional authorities ran with the original studies and it is only now that some are starting to back-pedal the low-fat recommendations. I say some, because it doesn't seem like some dietitians want to lose their grip on either their opinions or their control over who even gets to talk about it.
This poor use of data also results in confusion and breeds mistrust amongst the public. We're told to lower cholesterol, then we're told it doesn't matter. We're told that high fat causes breast cancer, then we're told it's OK. We're told that eating lots of veggies will protect us against cancer, and then we are told they don't. We're told that vitamin D is more important, and then, not so fast. It IS confusing. Much of the misleading early research needs to be recycled in the compost heap. And I don't mean just re-using the same old data points to build newer papers. I mean going out and getting some more meaningful data.
Here's the link to the original article that got me thinking:
http://www.ajpm-online.net/webfiles/images/journals/amepre/AMEPRE2099.pdf
Here's a link for an explanation of ecological correlation. If you aren't into statistics and just want to cover your eyes and ears, here's the short take-home: ecological correlation = BAD!!! When you see a chart plotting averages against rates, run for the hills!
http://en.wikipedia.org/wiki/Ecological_study
No comments:
Post a Comment