Thursday, December 8, 2011

Food Reward and Confirmation Bias - More Woo?

The comments on Dr. Guyenet's lastest post included more -Woo-bashing, as usual.  In the paleodome, bashing -Woo is about as popular as bashing Dr. Oz and Dr. Kruse.  Is it really warranted?

I guess I have always been kind of a weird statistician.  There is a typical protocol usually followed and recommended when a pile of data comes in.  One of the most comical (and I think most wrong) techniques is to scrub the data by plotting it all and then throwing out the outliers.  A typical stupid way to do this is to calculate the standard deviation, and then automatically throw out all the data points that are beyond three sigma.  This technique pretty much guarantees that the researcher will systematically throw out perfectly good data, and it also ensures that any totally cool thing about what they are studying will be tossed as well.  Sort of like cracking the egg, separating it, and then throwing out the yolk if you are Dr. Oz, or throwing out the white if you are some of the lower-protein paleo's, or throwing it all out if it tastes good if you are Dr. Guyenet.

In my twisted mind, ItstheWoo certainly points to some excellent data.  She is the poster child for outliers, and also the poster child for how outliers are usually treated by some really stupid academics and play-acting statisticians.  It it really necessary for commenters to continually remind each other how they flip through -Woo's posts without reading?  How many minutes did they waste typing that over and over again on numerous blogs?  Wouldn't it just be easier to read some of them, and, you know, maybe learn a little something about an outlier?

So, real researchers.........not sure if you are visiting yet, but here's a little advice anyway.  Before you just toss out data because it is "beyond three sigma", you need to take a look at it.  Taking a look at it doesn't include disparaging and making fun of the data point.  "Out! darn point!!  Be done with it!"  (What research dweeb does that in real life?  Obviously there is some other deep-seated hostility going on here.)  Data points can't just be removed and discarded because they "mess up the error term" or otherwise make either the analysis or the researcher uncomfortable.  I only do data checking to make sure there isn't a typo or other similar problem.   If I can't find a reason, it stays.  I have had to fight my position on this for years, and in many situations, even resorting to doing two analyses, one with all the data, and "one with some data points thrown out".  And that's what I call it.  It's not scrubbing.  The data hasn't been cleaned, it has been lobotomized and I'll have none of that.

And on a final note, I got a large spike in readership today.  I am drawing out a few more readers, commenters and lurkers, mostly from the low carb linking sites.  More data to follow......

7 comments:

  1. Something tells me Stephan is going to have a very successful academic career indeed. It's a pity that he's dead wrong.

    ReplyDelete
  2. Hi Sidereal. I think this whole mess shows how out of touch many researchers are with what is really going on. What is baffling to me is that when faced with real people providing real data, he always reverts to what the rats say. I just cannot understand how this can continue to be a successful strategy in the long run.

    ReplyDelete
  3. When your bosses career was made on rats what is one to do?

    ReplyDelete
  4. I think it is more than just pleasing the boss. I have met many who learned a "bit" of statistics and are proud of their data-handling techniques. They truly think the practice is not only acceptable, but indicated at every turn.

    ReplyDelete
  5. As a C programmer I can understand your point about not removing the outliers. It is often especially the outliers (in programmer parlance it would mean results that you didn't expect that way) that give hints of some hidden problems or misunderstandings.
    An outlier in a nutrition statistic is at another level of complexity, of course, but if your theory cannot explain why a given n=1 result does not follow the trend, then it means that the theory is incomplete and/or false. GT strawman version of the insulin hypotheses would indeed not be able to explain Kitavans (it is not GT theory by a long shot, only the carricatural version by his opponents).

    ReplyDelete
  6. Hi Gallier2! Thanks for visiting. I would agree that Taubes also has an incomplete theory on why we are fat, but I don't see him insisting that his is the only way, like Guyenet's supporters. I also believe from my own N=1 that Taubes totally wins on the discussion of HOW to lose the weight.

    ReplyDelete
  7. I think there are things wrong with the Kitavan information in the first place but don't quote me on that--I'm going on thirdhand information. But basically the means of obtaining data about their relative health had a whole bunch of holes in it.

    Also, people didn't take into account that some foods are protective against the sorts of health problems we are seeing here. You can't cut almost all animal fat out of your diet, all natural sources of minerals and the animal-based fat-soluble vitamins and then ramp your starch and sugar intake way the hell up AND then keep the lights on too late at night and not get enough good sleep in full dark, and expect that you will come out the other side unscathed.

    The Kitavans get full sun regularly, sleep in the dark at night, eat lots of coconut and fish fat and probably get some fish broth with the attendant minerals as well. I would be *surprised* if they weren't healthy.

    But take away the coconut and fish fat, take away the minerals, take away the sun exposure, ramp up their starch, add some sugar, throw in some PUFAs and watch them fall apart in a few generations.

    Low carb works (I think) because carbohydrate is the least necessary of the macronutrients, because processing it in your body burns up micronutrients you need for continuous good health, and because the carbohydrate foods are the least nutrient-dense compared to protein-rich and fat-rich foods, so ditching the latter two in favor of the former only exacerbates all the other issues.

    I don't think the Kitavans are a paradox at all.

    ReplyDelete