Foster's POUNDS LOST Study

A recurring theme in my career has been this discovery of a little packet of data, just stuffed in an unmarked envelope, left on my desk or slipped under the door, or pushed towards me in a conference room while others aren't paying attention.  Or, colleagues might pass by in a hallway, and furtively motion me over to some quieter place to "just take a look at something."

The challenge, after opening these envelopes and having a look inside, was for me to determine who gave me the secret data.  Surely, they would be wanting some sort of response.
I was reminded of this while I was reading through Dr. Foster's POUNDS LOST study, and wondering which researcher was getting a little packet of the real data hidden in an in-box somewhere.

I promised you all I would get back to the POUNDS LOST study.  Here's Dr. Foster's paper, finally published in 2010.  It's the study said to compare a low carb diet and a low fat diet for two years.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2949959/?tool=pubmed  They concluded that there really wasn't much difference between a low carb reducing and a low fat diet except some markers for heart disease (better in the low carb dieters!)

I really wanted to like this study.  Oh well....

I also really wanted to read this one very carefully before commenting on it.  Initially, it didn't seem like it would be as bogus as some of the other studies, but I still had concerns.

The first thing that really popped out for me is that while the participants were overweight, they were excluded if they had diabetes, high blood pressure, were taking lipid-modifying drugs, and other illnesses.  In short, they didn't even include people with metabolic syndrome.

This omission wouldn't have made such a difference if I hadn't read Dr. Gardner's A to Z study and also saw his lecture.  In the lecture, Dr. Gardner showed how the participants in his study did when they had insulin sensitivity issues that are associated with metabolic syndrome.  This information was in his lecture, not in the published paper.  In short, Dr. Gardner discovered that the participants tending towards metabolic syndrome did much better on the low carb diet than they did on a higher carb diet.  When all the data was thrown together, it didn't look like there was much difference between the diets, but when insulin sensitivity was taken into consideration, the low carb diet was the very clear winner for dieters with metabolic syndrome.

I think the American Diabetes Association has been taking a careful look at all of this.  It seems like every year, they relax their admonition against low carb diets a little bit more.  There are so many diabetics that prefer a low carb diet that the group is becoming difficult to ignore, both in terms of numbers and in terms of success.  (Oh yea, in terms of the actual data.)

The second thing I noticed about the POUNDS LOST study is their wonky description of the way they made up missing data.  (I usually don't read studies in order, I tend to jump to certain places first, like what methods of data analysis they use, and then go back and read the intro part more carefully.)  Lots of people dropped out of both groups over the two years.  My guess is that there were many critical reviews of the way data was made up in earlier studies done by Foster, so they made extra sure to explain it this time.  But, I read it, and kept re-reading it, and the BS meter started to move up a bit.  I couldn't figure out what they were talking about.  I am not a huge fan of making up data anyway, I think it is misleading.  This study just modifies the method a bit, but is still just making up data.

The other thing I found odd is that there was no actual data shown.  Instead, they used some statistical model, made up the missing data, and then showed the predictions from THEIR MODEL, not the actual pounds lost by anyone.  Now I am thinking that the only reason that they did this is to try to weasel out of something.  Nowhere in the paper do they talk about the validity of their model in terms of how it actually fits the real data.  All the charts show plots of the predictions, not any actual data.  The BS meter is really up higher now. I hope the actual data shows in somebody's in-box.

Then I went back to the top, and read carefully about the two diets.  One was called "Atkins", but the Atkins book wasn't given to the participants.  The description of the diet doesn't sound like Atkins at all.  This low carb group was on an induction-style diet for three months, and then started adding carbs.  

"After 3 months, participants in the low-carbohydrate diet group increased their carbohydrate intake (5 g/d per wk) until a stable and desired weight was achieved."

This isn't Atkins, this is something else.  And that something else is that they seemed to follow Atkins initially, lost lots of weight, and then went into maintenance mode.  The Atkins plan is to start slowly adding carbs back in as the desired weight is achieved.  In this study, the participants rose their carbs after three months, with the result being that of course they didn't lose as much as they would have if they had actually stayed on the Atkins plan.  And maybe its just the I'm-From-Missouri in me, but I'm thinking that if they wanted to test the Atkins diet, they would have actually tested the Atkins diet.  So here's the recap so far:

1.  We put a bunch of people on one of two diets, an Atkins diet and a low fat diet.
2.  After the first three months, the Atkins group looked much better.  Better weight loss.  Better blood lipid factors.
3.  We told the Atkins diet people it was OK to start adding 5 g carb per day every week instead of following the Atkins diet.  
4.  People on the diet that used to be the Atkins diet started gaining weight.
5.  By the end of the study, the weight loss of the two groups wasn't different.
6.  We concluded that the Atkins diet and the low fat diet are the same after 2 years.
The fourth thing I noticed is the reason I decided to call this study a bogus study.  I looked at the plots of the triglycerides.  It didn't make sense that the triglycerides went down so nicely in the low carb group during the first several months, and then went way up, to the point where they were even higher than in the other group that was eating so many carbs on purpose.  This just shouldn't be.  If these people were really following Atkins, their triglycerides would continue to stay low.  This plot is just further proof that these dieters weren't on Atkins anymore.
These researchers and their associates also made sure that the media was alerted to the ho-hum nature of the study.

Here's TheHeart.org, slamming reporting on the paper in a scholarly way, minimizing the success of the low carb approach.  The majority of the commentary was not by anyone who actually wrote the paper.  So, even though the low-carb diet was clearly better in weight loss and heart risk factors in the early stages of the study, everything eventually evened out and the early-stage success wasn't mentioned.  The reviewers did want to make sure that we all knew that the low carb people had bad breath, which was also rectified by the end of the study, but they still felt the need to bring it up anyway.  We haven't gotten actual values of actual weights, or any food logs, and yet the bad breath data was quite important?

http://www.theheart.org/article/1108461.do

There's no doubt about this article.  It's a slam for sure.
http://www.fathead-movie.com/index.php/2010/08/19/another-biased-study-maybe/ It's by the guy who did the movie "FatHead".  He brings up some of the same concerns I had about the low carb diet not really being an Atkins plan, and how the study is full of made-up data.  The comments on this blog post are pretty interesting, too.
A little while later, Tom Naughton wrote this enlightening blog post:

http://www.fathead-movie.com/index.php/2010/08/30/inside-information-from-fat-throat/

It seems like Tom Naughton was the recipient of the plain manila envelope, too!

Here's another blog post a bit later:

http://www.acupuncturebrooklyn.com/alternative-health/two-year-low-callow-carb-diet-study-misleading
She says it much better than I could ever do.