Wednesday, June 22, 2011

Scientific Method 101

Science.  It's like making sausage, really.  Maybe with some floor scrapings and a dash of meat glue.
I remember one of my first gigs as a professional.  I worked with a huge group of in-house consultants, a real "navy seal" team.  We had all the tools and the training and were ready for anything.
Due to the success of the division I was assisting, I was selected to help write a white paper about a popular statistical technique.  Our team met for months, thrashing out the outline and general flow of the paper.  Each participant was asked to write a case study, so that the final report would contain a nice selection of examples that readers could draw upon depending on their situation.
Like the other statisticians, I wrote my portion of the paper.  Then the papers were sent out for review by the other team members.  Several months later, the team leader decided that we all needed to revise our papers.  It seems like most of the team members were required to re-write their papers more in line with the scientific method.  We were supposed to have a hypothesis, and H0's, H1's, that sort of thing. 
I resisted rewriting my paper because that is not how it happened.  Basically, I was low on the totem pole, and so was assigned to sort of the Tora Bora of the company.  We didn't use lots of the fancy statistical techniques, but my clients were very happy with me, and quite pleased with the results. 
I was told by the team leader that there was no way that we could have made real improvement by just "random messing around".  He insisted it was impossible.
"So, if it is impossible, do you want to take away all the improvement they have seen?" I countered.
The leader insisted that the improvement could not take place without an H0 or H1 beforehand, and told me my example would not be included in the report.
Amazing!  That was a great learning experience for me.  It was then that I became fascinated with truth, and who gets to decide it.  I started reading about the scientific method, the philosophy of science, and realized that it really is just a philosophy, and for people who insist that it is the real truth, well, that's just their religion.
This story does have an interesting ending.  Despite not having my example published, the paper was widely lauded by the authorities.  My client went on towards continuing success.  The very simple technique was used by another sister division, and that division eventually applied for and won the Malcolm Baldridge Quality Award.  A version of the technique used was also used in a study that was eventually published in a "peer-reviewed" journal.  So, yes, I am a peer.
Lately, it does seem like there is much push-back from the authorities on the Paleo/Low-carb scene.  Plenty of experts are getting into the act, instructing all the lost low-carb souls on the ways of science.  But I think lots of this push-back is what my Missouri uncle used to call bovine effluvium.  It's just lots of talk designed to establish their authority, and it certainly takes nothing away from the actual improvements some of these poor souls on such a faddish and untested diet have actually experienced.

No comments:

Post a Comment