How Kevin Hall tried to kill the insulin hypothesis with pure spin

Kevin Hall, the senior NIH researcher recently published a paper in AJCN that has received a lot of media attention. He claims this study refutes the insulin hypothesis so completely that it is now ‘dead.’ That’s interesting, I thought, as I sat down to read the article.

It was therefore a little surprising to read this paper and realize that Hall’s conclusions were entirely his own opinion. He suffers so badly from confirmation bias that he may as well have written “My mind is already made up regarding the insulin hypothesis. Please do not confuse me with facts”.

Confirmation bias is a well-known psychological phenomenon whereby facts that agree with your pre-formed opinion are accepted as true and those that are not are ignored. All facts become filtered through this bias to confirm your previously held opinion. It’s also known as a closed mind.

So, let’s take a closer look at this paper and its claims. The paper is titled “Energy expenditure and body composition changes after an isocaloric ketogenic diet in overweight and obese men“. Let me give you some background. The award-winning science journalist Gary Taubes believes that obesity is essentially a disease of too much insulin – hyperinsulinemia. As refined carbohydrates stimulate insulin more than fat or protein, reducing carbs will result in greater fat loss.

Gary Taubes set up the non-profit organization NuSI to raise money to fund research and this paper is the first one published. 17 overweight men were admitted to a metabolic ward where all the food they ate was carefully measured. There was a 4-week run-in phase to establish a baseline where men would eat a high carbohydrate, high-sugar diet and then switch to a carefully designed low carbohydrate, low-sugar diet. Various measurements were then taken, including energy expenditure (EE – how many calories the body is burning), over the next 4 weeks.

Does a ketogenic diet cause fat loss?

Here’s the result. Over the 4-week KD, yes there was fat loss. There was an initial period of greater weight loss which all agree may be some diuresis. We can also agree that insulin levels were brought down by the KD. Secondly, using measures of EE there was an increase in calories burnt.

Those are all facts, not opinions, derived straight from the study. Isn’t that a good result?

Well, if you’re Kevin Hall, no. You need to find a way to spin this in a negative way. Then you can tell all your friends in the media so you can declare ‘I was right.’ Let’s see how this was done.

When patients embarked on their run-in phase, they were switched to a 2700 calorie/day high-sugar high-carb diet, meant to replicate the Standard American Diet (SAD) that caused the obesity epidemic. Nobody actually believes that this is a healthy diet, and nobody believes it should cause fat loss. But it did. Why?

Anybody who has done research knows why. It’s the effect of going into a study and knowing that people are testing you. It’s a universal effect. That’s the precise reason why we have run-in phases. To establish a baseline.

So, people lost weight on this SAD diet. But instead of using this new baseline, Hall decides that the downward trend is the new baseline. The unspoken premise or assumption is that if these people had taken another 4 weeks of the SAD, they would continue to lose weight at the same rate indefinitely. WHAT? Are you out of your mind? That’s completely illogical.

Let’s take an analogous situation. Suppose we are teaching math. We teach one semester with no tests, no exams, no checking of homework and no projects. Students are just supposed to spend 1 hour in class and 1 hour of homework a day. They all say they do it. Then, unbeknownst to them, we test them on a standardized test. They do really bad and score 65%.

Next semester, they have daily tests, a final exam, and daily checking of homework. They still spend 1 hour in class and 1 hour of homework. Scores should be theoretically unchanged, because they were doing the same amount of work. Of course, in reality this is completely false. Because they know we are regularly checking them, they do a better job. Now they score 80%.

This is the same effect we see when people enter a study. No matter what we are measuring, things improve simply by entering a study. It happens with blood pressure, blood sugars, cholesterol, diets, depression – everything. But the results do not get better indefinitely. It’s a one time benefit.

Students’ scores might improve from 65 to 80 in one semester. This does not mean that another semester of testing will raise their scores to 95. Instead they will likely stay at 80. But this is exactly what Hall does – he assumes that this one-time benefit will persist indefinitely.

By making this assumption that the SAD diet will continue to cause fat loss (which logic tells us is false) you can make a positive result negative. So, yes, KD does cause fat loss, but does not INCREASE fat loss and then you can make this your conclusion. Since most of Hall’s journalist friends never read the paper and only the abstract, it’s easy to convince them.

According to Hall’s assumption, you should therefore simply continue eating the SAD with 25% sugar and expect to lose weight indefinitely. Go ahead. See what happens. I already know. So do you. You’ll get fat, you’ll get type 2 diabetes, and then eventually, I’ll put you on dialysis and chop off your feet when they go gangrenous. But at least Hall can say that he was right.

What about energy expenditure on a ketogenic diet?

The second major issue concerns EE. When you switch from the baseline diet to the KD, the number of calories stays the same. If it causes continued weight loss, then you might expect to see an increase in EE in order for the body to lose weight. This is called a metabolic advantage. Surprise, surprise – that’s exactly what happened. So how can you spin this one? With language.

Look at how Hall describes the absolutely critical increase in EE. Here’s what he writes:

the KD coincided with increased EEchamber (57 ± 13 kcal/d, P = 0.0004) and SEE (89 ± 14 kcal/d, P < 0.0001) [emphasis mine]

Hall is telling you that this was merely a coincidence that patients are all burning an extra 57 calories per day. WTF??? There is nothing coincidental about it. You switched them to a KD. EE increased. The P value of 0.0004 means that there is a 99.96% chance that this is NOT A COINCIDENCE. Hall knows this as well as I do. This is basic statistics 101. Hall, a mathematician is surely aware of this.

Hall is saying “Oh, we switched their diet to test if EE would increase. It really was just a huge coincidence that all 17 men simultaneously increased their EE altogether at exactly the moment we thought it would. Ignore this, guys. Just write your newspaper article about how this proves it didn’t happen.”

So EE increased and yes, the effect waned over time. What did he expect? That things would continue indefinitely in a straight line? Life doesn’t work that way. Hall had assumed this would happen for fat loss during the SAD, but then correctly points out that EE does not. It doesn’t in either case, dude. Get a clue.

The key to lasting weight loss

The reason why EE is so critically important is that this is the key to lasting weight loss. Hall had just been profiled in the New York Times cover page measuring the EE of Biggest Loser Contestants. The reason they all regained their weight was that their EE slowed to such a degree that caloric reduction could not keep up.

So, an intervention like a ketogenic diet that increases EE is HUGE, GREAT NEWS. Except, of course if you’re Kevin Hall, because it means you were wrong. And you care more about your reputation than people’s health and well-being.

Dr. David Ludwig, a researcher from a little place called Harvard had shown exactly this same thing in his study from 2012. This study also tested for difference in energy expenditure following different dietary strategies. Once again, as Hall had shown, the EE is best with a very low-carbohydrate diet. So Halls study merely confirmed what had already been known.

Some people have also noted that this study controlled calories so it negates one of the KD’s biggest advantages, which is that it makes you feel full. Well, sorry, guys, that’s not the question it is designed to answer. Same goes for the fact that there are only 17 people in it. Again, that’s the study design, so it is what it is, and there’s no use complaining about it.

The problem is not the data but the ‘spin’

In the end, the main problem is not the study data. The data are excellent. The problem is the ‘spin’. Here’s the conclusion Hall writes in the abstracts conclusion (which is the single most important few sentences of the paper, the one that everybody reads).

The isocaloric KD was not accompanied by increased body fat loss but was associated with relatively small increases in EE that were near the limits of detection with the use of state-of-the-art technology.

I’ve highlighted what is fact. I’ve crossed out what is pure spin. Did the KD cause body fat loss? Yes it did. And that’s really, really important. Hall spins it this positive into a negative by moving the goalposts – “Oh but it didn’t do better than before”.

Then he says the increase in EE is “relatively small’. So what? Did it increase or not? In fact, your own study from the Biggest Loser suggests that weight loss will DECREASE EE, so even the stabilization (let alone the increase) of EE is critically important. That’s the gold medal, buddy! You just threw it in the garbage.

Hall then downplays this relationship by calling it an ‘association’. As if the change in EE just happened to occur at the same time as the change in diet. What a load of crap. You changed the diet and measured the change in EE. Nobody doubts that. It’s causation, pure and simple. So why try to spin this as an ‘association’ which is a mere ‘coincidence’? Pure spin.

Hall then further tries to downplay the importance of stable EE by saying it’s ‘near the limits of detection with the use of state-of-the-art technology’. So what? Who cares? Did it stabilize or not? Isn’t that great news? Did you not just show that weight loss efforts fail because of decreased EE?

Unfortunately, spin-doctor Hall is now entering a logic free zone, and many journalists like Julia Belluz and other bloggers are happy to take what is shared at face value. “In this first Biggest Loser study, I demonstrate why stable EE is critical for weight loss. In this second study I’ll show how stable EE is absolutely worthless. Ta Da!”

Hall wants desperately to save his own reputation, even if he has to sacrifice your health to do so. Sad. So sad.

The facts alone, without any spin would be this. A ketogenic diet, independent of calories, causes fat loss and causes an increase (or at least the stabilization) in EE. That’s the facts. And I love it. Because I can use these facts to help heal patients and save lives.
—
 
Jason Fung