Nutrition research is hard. But is that an excuse to not do it well?
Dr. Ludwig and colleagues recently published a research letter in the journal JAMA Open Network and an op-ed in The New York Times (behind a paywall) that sheds new light on the failings of nutritional research studies.
We have already discussed the weaknesses of nutritional epidemiology, but the current analysis focused on the higher-quality randomized clinical trials, and even further, it focused on only studies published in select, high-impact medical journals.
The research letter investigated discrepancies between what a randomized trial says it will measure — before beginning data analysis and before blinded data is unmasked — and what results it actually reports. To maintain the highest level of scientific integrity, there should be no discrepancy between these two. Scientists should report what they set out to study, understanding that changing important outcomes or other parameters along the way creates significant bias and lessens the quality of the results.
Dr. Ludwig and colleagues found that 86% of the diet trials studied had “substantive discrepancies” while only 22% of drug trials did. The authors conclude “Most diet trials in the best journals fail even the most basic of quality control measures.”
In his op-ed, Dr. Ludwig comments on how troubling this is since “epidemics of diet-related disease will shorten life expectancy and impose huge economic costs on the United States in coming years.” It’s a failure that we are still debating the same questions we have been asking for over a decade regarding carbs, fat, meat, sugar and sweeteners, largely due to inadequate research.
While more resources will certainly help (see our recent article on how the government doesn’t care about nutrition) we also need to change the culture of nutritional research. And by “we,” I mean academic journals by redefining what meets adequate quality for publication, media outlets by how they report nutritional science, and the rest of us by how we react to those media reports.
One important example is the issue of compliance. Dr. Ludwig mentions that we would never say a “drug failed” if the subjects simply didn’t take it. But it may not be that simple. Why did they not take they drug? Did they simply forget or were there unpleasant side effects?
The same applies to nutrition. If the diet was too difficult, or made someone feel poorly, then compliance would be low. The commonly held belief in nutrition research is that all diets lead to initial weight loss, but none provide lasting benefit since dietary maintenance is hard.
How do we balance that belief in light of Virta Health’s trial showing 74% compliance with the ketogenic diet at two years? Their trial shows that with adequate support, dietary compliance is sustainable, and so are the lasting health benefits.
So, do we need more metabolic ward studies proving the physiologic effects of a specific diet? Or do we need more real-world studies showing sustainable health benefits for individuals?
Whichever route we take, we would all benefit by listening to Dr. Ludwig’s plea:
Study authors and the media can help by avoiding the tendency to overstate the results of weak research, contributing to public confusion. And the public has a critical role to play, not only demanding government action but also volunteering for diet studies.
Current nutritional research may be bleak, but it isn’t hopeless.
At Diet Doctor, we’re committed to being your most trusted source for nutrition and health news. We will continue to point out strengths and weaknesses in nutritional science in our effort to help millions everywhere dramatically improve their health. Please continue to follow our news feed for more timely analysis of the latest nutritional and health information.
Thanks for reading,
Bret Scher MD FACC
More
New study falsely claims low carb worsens diabetes
New advice on heart health based on the same old weak evidence