Recently, a giant paper on carbohydrate consumption and mortality was published in The Lancet. The paper discussed the findings of a prospective cohort study and a meta-analysis of several cohort studies. Studies like this are often the ones that generate the most hype, which is always bizarre to me given that higher quality randomized studies almost never receive any attention. As a result of all the noise (see below), I had to discuss the study in question.
I want to quickly summarize the methods of the cohort study, what the findings were, and then discuss the implications of it.
Who And What Was Studied?
So the paper first discussed a prospective cohort study titled, The Atherosclerosis Risk in Communities (ARIC), which was a study looking at cardiovascular risk factors in four communities in the U.S: (Forsyth County, NC; Jackson, MS; suburbs of Minneapolis, MN; and Washington County, MD).
The participants that were recruited in these communities were aged 45-64 and were recruited between 1987 and 1989. There were a total of six follow-up study visits. Dietary information was collected at visit 1 and 3 via a 61-item food-frequency questionnaire (FFQ).
Now, if you’ve ever wondered what a food-frequency questionnaire is like, you’re in luck! Here's a copy!
There's also a nice video on the FFQ here.
Walter Willett (one of the authors of the study and a giant contributor to the FFQ) has been one of the prominent defenders of the FFQ and has continuously claimed that it is a validated tool.
However, most nutrition researchers lean towards the idea that the signal-to-noise ratio of FFQs is far too low and that we need to come up with more objective methods of figuring out what people ate rather than relying on their memory. Here's an excerpt from a recent paper titled, "Controversy and Debate: Memory based Methods Paper 1: The Fatal Flaws of Food Frequency Questionnaires and other Memory-Based Dietary Assessment Methods,"
Herein, we present the empirical evidence, and theoretic and philosophic perspectives that render M-BMs data both fatally flawed and pseudo-scientific. First, the use of M-BMs is founded upon two inter-related logical fallacies: a category error and reification. Second, human memory and recall are not valid instruments for scientific data collection. Third, in standard epidemiologic contexts, the measurement errors associated with self-reported data are non-falsifiable (i.e., pseudo-scientific) because there is no way to ascertain if the reported foods and beverages match the respondent’s actual intake.
Fourth, the assignment of nutrient and energy values to self-reported intake (i.e., the pseudo-quantification of qualitative/anecdotal data) is impermissible and violates the foundational tenets of measurement theory. Fifth, the proxy-estimates created via pseudo-quantification are physiologically implausible (i.e. meaningless numbers) and have little relation to actual nutrient and energy consumption. Finally, investigators engendered a fictional discourse on the health effects of dietary sugar, salt, fat and cholesterol when they failed to cite contrary evidence or address decades of research demonstrating the fatal measurement, analytic, and inferential flaws presented herein.
Anyway, back to this study. Dietary information was collected at visit 1 and 3 via the FFQ, which was used to calculate the average carbohydrate intake.
The primary outcome that the researchers were looking at was all-cause mortality and this information was collected through phone calls, records from hospitals and the state, and the National Death Index.
Because this is an observational study that has no randomization involved (random sampling, random assignment etc), several characteristics will differ between groups, which may contribute to the differences seen in mortality rates. The researchers assessed some of these covariates (which include age, sex, self-reported race, energy intake, study center, education, exercise during leisure activity, income level, cigarette smoking, and diabetes) and made statistical adjustments in their models.
The problem with statistical adjustments is that there are always quite a few covariates to adjust for, and several ways to adjust for them that result in small-moderate deviations in the effect size.
I discussed this in a previous blog post. Some researchers have proposed that rather than produce one or two statistical models (as done in this study), that all possible models (with different combinations of covariates) be reported, or the median of all possible models be reported (not done in this study).
Back to this study. The researchers in the ARIC study also excluded people who developed diabetes, heart disease or experienced a stroke before visit 3 due to potential confounding from lifestyle changes after these events. People who ate too few calories (<600), or those who ate too many calories (>4,200) were also excluded.
The researchers calculated the probability of death for each year of age in a carbohydrate intake category (>65%, 55–65%, 50–55%, 40–50%, 30–40%, and <30%). These probabilities of death were compared to the probability of death in a reference group that consumed 50-55% of its calories from carbohydrates AKA “moderate consumption.”
What Were The Findings?
Dietary Characteristics Of The Participants
The median carbohydrate consumption in this cohort was 48.9%. Those who were in the lowest quantile of carbohydrate consumption (<30%) were more likely to be males, young, other races that weren’t black, college graduates, have a high BMI, exercise less during leisure time, smoke, and have diabetes. These are some of the covariates that the researchers made adjustments for.
Average calorie consumption from animal fat and protein was higher than plant fat and protein in all the carbohydrate quantiles.
The group that ate the least amount of carbohydrates ate the most animal fat and animal protein and also ate the least amount of fiber, fruits, vegetables, and plant fats.
They also report the following,
“The animal-based low carbohydrate diet had more servings per day than did higher carbohydrate diets of beef, pork, and lamb as the main dish; beef, pork, and lamb as a side dish; chicken with the skin on; chicken with the skin off; and cheese. The plant-based low carbohydrate diet had more servings per day of nuts, peanut butter, dark or grain breads, chocolate, and white bread than did higher carbohydrate diets.”
Mortality In The ARIC Study
The group that consumed the least amount of carbohydrates had the highest risk of death. Let me unpack this statement.
3,026 people in the group ate 50-55% of their calories from carbohydrates. Of those 3,026 people, 1,162 (38.4%) of them died. This is the reference group, so they have a hazard ratio (which represent instantaneous risk over a certain period) of 1.
In the group that ate the least amount of carbohydrates (<30%), there were 315 participants of which 163 (51.7%) died. The hazard ratio for this group was 1.58 when adjusted for age, race, and gender. When age, race, gender, test center, total energy consumption, diabetes, cigarette smoking, physical activity, income level, and education were adjusted for, the hazard ratio dropped to 1.37.
Meaning that the group that ate the least amount of carbohydrates, after adjusted for several covariates, had a 37% higher risk of mortality than the reference group.
Something important to keep in mind is that the group that ate the least amount of carbohydrates (<30%), also had the least amount of participants than all the other groups. This group only had 315 participants, and there is often a lot of variability and uncertainty with small samples. We can also clearly see this based on the coverage of the confidence intervals. The confidence interval and the hazard ratio were 1.37 (1.16-1.63).
If we compare the coverage of this confidence interval to the group that consumed 30-40% of their calories from carbohydrates, which had a total of 2,242 participants, we can see that there is less uncertainty in the point estimate, 1.21 (1.11-1.32). However, the general trend does seem to be there. The group that consumed the most carbohydrates (>65%), which included a total of 714 participants, had a hazard ratio and confidence interval of 1.16 (1.02-1.33). So a 16% higher risk of mortality.
The authors also calculated the mean residual lifespan based on carbohydrate consumption. A 50-year old man who ate less than 30% of his total calories from carbohydrates was projected by the authors to have a life expectancy of 29.1 years, while a 50-year-old man who ate 50-55% of his calories from carbohydrates was expected to have a life expectancy of 33.1 years.
The authors also ended up pooling the results of several prospective cohort studies and mainly found that when compared to a moderate carbohydrate diet, a low carbohydrate diet was associated with a 20% increase in mortality (pooled HR 1.20, 95% CI 1.09, 1.32). When compared to a moderate carbohydrate diet, a high-carbohydrate diet was associated with a 23% increase in mortality.
I thought the authors summarized their points well as to what could be the cause of the differences in the mortality rates,
“There are several possible explanations for our main findings. Low carbohydrate diets have tended to result in lower intake of vegetables, fruits, and grains and increased intakes of protein from animal sources, as observed in the ARIC cohort, which has been associated with higher mortality. It is likely that different amounts of bioactive dietary components in low carbohydrate versus balanced diets, such as branched-chain amino acids, fatty acids, fibre, phytochemicals, haem iron, and vitamins and minerals are involved. Long-term effects of a low carbohydrate diet with typically low plant and increased animal protein and fat consumption have been hypothesised to stimulate inflammatory pathways, biological ageing, and oxidative stress.
On the other end of the spectrum, high carbohydrate diets, which are common in Asian and less economically advantaged nations, tend to be high in refined carbohydrates, such as white rice; these types of diets might reflect poor food quality and confer a chronically high glycaemic load that can lead to negative metabolic consequences.” (This was the case in the PURE study)
So can we blame the macronutrient composition itself? Absolutely not. This is not what the authors even argued. They suggest that people who ate low-carbohydrate diets hardly ate any plants, while those who tended to eat high-carbohydrate diets ate more refined carbohydrates.
While the conclusion the authors arrive at seems very reasonable to me, I am still not a fan of the use of the FFQ because I believe there is far too much measurement error. Also, the lack of exploring all possible models remains a problem (were the analyses even preregistered?), and why pool the effects of studies that aren’t very comparable?
This study certainly is interesting, but I think more attention needs to be given to food quality and we need better ways to measure the exposures. I hope studies like this do not end up having a large impact on policy because we simply don't know much.