Continuing the dialogue on EAT-Lancet
By Francisco Zagmutt, Jane Pouzou, and Solenne Costard
On September 28th, 2019, we published a correspondence with our concerns regarding the modeling methodology used in the EAT-Lancet report. The authors of the report were given the opportunity to respond, but since The Lancet only allowed one round of responses we decided to publish our rejoinder below.
But before we dive into the details, for readers interested in the EAT-Lancet report, plenty of opinions criticizing (e.g. here, here, and here) and supporting (e.g. here, here, and here) the report have been published. Some of these criticisms are based on the affordability of the diet, which may be out of reach for 1.6 billion people, and the lack of logistically sound means to sustainably produce the diet’s components. On the other side, of chief importance to this dialog is that it is now widely understood that although the diet is touted by its authors as a “planetary health diet”, it is solely based on health goals and more specifically, on minimizing projected deaths. This health-only diet was then checked against a subset of planetary bounds. This subtle but important distinction is key because the diet significantly departs from many well-known dietary guidelines such as the US Dietary Guidelines for Americans and the United Kingdom Eatwell guide, also established based on health criteria.
So, the fundamental question is that if the diet is very different to others that used similar health goals, something about this diet must be quite different. But such differences weren’t clear to nutritional experts we spoke with, so this motivated us to embark on a detailed review and replication of the authors’ findings. As we had questions and also couldn’t quite replicate their results, we contacted the authors directly in multiple instances but never received a response, so this lead us to write our initial Correspondence to The Lancet, to which the authors responded. Since The Lancet decided after long deliberation not to publish any further counter-responses, the text below is our response to Prof. Willett and colleagues.
We invite the EAT-Lancet authors to write us and we’ll publish their subsequent responses here. Additionally, we will be happy to host the full code and databases they used for their analysis here and we’ll also post ours so that anyone can use and compare our findings |
Our reply to the Author’s response to The EAT–Lancet Commission: a flawed approach?
We thank the authors for their response. Alas, they didn’t address our concerns1.
First, Willett’s response ignored a key point to this discussion: the under-estimation of uncertainty in the calculation of avoided mortalities. After including all sources of uncertainty (Relative risks (RR), prevalence of consumers per food item, amounts consumed, mortality rates, populations) when replicating the comparative risk method (Figure 1),2 we found the prevented mortalities from Eat-Lancet’s reference diet in the USA are not statistically different from the current diet once adjusting for the impacts of being over- or under-weight. As the EAT-Lancet reference diet is based solely on nutrition and health considerations3, uncertainty in the avoided mortalities propagates to all subsequent calculations and thus, should be properly included to result in robust recommendations.
Figure 1: Uncertainty distribution for avoided mortalities based on adoption of the EAT-Lancet diet which includes only RR uncertainty, compared with the total uncertainty about the diet when uncertainty is included for all variables in the calculation.
Second, Willett missed our point that their assumption of perfect adherence to their reference diet vs variable adherence in the status-quo diets results in an inherently biased comparison. In all three versions of the health impact modeling, the status quo diet of a country is compared against the scenario that every person consumes exactly the reference diet. Whether or not the adiposity-related RRs are applied to estimate the effect of the caloric change, the comparison will overestimate the avoided mortalities of the reference diet because it does not compare the diets on the basis of relative composition alone. Willett further confirmed this assumption of global adherence when discussing how the diet will improve nutrition in developing countries based on the substitution from low-quality to high-quality foods.
Indeed, the effects of adiposity are also important; in the Springmann et al.(2018)2 method used by EAT-Lancet, the authors assumed global adoption of their diet with perfect compliance to recommended daily consumption amounts and calories and compared it against the status quo diet with over- and undernutrition. This comparison overestimates the mortality prevented by changing populations to the reference diet, as confirmed by Willett’s comments regarding the effect of adding adiposity. Springmann et al. (2018) assumed that the dietary change would eliminate adiposity-related deaths through appropriate caloric intake, which correspond to over 50% of their total prevented mortalities2 (Figure 2). To illustrate this point, we calculated that if one were to consume the maximum of each recommended intake range in the EAT-Lancet diet, the caloric intake would be approximately 3852 kcal daily, far more than the 2500 kcal that is cited as the average energy needs of a 70kg male and the intake assumed for the reference diet. Certainly, not many individuals would consume all the max intake ranges, but some might get close to it. By not allowing for this variation in their reference diet, the caloric intake of the reference diet is not matched to the current intake of the population, so the comparison unreasonably assumes a change in energy balance that is not solely the result of the dietary component changes from the reference diet.
Figure 2: Component A of Figure 2 from Springmann et al., (2018). This figure shows the proportion of attributable mortality prevented by each aspect of the various diets examined in this paper.
As explained earlier, our replication of the USA results suggests that the conclusion that the reference diet has benefits beyond the benefits of caloric restriction is no longer valid once all uncertainties are properly included.
In his reply, Willet mentions that the three methods used to calculate preventable deaths are published “with detailed methodology”. However, neither the EAT-Lancet report – nor two out of the three publications he cited as providing the detailed methods – described a systematic approach to select literature or adhere to the GATHER statement as required by The Lancet4, to estimate the health effects of the reference diet. Although the Global Burden of Disease version adheres to these principles, it evaluates mortalities prevented by the GBD optimal intakes, which are different from the reference diet as presented by the EAT-Lancet report. Consequently, while it is true that the methods rely on “many published meta-analyses that were the basis for the relative risks”, nowhere is explained how those meta-analyses were selected. For example, no reasoning is provided for selecting Chan et al. (2011)5 over five more recent meta-analyses on red meat consumption and colorectal cancer6, 7,8,9,10. As the epidemiological evidence on dietary health effects is often weak and inconsistent11, knowing how the literature was selected is critical to our understanding of the validity of this work.
Figure 3: Table 1 from Feskens et al. (2013). RR for red meat and diabetes used by Springmann et al (2018) and the appropriate unprocessed red meat RR.
Willet also argues that the comparable estimated avoided mortalities between the three methods points to the robustness of their calculations, but in fact those methods included different types of health impacts (e.g. the Alternative Healthy Eating Index (AHEI) considers respiratory and neurodegenerative illnesses, which the other two do not). A critical example of this mismatch is that the reference diet does not specify a recommendation for sodium,3 yet sodium is the cause of over 50% of diet-attributable mortalities in the GBD estimation,12 and none of the mortalities in the comparative risk method.2 Willett’s explanations regarding adiposity impacts in the correspondence response further highlight the difference between these methods: adiposity contributes to the majority of prevented mortalities estimated by Springmann et al. (2018), but was excluded from the other two methods.12,13 So in simpler terms, the three methods predicted similar prevention of mortalities but for widely differing risk factors. These differences call into question the veracity of Willet’s arguments on the robustness of their calculations.
As in the report, in his response Willet again reports the wrong RR estimates for the association between type-2 diabetes mellitus (T2DM) and read meat. The total red meat RR for T2DM reported by the original article cited14 was 1.13 (95% CI 1.03-1.23). Furthermore, as the reference diet does not contain processed meat, the RR for fresh red meat, not total red meat should have been used by the authors as the latter includes both fresh and processed red meats (Figure 3). Thus, the correct RR to use for red meat consumption under the reference diet is 1.15 (0.99-1.33)14, a non-significant association- instead of the 1.13 (1.03-1.23) reported (note this figure is different from the figure reported in the authors’ response). Since the correct RR is not statistically different from one (i.e. no health effect), the authors should have excluded T2DM from their calculations of avoided mortality for red meat, just as they did for coronary heart disease which has a significant association with processed but not with fresh red meat.2 Willett argues that the production of red meat means that both fresh and processed red meat would be consumed and therefore use of the total red meat RR is justifiable; this statement has no basis, particularly since the reference diet explicitly excludes processed red meat.3 Even if the results for these estimates are “similar” with the correct RR, this kind of error emphasizes the need for transparency in inputs, methodology, and results.
Willett claims that nutrition in developing countries will be improved through adherence to the diet. Yet, many individuals in low and mid-low income countries cannot adhere to current recommendations due to a variety of barriers,15, 16, 17 including economic constraints, and so it is reasonable to hypothesize they would not be able to adhere to the reference diet recommendations either. In such situations, an important strategy is backyard animal husbandry,18 but under the reference diet the consumption of animal proteins is minimal. This criticism has also been discussed at greater length by others with personal experience engaging in nutritional issues in developing countries. Reduction of animal production could make these foods less available to children and pregnant or breast-feeding mothers. Willet states that the health impacts on children are excluded from the study as a variety of reasons makes their impact on the food production system negligible; however, the effect of the food production system on those under two years old is far from negligible. While issues of obesity have increased in developing countries, the question remains as to whether the reference diet proposed by the authors is the best alternative to address all of these issues, or if there are alternatives that would also optimize maternal and child nutrition in low and mid-low income countries.19
We agree with Willet’s conclusion that unhealthy diets contribute to premature deaths, but contest the accuracy of their calculations. Willet writes that we “do not provide any counter evidence to the three published reports, which puts into question the validity of [our] critique”. We in fact contacted the authors multiple times trying to share our findings to no avail but would still be glad to compare our models to theirs. Thus, we propose they make their code and databases publicly available and we will do the same.
References
1. Zagmutt FJ, Pouzou JG, Costard S. The EAT–Lancet Commission: a flawed approach? The Lancet 2019; 394: 1140–1.
2. Springmann M, Wiebe K, Mason-D’Croz D, Sulser TB, Rayner M, Scarborough P. Health and nutritional aspects of sustainable diet strategies and their association with environmental impacts: a global modelling analysis with country-level detail. Lancet Planet Health 2018; 2: e451–61.
3. Willett W, Rockström J, Loken B, et al. Food in the Anthropocene: the EAT–Lancet Commission on healthy diets from sustainable food systems. The Lancet 2019; 393: 447–92.
4. Stevens GA, Alkema L, Black RE, et al. Guidelines for Accurate and Transparent Health Estimates Reporting: the GATHER statement. The Lancet 2016; 388: e19–23.
5. Chan DSM, Lau R, Aune D, et al. Red and Processed Meat and Colorectal Cancer Incidence: Meta-Analysis of Prospective Studies. PLoS ONE 2011; 6: e20456.
6. Alexander DD, Weed DL, Miller PE, Mohamed MA. Red Meat and Colorectal Cancer: A Quantitative Update on the State of the Epidemiologic Science. J Am Coll Nutr 2015; 34: 521–43.
7. Aune D, Chan DSM, Vieira AR, et al. Red and processed meat intake and risk of colorectal adenomas: a systematic review and meta-analysis of epidemiological studies. Cancer Causes Control 2013; 24: 611–27.
8. Schwingshackl L, Schwedhelm C, Hoffmann G, et al. Food groups and risk of colorectal cancer: Food groups and colorectal cancer. Int J Cancer 2018; 142: 1748–58.
9. Vieira AR, Abar L, Chan DSM, et al. Foods and beverages and colorectal cancer risk: a systematic review and meta-analysis of cohort studies, an update of the evidence of the WCRF-AICR Continuous Update Project. Ann Oncol 2017; 28: 1788–802.
10. Zhao Z, Feng Q, Yin Z, et al. Red and processed meat consumption and colorectal cancer risk: a systematic review and meta-analysis. Oncotarget 2017; 8: 83306–14.
11. Ioannidis, J.P.; The challenge of reforming nutritional epidemiologic research. JAMA. 2018;320(10):969-970. doi:10.1001/jama.2018.11025
12. Afshin A, Sur PJ, Fay KA, et al. Health effects of dietary risks in 195 countries, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet 2019; 393: 1958–72.
13. Wang DD, Li Y, Afshin A, et al. Global Improvement in Dietary Quality Could Lead to Substantial Reduction in Premature Death. J Nutr 2019; 149: 1065–74.
14. Feskens EJM, Sluik D, van Woudenbergh GJ. Meat Consumption, Diabetes, and Its Complications. Curr Diab Rep 2013; 13: 298–306.
15. Gee S, Vargas J, Foster AM. “We need good nutrition but we have no money to buy food”: sociocultural context, care experiences, and newborn health in two UNHCR-supported camps in South Sudan. BMC Int Health Hum Rights 2018; 18: 40.
16. Farris AR, Misyak S, O’Keefe K, VanSicklin L, Porton I. Understanding the Drivers of Food Choice and Barriers to Diet Diversity in Madagascar. J Hunger Environ Nutr 2019; : 1–13.
17. Kavle JA, Landry M. Addressing barriers to maternal nutrition in low- and middle-income countries: A review of the evidence and programme implications. Matern Child Nutr 2018; 14: e12508.
18. Thompson B, Amoroso L. Improving Diets and Nutrition: Food-based Approaches. Rome, Italy: Food and Agriculture Organization of the United Nations, 2014 http://www.fao.org/3/a-i3030e.pdf (accessed Oct 10, 2019).
19. Raymond J, Kassim N, Rose JW, Agaba M. Optimal dietary patterns designed from local foods to achieve maternal nutritional goals. BMC Public Health 2018; 18: 451.
