EA - GiveWell Misuses Discount Rates by Oscar Delaney

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GiveWell Misuses Discount Rates, published by Oscar Delaney on October 27, 2022 on The Effective Altruism Forum.SummaryGiveWell currently uses a time discount rate of 4% for all their cost-effectiveness analyses (CEAs).I argue that it is a mathematical mistake to pick any single best guess value to use for the CEAs.Instead, GiveWell should use a probability distribution over possible discount rates.This is not just an aesthetic judgement for mathematical puritans; it materially changes the CEAs, notably by making all the deworming interventions more attractive relative to other interventions.This is because deworming interventions rely on multi-decadal effects, and so a lower discount rate would make them much more valuable.Epistemic StatusOn the object level, I cannot think of any reasons to justify GiveWell's current modelling choice over my proposal.However, I still doubt my conclusion because on the meta level it seems like an obvious thing that would be surprising if no one at GiveWell had ever thought of doing, which is evidence I am missing something important.MainGiveWell’s CEAs are an impressive attempt to model many different factors in assessing the near-term impacts of various interventions.[1] I will ignore all of this complexity. For my purposes, it is sufficient to note that the CEA for most interventions is well characterised by decomposing impact into several constituents, and multiplying these numbers together. Consider Helen Keller International’s Vitamin A Supplementation program:V=M×R×1C [2] where:V is cost-effectiveness [deaths/dollar],M is baseline mortality [deaths/year/child],R is mortality reduction [%], andC is treatment cost [dollars/child/year]Obviously, all of these terms are uncertain. Treatment costs we can estimate quite accurately, but there may be fluctuations in the price of labour or materials needed in the distribution. Mortality data is generally good, but some deaths may not be reported, and mortality rates will change over time. The mortality reduction is based on a solid-seeming meta-analysis of RCTs, but things change over time, and circumstances differ between the trial and intervention locations.GiveWell’s model makes a subtle mathematical assumption, namely that the expectation of the product of these three random variables is equal to the product of their expectations:E[V]=E[M×R×1C]=E[M]×E[R]×E[1C]This is not, in general, true.[3] However, if the three random variables are independent, it is true. I cannot think of any plausible ways in which these three random variables correlate. Surely learning that the price of vitamin A tablets just doubled (C) does not affect how effective they are (R) or change the baseline of how many kids die (M). Thus, while GiveWell’s method is mathematically unsound, it gives the correct answer in this case. It could well be that GiveWell has considered this, and decided not to explain this in their CEAs because it doesn’t change the answer. I think this would be a mistake in communication, but otherwise benign.The one place where I believe this mathematical mistake translates into an incorrect answer is in the use of discount rates. From GiveWell’s explanatory document:“The discount rate's primary effect in the cost-effectiveness analyses of our top charities is to represent how much we discount increases in consumption resulting from the long run effects of improved child health for our malaria, deworming and vitamin A charities (which we call "developmental effects"). It also affects the longer-run benefits from cash transfers. We don't discount mortality benefits in our cost-effectiveness analyses.”This figure shows the cost-effectiveness of all the charities in the CEA spreadsheet, when varying the discount rate.[4]Deworming interventions, shown in dashed lines, v...