Possibilities and Pitfalls of Basic Income Experiments
This article briefly discusses the findings of my book
A Critical Analysis of Basic Income Experiments for Researchers, Policymakers, and Citizens
Published by Palgrave Macmillan 2018[1]
If the public debate over Universal Basic Income (UBI) is going to benefit from the many UBI experiments happening around the world, the journalists, policymakers, and citizens involved in the debate need to understand what UBI experiments can and—more importantly—what they cannot do. UBI experiments can increase our knowledge of some of UBI’s effects, but they cannot do what many journalists, citizens, and policymakers seem to expect them to. In December 2016, MIT Technology Review perfectly illustrated the common overblown expectations of UBI experiments when the ran the headline: “In 2017, We Will Find Out If a Basic Income Makes Sense.”[2] As much as we might want UBI experiments to be definitive tests of UBI’s efficacy, there is no some crucial piece of information that experiments can find and that will make the difference in most people’s decision whether to support or oppose the introduction of UBI.
Social science experiments are not like medical experiments. A vaccine trial, for example can directly address the bottom-line question of interest to just about everyone: is the vaccine is safe and effective? Researchers can conduct a random control trial (RCT) by selecting an “experimental group” of—say—1000 people who get the real vaccine and a “control group” of 1000 people who get a placebo. They observe both groups to see whether the experimental group is less likely to contract the disease and more likely to suffer medical complications. Vaccine trials are imperfect tests. RCTs might have difficulty determining whether the vaccine is truly safe and effective in the long run, among people of all ages, and for people with all possible complicating risk factors, but reporting differences between the control and experimental group in an RCT gives people useful and relatively straightforward information on the bottom-line question that interests them about vaccines. If comparison of the experimental and control groups indicates that a vaccine is both safe and effective, people should take it. If that comparison indicates it’s either unsafe or ineffective, people should not. If there is good reason to believe the test is inconclusive, researchers should conduct more tests.
We can give a UBI to an experimental group for a while and compare their behavior and life outcomes to a control group but that comparison is not a test of UBI in the sense that a vaccine trial is a test of the vaccine. If a UBI experiment is not a test, what is it? It is an indirectly and often inconclusive method of increasing our knowledge of some but not other aspects of UBI. UBI experiments cannot be decisive tests in the way vaccine trials are because they say much less about the long-term effects of a fully implemented, national UBI program than vaccine trials do about the long-term effects of a fully implemented, national vaccine program, and because to the extent RCT findings do say something about a national program, ethical disagreement affects how we evaluate those effects.
Unlike a vaccine, UBI has many effects that depend heavily on how people interact with each other in local and national markets and in nonmarket settings. The change in behavior of one randomly selected worker who receives UBI might be very different from the change in behavior of 100 million workers who all receive UBI. Employers’ reaction to the changes in behavior of one randomly selected worker with a UBI might be very different from their reaction to the changes in behavior of 100 million workers who all have UBI. The educational effects of experimental UBI on one randomly selected five-year-old child in a 3-year UBI might be very different from the educational effects on 5 million five-year-old children in a country that has a fully implemented UBI, which will be in place for their entire school career.
The things we want to know about UBI are far more complex and difficult to observe than the things we want to know about vaccines. How much will UBI raise beneficiaries’ incomes once they and other people they interact with their behavior in reaction to UBI and to each other’s reactions? How much will that increased income raise beneficiaries’ welfare—according to what measures of welfare? Will beneficiaries react to UBI in ways that make it more expensive or in ways that policymakers believe are desirable or undesirable? What is the relative ethical importance of each of these factors in evaluating UBI?
Ethical disagreement affects how we evaluate almost all of UBI’s effects. For example, if the experimental group works less than the control group is that a good thing because it empowers disadvantaged people to command better wages, to demand improved working conditions, to pursue more education, or to spend more time with their families? Or is it a bad thing because it allows them to violate some principle that non-wealthy people must work as much as possible? Empirical findings are vulnerable to spin and misuse if people use them not to help make up their minds but as ammunition to support their already considered beliefs on issues like these.
UBI experiments will produce better or more quantifiable information about some effects than others. This innocuous fact makes the experiments vulnerable to the streetlight effect—drawing attention to questions that are easier-to-answer but less important at the expense of questions that are harder-to-answer but more important. For example, RCTs might be able to say something about the short-term effects of UBI on economic insecurity, but that information does not answer the questions we really want to ask about economic security, such as whether new born children whose families will never face food or housing insecurity because of a national UBI tend to grow into heathier, better educated, happier, more productive, and more pro-social adults than children who grow up with food, housing, and other economic insecurities.
RCTs can answer the question of whether the control group works more hours than the experimental group, but they can’t answer whether employers will respond to a national UBI by offering better wages and working conditions and whether these workplace improvements will partially reverse the initial decline in labor hours. Like a bright light, UBI experiments will draw the attention of everyone—even the most rational, knowledgeable researchers—toward quantifiable numbers produced by the comparison between the control and experimental groups and away from the more important but hard-to-answer questions.
Partly because of the complexity of UBI’s effects and partly because of ethical disagreement, UBI does not lend itself to a simple, bottom-line question analogous to the question of whether vaccines are safe and effective. In the medical sense, UBI is both safe and effective. UBI will raise the incomes of net beneficiaries without holding them to any obligation to work or prove they are needy, and it will not make them break out in hives or develop any other medical complication. The central disagreement about UBI is not over the unknowns but over the ethical value of its well-known effects: is it right or wrong for a government policy to raise the incomes of low-income people whether or not they work? It is reasonable for people to have made up their minds one way or the other based on their answer to this ethical question.
Little if any empirical investigation addresses the question of whether UBI can significantly raise net-beneficiaries’ incomes whether or not they work, because there is overwhelming evidence and very little disagreement that it can; the disagreement is whether it should. Empirical investigation of the effects of UBI does little to settle that basic moral disagreement. We could ask the question, is a UBI of $X sustainable, but for most relevant levels of X, that question is in little doubt and the answer to it will only be decisive for the group of people who support a UBI of $X if it is sustainable. UBI opponents and skeptics are not by-and-large driven by the belief that proposed levels are unsustainable. If they were, the political debate would be made up of UBI supporters who disagree about how high it should be.
The above issues make it impossible for researchers to formulate a bottom-line question analogous to whether a vaccine is safe and effective or to give a definitive answer to any of the bottom-line questions of interest to people with different ethical positions. The ethical evaluation of the many tradeoffs involved and even the question of whether some effects should be considered positive or negative are in the eyes of the beholders.
That being the case, why do people conduct UBI experiments at all? People conduct UBI experiments partly for strategic political reasons and because even with all these difficulties, more knowledge is better than less. Not everyone has a strong opinion about UBI, and reasonable people who do have strong opinions remain open to changing their minds about it or about some aspect in light of new knowledge.
But information is not knowledge. More information only produces better knowledge if people understand it and its relevance. The researchers who conduct experiments do not have the ability to clear up all the potential misunderstandings discussed above. Researchers are trained to conduct experiments and report their findings to other specialists. To the extent that researchers present their findings to nonspecialists, it is usually to help them understand the research on its own terms—e.g. this is what an RCT is; this is a control group; this is an experimental group; these are the measured differences between the control and the experimental group. If laypeople’s eyes haven’t already glazed over, they almost certainly will when researchers add a bunch of caveats about the limited relevance of that comparison. As caveats get longer, journalists, policymakers, and citizens become more likely to skip to the numbers. A full understanding of the caveats is likely to leave people disappointed at how little experimental findings indicate about the actual market effects of a fully implemented, long-term, national UBI program.
Researchers could combine their experimental findings with evidence from other sources and use tools such as simulation models to translate differences between the control and experimental group into estimates of actual market outcomes. They could combine these results with yet more data and modeling to connect estimates of market outcomes to estimated answers to the various bottom-line questions relevant to people with differing ethical positions. But this would involve doing several more nonexperimental studies in addition to the UBI experiment. And their results would be driven more by the assumptions of those models and those other sources of evidence than by the actual findings of the experiment they are reporting on.
My recent book, A Critical Analysis of Basic Income Experiments for Researchers, Policymakers, and Citizens, examines the difficulty of conducting and reporting the results of UBI experiments in an effort to help researchers, policymakers, and citizens gain as much useful knowledge as they can from the information UBI experiments do provide.[3] This article is an attempt to summarize some of the most important arguments from that book.
The book discusses several general problems that virtually any UBI experiment will have to deal with: community effects, long-term effects, observer effects, the streetlight effect, the difficulty of separating the effects of the size and type of program being studied, the practical impossibility of testing a genuine UBI under most circumstances, and the problems created by using a means-tested program as an experimental approximation of UBI.[4]
Although the book does not take a position on whether people should or should not conduct UBI experiments, it discusses the scientific and strategic reasons for having or not having experiments and the record of past experiments in achieving their goals. Virtually all experiments have succeeded in gathering useful information. Not all have successfully led to a more-knowledgeable public debate over UBI. For example, experiments conducted in the 1970s were badly misunderstood, and their findings were often misused by people intentionally or unintentionally misleading the public.[5] From UBI supporters’ point of view, some experiments have been a strategic success in building the movement and others have not. Arguably, the 1970s experiments had a negative effect on the movement at the time but are having a positive effect on the movement today. Experiments conducted in Namibia and India about 10 years ago appear to have had a large positive effect in building the worldwide UBI movement. The effects of the more recent experiments remain to be seen.[6]
The book’s goal is not to criticize contemporary experiments but to offer some useful analysis for the people commissioning, designing, conducting, reporting on, and reading about them. To get the most out of an experiment, all of those people need to know what questions about UBI’s effects are important to the debate in the relevant political context, what questions about UBI’s effects are answered by the experiments, and most importantly what the experiments findings do and to not indicate about the important issues in the UBI debate. Researchers and journalists conducting and writing about current and future experiments need to understand how their finding have been misunderstood and misused to make their findings relevant to the contemporary debate.[7]
The book discusses the surprisingly complex political economy that has brought about UBI experiments as a response to a movement more interested in the immediate introduction of UBI than the experimentation with it. UBI experiments are a risky strategy for the UBI movement, but as long as UBI remains a political longshot, experiments present the possibility of shortening the odds.[8]
Anyone deciding to go forward with a UBI experiment should be aware of the inherent complexity of the material and the differences in background knowledge of the people involved. They should, therefore, also be aware that the results are vulnerable to misunderstanding and misuse, and they need to come up with strategies to increase understanding and decrease misuse as much as possible.
That’s a difficult task. The book can do no more than begin the attempt to come up with those strategies. The book recommends the following. Treat experiment(s) as a small part of the effort to answer the questions necessary to evaluate UBI as a policy proposal. It is not enough simply to explain the experiments on their own terms (what is an RCT; what is a control group; what is an experimental group, etc.) with a list of caveats on their limits. Experiments don’t have to be conducted in conjunction with other research efforts to answer all the questions about UBI, but experiments in isolation must not be presented as saying very much at all about UBI as a policy. The true value of an experiment is its small contribution to this larger effort. For nonspecialists to understand this, someone needs to help them understand the limits of experimental methods and the additional evidence that would be necessary to connect experiment findings to the things they actually want to know about a fully implemented UBI program.[9]
In addition to many more specific suggestions, the book stresses four broad strategies to help experiments enlighten the discussion of UBI.
- Work backwards from the public discussion to the experiment and then forward again. Anyone commissioning, conducting, or writing about experiments should respect the national or regional discussion of UBI. Find out what people most want to know. Design a study oriented as much as possible toward the questions that are important to that discussion with careful attention to the extent to which experiments can and cannot contribute relevant evidence and the extent to which nonexperimental data and modeling can help.[10]
- Focus on the effects rather than the side effects of UBI. The streetlight effect has led past experiments to focused on quantifiable side-effects, such as labor effort and cost at the expense of more important but less quantifiable issues, such as whether UBI has the positive effects on people’s long-term wellbeing supporters predict.[11]
- Focus on the bottom line. Although the public discussion varies enormously over time and place, and not everyone agrees on any one bottom line, the desire for an answer to questions is ubiquitous. Therefore, experimental reports must address how people with different ethical positions can use the results toward making an overall evaluation of UBI as a long-term, national policy. Experiments alone cannot provide enough evidence to answer a bottom-line question, but researchers can relate all of their findings to it. Citizens and policymakers often need a great deal of help to understand that relationship meaningfully.[12]
- Address the ethical controversy. Researchers cannot resolve the controversy over the ethical evaluation of UBI, nor should they try. But they do the public a disservice by ignoring it. They can better head off spin by recognizing the controversy and explaining what the findings mean to people who hold different ethical positions that are common locally and internationally.[13]
The overall cost-effectiveness of a fully implemented, national UBI is probably the closest thing to a bottom-line question relevant to people on all sides of the relevant ethical disagreements, but issue-specific bottom-line questions for any variable of interest are also relevant. [14] The book discusses claims made by supporters and opponents and tries to identify testable empirical questions about those claims. Several empirical claims that should not be ignored cannot be tested on an experimental scale. Evidence about these claims will have to come from other sources, which will have to be combined with experimental evidence to connect any experimental findings any relevant bottom-line question.[15]
Although experiments alone cannot conclusively answer any questions about a national UBI, the book identifies many claims that UBI experiments can examine if only partially, indirectly, and/or inconclusively. It discusses the implications these limitations have for conducting a study and communicating its results.[16] The book does not take a position on the question of whether UBI experiments should or should not be undertaken. That answer depends on the particularities of the local political context. The question is not whether to have an experiment. Experiments are happening right now all over the world. The question is how to learn the most from them.[17]
The book concludes with a discussion of how to work forward from the experimental results to the public discussion with the awareness of the role those claims play in the political economy of the UBI discussion so that they might be explained in ways that overcome communication barriers and reduce the problems associated with misunderstanding and misuse of experimental findings.[18]
I wish I could say this strategy fully resolves the problem, but that isn’t possible. A social science experiment is a very limited tool, and its implications are inherently difficult to understand. The effort to treat experiments as a small and incomplete part of a wider effort to answer all the important empirical issues about UBI will help but won’t eliminate misunderstanding.[19]
There will always be gaps in understanding between the people involved in the discussion of such a complex issue and such complex evidence. If a nonspecialist learns everything specialists know, they become a specialist. But experimentation and communication can always be improved. I hope this book, this summary article, and this special issue make a small contribution to that effort.
[1] This article summarizes and draws heavily on the book, A Critical Analysis of Basic Income Experiments for Researchers, Policymakers, and Citizens, Karl Widerquist, Palgrave Macmillan 2018. I summarized that book very differently in the article, “The Devil’s in the Caveats: A Brief Discussion of the Difficulties of Basic Income Experiments,” Karl Widerquist, CESifo Forum 19 (3), September 2018, 30-35.
[2] Jamie Condliffe, “In 2017, We Will Find out If a Basic Income Makes Sense,” MIT Technology Review, December 19 2016.
[3] Widerquist, A Critical Analysis of Basic Income Experiments.
[4] Widerquist, A Critical Analysis of Basic Income Experiments pp. 19-42.
[5] Karl Widerquist, 2005. “A Failure to Communicate: What (if Anything) Can We Learn From the Negative Income Tax Experiments?” the Journal of Socio-Economics 34 (1): 49–81; Widerquist, A Critical Analysis of Basic Income Experiments, pp. 43-56.
[6] Widerquist, A Critical Analysis of Basic Income Experiments, pp. 57-70.
A translation in Chinese can be found here.
