If you think of one’s opinion about RCT’s as a spectrum, I fall on the “I think it’s not a bad idea at all” part of it. How might I be wrong? Five articles that help me understand this.
- “Lately I find myself cringing at the question “what works in development?” I think it’s a mistake to think that way. That is why I now try hard not to talk in terms of “program evaluation”.“Does it work?” is how I approached at least two of the studies. One example: Would a few months of agricultural skills training coax a bunch of ex-combatants out of illegal gold mining, settle them in villages, and make it less likely they join the next mercenary movement that forms?
But instead of asking, “does the program work?”, I should have asked, “How does the world work?” What we want is a reasonably accurate model of the world: why people or communities or institutions behave the way they do, and how they will respond to an incentive, or a constraint relieved. Randomized trials, designed right, can help move us to better models.”
..
..
Chris Blattman on the issue. (Note that this was written in 2016)
..
.. - “In the early 2000s a group emerged arguing that important improvements to development and hence to human well-being could be achieved through the wide spread use of independent impact evaluations of development programs and projects using randomized control trial methods (RCT) of choosing randomly “treatment” and “control” individuals. I have been arguing, since about that time, that this argument for RCT in IIE gets one small thing right (that it is hard to recover methodologically sound estimates of project/program causal impact with non-experimental methods) but all the big things wrong.”
..
..
You can’t write anything about RCT’s without writing about Lant Pritchett’s opinion about them.
..
.. - “Like other methods of investigation, they are often useful, and, like other methods, they have dangers and drawbacks. Methodological prejudice can only tie our hands. Context is always important, and we must adapt our methods to the problem at hand. It is not true that an RCT, when feasible, will always do better than an observational study. This should not be controversial, but my reading of the rhetoric in the literature suggests that the following statements might still make some uncomfortable, particularly the second: (a) RCTs are affected by the same problems of inference and estimation that economists have faced using other methods, and (b) no RCT can ever legitimately claim to have established causality.”
..
..
Angus Deaton weighs in (and if you ask me, this is my favorite out of the five)
..
.. - “The economists, like the medical researchers, seem to have lost touch
with their proper role. They are not ethically assigned to master our lives.
The mastering assignment is what they assume when they focus on
“policy,” understood as tricking or bribing or coercing people to do what’s
best. It sounds fine, until you realize that it is what your mother did to you
when you were 2 years old, and had properly stopped doing to you by the
time you were 21. The field experimenters scorn adult liberty. And that is
the other way many economists have lost touch. As noted by the
economist William Easterly, another critic of the experimental work, and as
argued at length by your reporter in numerous books, the real way to solve
world poverty is liberty. Not dubious, fiddly, bossy little policies handed
down from the elite. ”
..
..
Dierdre McCloskey (as usual) doesn’t pull punches.
..
.. - A set of links about the topic from Oxfam.