research institute for compassionate economics

Proof, policy, and patience

Written by Dean Spears on March 11th, 2014

Without much additional comment, I would like to invite you to read Wolf-Peter Schmidt’s important recent essay: The elusive effect of water and sanitation on the global burden of disease, online for free at http://onlinelibrary.wiley.com/doi/10.1111/tmi.12286/full

Schmidt observes that sanitation randomized experiments may be “almost impossible in settings where they are most needed,” and that the literature “only represents results from those trials and studies that are feasible — they would not be there, otherwise.” This is a significant observation: even replications in, for example, new states or new countries cannot solve this problem if the most difficult places are always excluded, within each new context. Jeff Hammer and I have been reflecting on this problem in our own experimental study of sanitation in Maharashtra: although the experiment was originally planned to occur in three districts, ultimately the government only attempted the experiment in the richest district with the most human development and highest female literacy.

More broadly, we now have a growing set of randomized experiments (from many research groups and authorship teams, including ours) that show at best small “first stage” effects on open defecation. This highlights the challenge of sanitation behavior change. However, it would hardly be informative about the effect of sanitation on human capital if not improving sanitation does not improve health!

So, what are we to do? Keep trying, of course, to do better research. But what shall we tell the minister? Personally, I believe there is more than enough evidence — converging from many methods of research — that widespread open defecation in densely populated north India is a human development emergency.

Schmidt’s conclusion: “For now, accepting the often fatal methodological flaws in quantifying health effects of water and sanitation may be an intellectual challenge, but perhaps a necessary step. We may at some point be forced to get out for a bit and walk through an urban slum during the wet season. The lack of high-quality trials on urban sanitation or rural water access should not stop us from opening our eyes – the oldest form of impact assessment. This may sound fantastical, but perhaps, only to the ears of a 21st century academic.”