Miracles and methods
I once saw a miracle. It was near a ragged wall of broken rocks near Udaipur. One side of the wall had patches of yellowing grass, not taller than an inch, with the occasional dry stalk. The other side, protected from goats by the wall, was a sea of green and yellow grass, up to a couple of feet tall. Here was something utterly simple that had the power to bring life back to these barren hillsides.
Or so I thought. It took a friend, familiar with the area, less than five minutes to set me straight. I was forgetting, he observed, all the grass from the unwalled side that had already fed some passing goat, and all the grass that someone had cut and sold. If we added up all the grass that had grown on the unwalled side, while the grass on the other side grew tall, we would probably end up with close to what was there across the wall. We might still support building the wall on the ground that tall grass is good for soil conservation and water retention, but we should not be expecting a miracle.
It is easy, especially if you are an enthusiast, to find miracles where there are none. Assessing benefits of an intervention, as I learnt the hard way, requires thinking hard about the appropriate metric and it is easy to get it wrong. For example, as Jonathan Murdoch has shown, the Grameen Bank of Bangladesh has shown the way in profitable lending to the poor, in part by counting its fresh deposits as a part of its earning. But it has forgotten that these deposits would eventually need to be repaid.
If measuring benefits is hard, measuring costs can be a nightmare. You have an innovative programme that involves the community in managing the local greenery and it is quite apparent that your programme is doing wonderful things for the environment. But you also know that you have a lot of charisma and suspect that it might have a lot to do with the way the community has involved itself in your programme. How do you, short of cloning yourself, even start to come up with the cost of reproducing the programme elsewhere? Making the programme scaleable from the beginning is probably the best way to avoid this problem, but it does impose constraints on the process of innovation.
Once you have solved the problem of measuring costs and benefits, you hit the really hard problem