Evidence and results wonkwar final salvo (for now): Eyben and Roche respond to Whitty and Dercon + your chance to vote

In this final post (Chris Whitty and Stefan Dercon have opted not to write a second installment), Rosalind Eyben and

Chris Roche reply to their critics. And now is your chance to vote (right) – but only if you’ve read all three posts, please. The comments on this have been brilliant, and I may well repost some next week, when I’ve had a chance to process.

Let’s start with what we seem to agree upon:

• Unhappiness with ‘experts’ – or at least the kind that pat you patronizingly on the arm,
• The importance of understanding context and politics,
• Power and political institutions are generally biased against the poor,
• We don’t know much about the ability of aid agencies to influence transformational change,
• Mixed methods approaches to producing ‘evidence’ are important. And, importantly,
• We are all often wrong!

We suggest the principal difference between us seems to concern our assumptions about: how different kinds of change happen; what we can know about change processes; if how and when evidence from one intervention can practically be taken and sensibly used in another; and how institutional and political contexts then determine how evidence is then used in practice. This set of assumptions has fundamental importance for international development practice.

Firstly, we understand social change to be emergent and messy. Organised efforts to direct change confront the impossibility of any of us ever having a total understanding of all the sets of societal relationships and contested meanings that generate change and are in constant flux. New inter-relational processes are constantly being generated that in turn affect and change those already in existence. Complexity theory privileges a concern for process as much as goals and supports an approach that seeks to make a difference by working through relationships rather than focusing on narrowly defined pre-set projects and outcomes. It encourages being explicit about values and a concern for how an organisation’s intervention is judged by others, in particular by those that are meant to ultimately benefit, and the creation of effective feedback mechanisms – including, but not limited to, those produced by high quality research.

At their best, development practitioners often have to surf the unpredictable realities of national politics, spotting opportunities supporting interesting new initiatives, acting like entrepreneurs or searchers, rather than planners. They are keeping their eye on processes and looking to ride those waves that appear to be heading in the direction that matches their own agencies’ mission and values, and which can support local coalitions for change.  On the contrary, assuming that development practitioners are in control and that change is predictable – as expressed through some of the demands of evidence-based planning approaches – prevent them from responding effectively to feedback in an often unpredictable and dynamic policy environment, and can, if badly managed, chain them to a desk. Ben Ramalingam’s blog site – Aid on the Edge of Chaos – offers current insights on complexity thinking in development.

That it is relatively easier to eradicate rinderpest in cattle and build bridges than tackle police corruption or reduce violence against women is because the first are examples of what Dave Snowden describes as complicated problems and the latter are complex – an effect of there being so many collaborators involved in non-routine interventions with absence of consensus among them.  Such issues can’t be ‘solved’ like a Sudoku puzzle. In that respect, we were puzzled by Chris and Stefan’s two examples of what we would describe as complex issues. We found the first – the effect of political quotas for women in rural India – to be somewhat superficial and wondered why so little reference was made to the considerable number of studies from political sociology on the same topic that ask more probing questions and arguably provide more insightful understanding of what has been learnt in different contexts.  The World Bank study on whether top-down large scale interventions can stimulate bottom-up participation was on the other hand  puzzling for exposing myths that perhaps only World Bank staff had previously believed in, while ignoring the very considerable body of sociological and anthropological knowledge on this topic. It led us to wondering whether you need economists to find something out for it to be accepted as evidence.  Perhaps that explains some of ‘the evidence-barren areas in development’………

Which brings us to the second set of assumptions about how we know and therefore what is judged as evidence.  This is about more than pluralism and mixed methods, though we recognise that recent advances, in this case funded by DFID, are important.  Let’s start by insisting that a criterion for rigorous research is that it should be explicit about its assumptions or world-view. We suggest that a weakness in many studies is that they usually focus solely on the methodological and procedural and render invisible their ‘philosophical plumbing’. The evidence-based approaches that Stefan and Chris advocate are imposing a certain view of the world, just as our approaches do. Their claims to the contrary foreclose any possible discussion about the different intellectual traditions in interpreting reality.  Theory invites argument and debate.

An interesting paper by Greenhalg and Russell on evaluating health programmes notes how experimental approaches often ignore thetricky philosophical and political questions. Like the authors of that article, we take an approach that recognizes the partial (in both senses of the word) nature of our knowledge. How does this approach try to deal with unavoidable bias?  Through seeking to use dialogic, democratic methods in which multiple perspectives and understandings of what is at stake are explored, and the use of multiple and hybrid approaches.  The implications for practice are to be involved in mutual single and double-loop learning and adaptation as you go along. This does not preclude specific studies commissioned from ‘experts’, but it is not they alone who should define the problem nor should they assume that only their kind of knowledge has validity for collective efforts to try to secure greater equity and social justice.  Knowledge and power are bed-mates.  Our critique of ‘expertise’ – the laboratory references are an extreme example of the trend – is that expertise often uses its power to ignore other ways of knowing and doing, something Chris and Stefan would seem to agree with. Might it be that some of these ways might prove to be pretty good at tackling police corruption or reducing violence against women?

This is where reflexivity comes in.  Those of us working as practitioners, bureaucrats and scholar activists in international development cannot escape the contradiction that we are strategizing for social transformation from a position in a global institution – international development – that can and does sustain inequitable power relations, as much as it succeeds in changing them. Reflexive practice seeks to address these power inequities by recognizing that (a) many problems we seek to address are the products of human interaction – and some very important problems for people with less voice go ignored for that reason, and  (b) even if people are in agreement about there being a problem, they will often offer multiple diagnoses for its existence, and thus of course (c) multiple solutions, which need to be debated democratically with different kinds of evidence, based on alternative ways of knowing, and having the space to be heard.

We are heartened to note that Chris and Stefan believe “that all actions by external actors will interact with political forces and vested interests” and that “in many of the settings where development actors want to make a difference, power and political institutions are biased against the poor”. We would therefore assume that a reflexive donor would recognise that their power and agenda need examination as much as anyone else’s.

Chris and Stefan suggest ‘the commitment to evidence has opened up the space fundamentally to challenge conventional, technical approaches to aid.’ We would agree, but it would seem that the exception to this is when it comes to addressing the power of donors such as DFID, being honest about the domestic political pressures they are under, and assessing the possibility that their behaviour (including how evidence-based approaches are managerialised) may on occasions be undermining processes of development and social transformation. Is DFID drawing upon anthropologists or ethnographic researchers, as the Police in the UK have recently done, to understand how its policies on, for example, results or value for money change behaviour in the agency, and its relationships with others?

To imply that we are suggesting that ‘it is not worth trying to provide the best and most rigorous evidence to those who need to make difficult decisions’ is simply a wilful mis-stating of our position. On the contrary we are arguing there is more ‘evidence’ out there than some seem to admit because their world view precludes seeing this as such. Where we in particular see the need for more evidence is about how the evidence-based and results agenda plays out in practice. How it affects the behaviour of development agencies and their staff as well as their ability to support the promotion of the kinds of transformational change which are likely to make a significant difference to the lives of people living in poverty and injustice. It is odd that those that argue for more evidence seem rather reluctant to admit that this is needed!

This is a debate we are keen to pursue further in the upcoming Big Push Forward conference on the Politics of Evidence.

The evidence debate continues: Chris Whitty and Stefan Dercon respond from DFID

Yesterday Chris Roche and Rosalind Eyben set out their concerns over the results agenda. Today Chris Whitty (left), DFID’s Director of Research and Evidence and Chief Scientific Adviser and Stefan Dercon (right), its Chief Economist, respond.

It is common ground that “No-one really believes that it is feasible for external development assistance to consist purely of ‘technical’ interventions.” Neither would anyone argue that power, politics and ideology are not central to policy and indeed day-to-day decisions. Much of the rest of yesterday’s passionate blog by Rosalind Eyben and Chris Roche sets up a series of straw men, presenting a supposed case for evidence-based approaches that is far removed from reality and in places borders on the sinister, with its implication that this is some coming together of scientists in laboratories experimenting on Africans, 1930s colonialism, and money-pinching government truth-junkies. Whilst this may work as polemic, the logical and factual base of the blog is less strong.

Rosalind and Chris start with evidence-based medicine, so let’s start in the same place. One of us (CW) started training as the last senior doctors to oppose evidence-based medicine were nearing retirement. ‘My boy’ they would say, generally with a slightly patronising pat on the arm, ‘this evidence-based medicine fad won’t last. Every patient is different, every family situation is unique; how can you generalise from a mass of data to the complexity of the human situation.” Fortunately they lost that argument. As evidence-informed approaches supplanted expert opinion the likelihood of dying from a heart attack dropped by 40% over 10 years, and the research tools which achieved this (of which randomised trials are only one) are now being used to address the problems of health and poverty in Africa and Asia.

The consequences of moving from expert (ie opinion-based, seniority-based and anecdote-based) to evidence-based healthcare policy, far from being some sinister neocolonial experiment, have been spectacular. To quote a recent Economist headline, ‘Africa is currently experiencing some of the fastest falls in childhood mortality ever seen, anywhere’. It is a great example of the positive side to modern Africa the current excellent Oxfam publicity campaign (right) is all about. This success is based on many small bits of evidence, from many disciplines, leading to multiple incrementally better interventions. Critically, it also involves stopping doing things which the expert consensus agreed should work, but which when tested do not. It is no accident that one of the most evidence-based parts of development is also one where development efforts have had some of their greatest successes.

Proper evidence empowers the decision-maker to be able to make better choices. This is a good thing. In every discipline, in every country, where rigorous testing of the solutions of experts has started, many ways of doing things promoted by serious and intelligent people with years of experience have been shown not to work. International development is no different, except that the communities we seek to assist are more vulnerable, including to our bad choices.

Much of what we all do in international development has very limited evidence that it does any good  (in this it is no different from many other policy areas) – which is not the same as saying it is pointless. Rather we don’t know what is pointless. Some of our actions will work better than we think, much of it will work much less well than we hope, and some of it will be damaging the poorest without us realising it. In the evidence-light areas we just don’t know which are which.

We must have the humility to accept that we are all often wrong, however reflexive the practitioner, however deep their reading and experience and passion to do good. Evidence-based approaches are not about imposing a particular theory or view of the world. It is simply about taking any opportunity to test our own solutions in the best way available, using evidence honestly when it is available to inform (note the word) decisions, and when the facts change, changing our minds.

This honesty includes saying to decision-makers when evidence is methodologically weak, mixed or missing so they know they are on their own, unable to rely on (or make a claim on) the evidence. The worst possible solution, which we know Chris and Ros would also deplore, is using the social power of the ‘expert’ to imply we know the answer when we actually have no solid evidential basis for our opinion or prejudice.

A few false assumptions about evidence-based decision making

Some of those who express unease about evidence-based policy and practice seem to assume that it is always based on randomised trials and quantitative methodologies: not so. Methods from all disciplines, qualitative and quantitative, are needed, with the mix depending on the context. Randomised trials are one tool amongst very many, although a good one in the right setting. The argument that evidence-based approaches can “only apply in cases of individual treatment and not the wider community level” ignores over 30 years of methodology which has done exactly that, with very convincing results.

A sterile argument  between people who are on the one side believe that a  randomised trial can answer any question (they can’t), and people who do not appear to be aware of any  methodological advances since the 1970s except in their own narrow field is a depressingly familiar experience. We know this does not apply to Rosalind and Chris, but listening to people passionately critiquing methodologies they have not taken the trouble to understand does no good to anyone. This applies both to a randomista who seems to believe that all there is to social research is a few focus groups and in-depth interviews, and to people from a more qualitative social science background who would have trouble explaining the difference between cluster randomised and step-wedge design but assume both are irrelevant to social research anyway (both can be used to measure societal rather than individual effects).

It is tempting to take every point the authors make where we have concerns about their factual basis and logical framework but we will take just three.

“Evidence-based approaches are pre-occupied with avoiding bias and increasing the precision of estimates of effect”. On less bias – generally true. Please complete the sentence ‘More biased research is better because…’. On precision – no, incorrect, the range of situations where a more precise answer is a better answer is small.

One statement we would like to address head-on starts “Evidence-based approaches became linked to value for money concerns to deliver ‘results’…”. We agree- and this is a good thing. Doing a pointless thing, professionally delivered and passionately believed in, is always going to be poor value for money. Testing what works and what does not therefore is essential to value for money. More importantly, doing pointless things diverts very limited human and financial resources, in an ocean of need, away from those who could best use them- not what any of us are in international development to do.

Is it “technical approaches” on the one hand, and “power, political economy” analysis on the other?

Rosalind and Chris’ key criticism is that evidence-based approaches “deflect attention from the centrality of power [and] politics […] in shaping society”, and they offer “power analyses” as an apparent alternative to assessing rigorously what works. This creates a false dichotomy, as if a choice has to be made between a “technical, rational and scientific approach to development” and an approach that recognises politics and the role of power. It is easy rhetoric, but troubling and, if taken much further, even dangerous. Understanding power and politics and how to assist in social change also require careful and rigorous evidence, and again, results are not simply what experts would have expected a priori. Recent studies on the positive impacts of female leadership quotas in rural India are for many of us rather surprisingly good news, even if one can fairly worry about its applicability in other settings, while the struggle to find systematically a positive impact of decentralisation and community-driven development programmes is important to internalise in our actions for change, and highlights the importance of understanding contexts and politics. In these cases, it is not a matter of just RCTs, but of rigour, and of combining appropriate methods, including more qualitative and political economy analysis.

Strong analysis of politics and power without offering much in terms of what can be acted upon is similarly unhelpful. They criticise an evidence-focused agenda by stating that “to act ‘technically’ in a politically complex context can make external actors pawns of more powerful vested interests and therefore by default makes them, albeit unintentionally, political actors.” But all actions by external actors will interact with political forces and vested interests. In many of the settings where development actors want to make a difference, power and political institutions are biased against the poor. Being able to act on strong evidence of what works in constrained political settings is crucial.

A reductionist and misinformed view of evidence as purely ‘technical’ or as being only about “what works” is unhelpful – it is also about generating evidence and understanding (and learning) on why interventions and approaches may work, including understanding the social, political, and economic factors that may enable or constrain success of different approaches. Far from the search for evidence pushing us in a ‘technical’, apolitical direction it has reinforced the importance of understanding and trying to tackle the underlying causes of poverty and conflict. There is agreement on the importance of politics and institutions in shaping growth, security and human development. However, the ability of external actors to influence institutions is much less clear and this is where DFID research is now focussed. Ros and Chris have misread the context – the commitment to evidence has opened up the space fundamentally to challenge conventional, technical approaches to aid.

Why it matters for international development

There are large areas of international development where decision-makers are largely flying blind – forced to make decisions purely on gut feeling and ideology not because they wish to because they have no option. Try making difficult decisions in education policy compared to health policy and the difference in usable evidence is dramatic – yet both are complex, social and context-dependent parts of human life. It is always puzzling when people say airily ‘health is easy’- it is not, and is an intensely political and social subject requiring interventions at societal level.

Today we can eradicate rinderpest in cattle and build bridges over the Zambezi based on rock-solid evidence from many disciplines, but do not have anywhere near as clear an idea how to reduce violence against women or tackle police corruption. All are great challenges with social dimensions but in two of them people have set about finding and testing solutions in a systematic way over many decades.

Having robustly tested evidence-based solutions certainly does not eliminate politics: the decision whether to build a bridge, what sort and where, is an intensely political choice – but at least those making the choice now have a fair assumption it will stand up- based on hundreds of years of incremental evidence. The evidence-barren areas in development are a collective, and in our view shameful, failure by us all in the academic and practitioner community. We should never excuse them with the feeble assertion that it is too difficult or complicated. Development is difficult and complicated – but the bases for making decisions will gradually improve if we are serious about improving it.

In conclusion, we collectively have the capacity to be able to give to our successors in every continent a far better basis on which to make their decisions for their lives than our generation have. To imply it is not worth trying to provide the best and most rigorous evidence to those who need to make difficult decisions because they will have other influences as well is like saying to someone going for a walk in dangerous mountains that they do not need a map because there will be many other factors that will determine where they go. That is true – but they are still less likely to fall off the cliff if they have one.

Where evidence is clear-cut we should be making that plain to decision makers – and where it is not we should say that as well, be honest about what is there and try to get better evidence for the future. That, in essence, is what evidence-based decision making is about – and all it is about. If the academic community is serious about trying to assist those working in the field (including in Oxfam), and above all empowering the most vulnerable communities to make the most informed possible decisions available for their own development, we should be putting our greatest efforts into supporting decision-makers to use the best evidence, and finding better methodologies in areas where we currently have very weak evidence. There are many, and this should be tackled as a matter of urgency.

Tomorrow, Chris Roche and Rosalind Eyben respond

The political implications of evidence-based approaches (aka start of this week’s wonkwar on the results agenda)

The debate on evidence and results continues to rage. Rosalind Eyben (left) and Chris Roche (right, dressed for battle), two of the organisers of April’s Big Push Forward conference on the Politics of  Evidence, kick off a discussion. Tomorrow Chris Whitty, DFID’s Director of Research and Evidence and Chief Scientific Adviser, and Stefan Dercon, its Chief Economist, respond

Distinct from its more general usage of what is observed or experienced, ‘evidence’ has acquired a particular meaning relating to proof about ‘what works’, particularly through robust evidence from rigorous experimental trials. But no-one really believes that it is feasible for external development assistance to consist purely of ‘technical’ interventions. Most development workers do not see themselves as scientists in a laboratory, but more as reflective practitioners seeking to learn how to support locally generated transformative processes for greater equity and social justice. Where have these experimental approaches come from and what is at stake?

The origins and critiques of evidence-based approaches

Evidence-based approaches are pre-occupied with avoiding bias and increasing the precision of estimates of effect. In the UK they spread beyond clinical practice when the government elected in 1997 was keen to demonstrate that its decisions would not be driven by political ideology but rather by objective evidence. Evidence-based approaches became linked to value for money concerns to deliver ‘results’ as efficiently and effectively as possible, by a government recently described as ‘truth junkies’.

Yet, even within medicine, the leap from evidence-based clinical practice into evidence-based policy was challenged. A British Medical Journal article by Nick Black in 2001 drew on an extensive body of contemporary literature on policy processes to argue that policy was shaped by institutional arrangements, values and beliefs and a variety of different sources of information.

Opponents of evidence-based education critiqued its positivist assumptions; its linear cause-effect thinking; and the poor understanding of the tensions between scientific and democratic control of educational practice. An OECD report in 2007 on the reasons for the uptake of evidence-based education in Sweden, the UK, Canada, the USA and Australia, noted the increasing pressure for greater accountability of expenditure and effectiveness, an explosion in the search for measurable outcomes, and demands that impacts and effectiveness be given a monetary value.

The same report noted that evidence-based approaches were largely absent in OECD countries ‘less used to empirical and quantitative methodologies in the social sciences.’ The de-politicization of policy making is one of the reasons given by development researchers for its neglect in France.

In UK social policy, evidence-based approaches with their ‘gold standard’ of experimental or quasi-experimental design, have been criticised as being inapplicable to complex issues. In What Works, Tony Harrison argued that evidence-based approaches can only apply in cases of individual treatment and not at the wider community level where multiple perspectives come into play and no agreement exists about the nature of the problem.This of course is the case with most development programmes, and in particular those that seek transformational change.

Evidence based approaches in development: an anti-politics firewall?

Arguably evidence-based approaches build an anti-politics firewall. Development assistance becomes a ‘technical’ best practice intervention based on rigorous objective evidence, delivering best value for money to domestic taxpayers and recipient country citizens mostly without interfering in that country’s politics. They are the latest manifestation of  a certain long-standing approach to development that as Timothy Mitchell wrote in Power of Development, speaks to the sector’s ‘need to overlook its internal involvement in the places and problems it analyses and present itself instead as an external intelligence that stands outside the objects it describes’.

In the 1930s Africa was seen as ‘a living laboratory’ to achieve improvements in the welfare of the populations. Evidence-based approaches are reviving the development-as-laboratory idea. In 2012 the World Bank established a Gender Innovation Lab to design ‘innovative interventions to address gender inequality and to develop rigorous research projects in order to produce evidence on what works and what does not’.  Jeffrey Sachs’ Millennium Villages have been framed as  ‘laboratories to lift people out of poverty’. The most well-known is the J-Pal Poverty Action Lab whose mission is to reduce poverty ‘by ensuring that policy is based on scientific evidence’.

In the absence of political debate, this approach can exacerbate the tendency to see people as subjects requiring treatment, rather than as citizens with political voice. Power silences any challenges to the technical framing of ‘the problem’, foreclosing discussion of the structural causes and consequences of inequity and how these should be tackled. To act ‘technically’ in a politically complex context can make external actors pawns of more powerful vested interests and therefore by default makes them, albeit unintentionally, political actors.

High stakes

Evidence-based technical approaches can therefore deflect attention from the centrality of power, politics and ideology in shaping society. We agree with the view of the Developmental Leadership Program that recent research suggests that  the development sector should be ‘at the frontier of a narrative shift between a technical, rational, and scientific approach to development, and a recognition that politics matters; that poverty reduction is not a technical problem but requires significant social change, and that this social change is, and must be, both political and locally led.’

However this has some significant implications for external actors. We need to be self-aware to avoid disempowering others.  This requires undertaking power analyses with ourselves factored in – as organisations and individuals who can make a positive or negative contribution, often inadvertently.  It means engaging with a wider and more diverse group of policy actors in the state, civil society and the private sector; whenever possible, supporting debate, locally-driven problem solving, and independent research. It means avoiding overly linear project-based aid modalities that demand omniscience before they have even begun.

As Michael Sandel has recently argued in his book about the moral limits to markets, how we put values (and prices) on things can change their meaning, as well as change the relationship between economic actors.  More information, (dare we say ‘evidence’), is needed to draw some firmer conclusions about the consequences of evidence-based approaches to designing projects and assessing results. This is why the Big Push Forward is currently seeking to crowd-source more information from development practitioners about how they actually experience the ‘results’ agenda, and why we believe this issue needs more debate.

And make sure you come back tomorrow for DFID’s counterblast

Should men boycott all-male panels at conferences?

A conversation on twitter this weekend triggered (yet another) ethical dilemma. Gosh it’s exhausting trying to be a do-gooder. Claire Melamed started it by sending round a link to an article arguing that men should sign a pledge stating publicly that they will refuse to take part in all-male panels at tech conferences (which are regularly men-only affairs, apparently). As a regular token NGO speaker at various talkshops, would I make a similar pledge, she asked? Owen Barder is already signed up, she added.

They may not be as extreme as geeky tech events, but lots of development gabfests do indeed feature men on the panel talking to women (and men) in the audience. That violates basic fairness, inhibits the profile and (possibly) career development of half of the potential talent pool, and is likely to distort the agenda and resulting discussion (less focus on care economy, women’s rights etc). So obviously, the answer is yes to a boycott, right?

Except…..

Most people who contact me don’t know the final panel line-up yet. They are in the process of contacting a range of potential speakers, both men and women. Prominent women in the development debate (like Claire and her outgoing boss at ODI Alison Evans) are in huge demand, so presumably have to say no quite a lot of the time. Should I say ‘provisionally yes, but if you end up with a male-only line-up, I’ll withdraw at the last minute’? That seems to me to cross the line from principled to prima donna – pretty unfair on already stressed-out conference organisers who may be trying ever so hard to ensure a balanced line up. Or should I say ‘are you committed to inviting a decent number of women speakers to ensure a gender balance on your panels?’ – everyone is going to say yes, but how do you measure how serious they are?

Then of course there’s the organisational profile thing. In fantasy mode, suppose I get a call saying ‘Barack Obama, David Cameron and Jim Kim are speaking on development, and need a token NGO person, could you do it? Christine Lagarde is busy that day, sorry.’ Am I really going to say no?

And what about a panel with all male speakers and a woman chair (a pretty common occurrence)?

And why privilege gender over eg ethnicity – what about all-white panels on development (which are even more common than all-male ones)?

Oh dear. The torments of the self-obsessed liberal.

Tell me what you think, and depending on the response, I may well set up another online poll to help solve my dilemma. Meanwhile, the interns poll is still getting votes (see right), and the agnostics (NGOs should decide for themselves whether to pay interns) has overtaken the ‘pay all interns’ lobby and is drawing away. Unexpected result – love it.

Calling all Euro/Aid sceptics – here’s some top quality aid from the EU

Oxfam programme researcher John Magrath (he’s the one on the left in the pic) has been looking at some European aid, and is impressed with what he found

European Aid gets a lousy press. If you’re a reader of the UK’s Daily Mail (and nearly 2 million Brits are) then you’ll be used to headlines such as “Dance lessons in Africa, jets for tyrants, derelict offices….how EU wastes aid billions“.

(I must pause this blog to say that merely checking the reality of the above headline on the Daily Mail’s website – where it is accompanied by a photo of a masked figure of what Westerners used to call an African “witch doctor” – has made me want to devote the rest of this post to a rant about the Daily Mail’s politics and attitudes….but…deep breath and back to EU aid…)

EU aid is currently under greater threat than ever in Europe’s acrimonious budget discussions – partly, it’s a useful way for Euro sceptics to attack the whole concept of the European Union and the list of anti-Europe stories (remember bent (or was it straight?) bananas etc).

Does EU aid deserve its reputation? DFID doesn’t think so, giving it good marks in its multilateral aid review. So it’s rather nice to be able to point to an EU-funded aid programme that really has done a lot of good – and indeed, is ahead of the game when it comes to thinking about what sort of aid works best. It didn’t make the headlines, but then, good news rarely does.

The European Union Food Facility (EUFF) was a €1bn fund for access to agricultural inputs and services, and improvements in agricultural productive capacity, set up in response to the increase in food prices from late 2007 through 2008.

In all, the EUFF funded over 240 projects in more than 50 countries. On 17 December, the EU published its own Final Evaluation of all the programmes. Oxfam took part in a big way in six EUFF programmes in Nepal, Pakistan, Ethiopia, Eritrea, Tanzania and Mali, as well as being part of consortia in Liberia, Sierra Leone and Kenya. Coincidentally, I’ve been looking through the end of programme evaluations we submitted and today we’ve published our own case study.

And what did we find? That the bold claims made for the EUFF in a press release on 2 December 2011 by WFP, FAO and IFAD seem correct. Their release said: “The EU Food Facility has been a tremendous success. It proves that linking relief, rehabilitation and development can have a concrete impact on people’s food security”. They added it “provided tangible evidence that investing in agriculture and nutrition improves global food security” and “by linking farmers to markets and financial services, assisting in facilitating sustainable and profitable farming practices and creating new revenue streams, the effects of the EUFF will continue into their futures”.

Indian seed fair

According to the release, “lessons learned from the initiative underscore the importance of:
-focusing on marginalized farmers with high production potential,
-combining input distribution with extension services,
-building capacities of smallholder farmers and their communities,
-rehabilitating rural infrastructures, and
-involving all actors of the value chain in local seed production.”

In Oxfam’s experience this approach really broke down the silos between humanitarian and long term development aid. In every country farmers faced the same or very similar obstacles. They needed certain basic inputs, in particular seeds and irrigation water; they required both access to markets and ‘power in markets’; they needed cash and/or credit; and they wanted services such as agricultural advice and veterinary help. So the vision and capacity to make multiple interventions – and at many levels – were crucial.

For the EU this approach has been rather a long time coming – about three years in fact -  but it’s nearly there. The EU’s 2010 Food Security policy aims to get all member states working this way, then this October the EU focused on “building resilience” and integrating humanitarian and development work . And now, finally, the long-awaited implementation plan for the food security policy should be endorsed by Development Ministers next May (2013) – and not a day too soon.

I summarised our interventions in this way, on a spectrum from emergency response to long term development:

Social protection

• Collective cash-for-work for infrastructure (e.g. roads, dams, irrigation works, tree planting, re-greening, etc)
• Unconditional cash transfers to meet food needs during the hungry season
• Food vouchers and support to traders
• Small livestock and veterinary services
• Beneficiary involvement, agency accountability

Inputs

• Seeds, tools, feed, micro-irrigation,  seed banks, grain stores
• Training on improved agricultural practices (e.g. composting)
• Land rights
• Access to credit
• Stoves

Organizational development

• Formal creation of producer groups; also irrigation management groups, grain store groups, pasture management groups, etc
• Capacity building for new and existing groups
• Emphasis on women’s groups and women’s involvement
• Farmer to farmer
• Scaling up and broadening out organizations

Power in markets

• Business training
• Value chain analysis, market information
• Linkages with the private sector e.g. assistance in negotiations
• Access to credit, bank loans, micro insurance

Convening and brokering

• Linkages with state authorities and service providers at local, regional and national levels
• Linkages with the private sector
• Advocacy on budgets, policy changes, frameworks, etc, from local to national level

Not every programme achieved the same level of success but generally, the results were impressive. If you’ve an antipathy towards statistics, stop reading. If not, here goes…..

In Nepal the number of households facing acute food insecurity was reduced from 21 to 13 per cent and there was a significant increase ofbetween 40 and 70 per cent in productivity per hectare of major cereals and vegetable crops by targeted smallholder farmers. The programme supported 141 micro-irrigation systems which alone increased productivity by 50 per cent. Prior to the programme only 16 per cent of farmers used improved seeds, but, by 2011, 100 per cent were using them.

According to the evaluation: “Every household is now engaged in kitchen gardening and 72 per cent of families have been eating vegetables five days a week as against the previous time when only 27 per cent of families could eat vegetables for only two days a week”.

In Ethiopia, nearly 14,000 farmers were organized into 619 market-oriented co-operatives and producer groups and linked with service and input providers. Once they had the wherewithal, organization and confidence to grow more crops like malt barley and potatoes, they were linked to and able to negotiate with potential buyers in the nearest town. Twenty-nine Producer Organizations gained a contract to sell their barley to a brewery and malt factory at a rate 15 per cent higher than the local market price.  Seventy two per cent of beneficiaries reported that their income increased by over 30 per cent.

OK, that’s the statsfest over. Back to plain English – what worked well and why? The Ethiopia programme summed it up:

To succeed you need to:

• collaborate with a wide variety of stakeholders, including the private sector;
• design projects that are community-managed;
• organize farmers into groups; and
• provide a cash transfer so people can withstand shocks, keep assets and develop infrastructure.

In one way, it sounds simple – putting into practice the three-fold support identified by Wiggins and Leturque as essential to liberate the potential of small farmers:

• Stability in the face of price volatility and incentives to earn more money from farming;
• Investment in public goods;
• Strategies to overcome the problem of chronic failure in rural financial markets.

In other ways, though, a multi-faceted, multi-level intervention of this sort is fiendishly complicated and poses all sorts of managerial and logistical challenges.

Collaborating with many partners takes time, energy and resources and forging successful collaboration is time-consuming and laborious. Fortunately the Oxfam programmes concerned already had a strong presence in the areas going back many years and had strategic partnerships with local bodies. But the number and variety of meetings required almost make it sound like a parody of development work – familiarization workshops, consensus building training workshops, multi-stakeholder taskforces, thematic workshops, regional value chain development forums, taskforce forums, a national learning event etc etc. But this networking was key to the success of the programme.

So we would agree with WFP, FAO and IFAD: “As food prices are expected to remain high and volatile in the coming years, it is essential to maintain the momentum created by the EUFF in promoting agriculture as the most effective means of reducing global hunger and poverty…. ”.

It might also convince some people that EU aid isn’t all bad.

(The EU Final Evaluation concludes with the suggestion that the EUFF is made into a permanent “Stand-by” instrument. It makes sense to me, but I’ll let you know Oxfam’s official position once we’ve had a proper discussion here).

Robert Chambers – why don’t all development organizations do immersions?

Following on my review of Robert Chambers’ new(ish) book, ‘Provocations for Development’, I’m posting a couple of edited-down excerpts that caught my eye. Today, immersions –  written in 2007 and a nice illustration of how Robert combines both the politics and practicalities of aid work.

Immersions can take many forms, but an almost universal feature is staying in a poor community, as a person, living with a host family, helping with tasks and sharing in their life. The overnight stay is vital for relationships, experience, and relaxed conversations after dark and talking into the night. There may be activities like working with and helping the family, listening and dialogue, learning a life history, keeping a reflective diary or trying to explain your work and its relevance, but the essence is to be open much of the time to the unplanned and unexpected, to live and be and relate as a person. The unplanned incident is so often the most striking, moving and significant. Much is experienced and learnt, but what that will be is hard to predict.
 
Agreement seems universal that immersions give insights and experiences that are not otherwise accessible. Those who participate learn in a personal way about people’s lives, livelihoods and cultures and the conditions they experience. The world can be seen the other way round, from the perspective of people living in poverty.
 
Quite often there are stark and startling insights and impacts. Ravi Kanbur had an immersion with SEWA in India as part of the preparation for the World Development Report 2000/2001 for which he was Task Manager. He spent three days in a remote village, Mohadi. Parents were keen for their children to learn to read and write but the schoolmaster only came once a month. But he turned up on the second day when he had heard there were visitors. He launched into a litany of the difficulties of teaching the village children whom he described as ‘junglee’(from the jungle). This “Master of Mohadi” incident, Kanbur wrote, ‘encapsulated for me the gap between macro-level strategies and ground-level realities’.

All this is enough to justify immersions over and over again. If this were all, the case would already be overwhelming. But people repeatedly say they gained much more than just useful insights and knowledge. They stress, and often give more importance to, the experiential learning, the personal and emotional impact. Fred Nunes writes that [former World Bank President] Jim Wolfensohn “wanted managers who had heart as well as intellect”. The aim was to “rekindle the staff’s passion for poverty reduction”. For Taaka Awori:

“All of me was learning, not just my mind, as is usually the case. The immersion allowed me to stop analysing people living in poverty as objects of development, but rather just to be with them and allow the learning to emerge.”

Why did immersions not take off earlier?

If these experiences mean so much, and can make such a difference, why have they not spread more and been more widely adopted? They cost less than going to a workshop. They take little time – usually not more than a week. It is not as though most organisations lack money: training and capacity-building funds for professional development are frequently underspent.

Three clusters of forces stand out.

The first is personal. It is easy to make excuses, especially being too busy with important work. There is time for a workshop, within our comfort zones, but not for an immersion which is outside, unfamiliar, threatening. For myself, I am reluctant to give up what is known, cosy, and controllable for the unknown, perhaps uncomfortable and uncontrollable. I fear behaving badly and making a fool of myself. And here I and others must thank Ravi Kanbur for his “I don’t think I want to go to that temple any more”: he asked twice to visit an inviting-looking temple before realising that his host family were excluded from the temple because they were lower caste. This makes it easier for me to acknowledge my own shameful mistake, so hurtful to our host lady in Gujarat, of going to bed instead of meeting the people who had come across the desert to meet us. And then there are other arguments that can be mustered: ‘I know all about that. I grew up in a village (or slum). I don’t have anything to learn about that’.

The second cluster of forces is institutional. These are so many: values and incentives that reward writing good memoranda and reports and speaking well in meetings with important people; and the low value given to listening to the unimportant poor. There are senior staff who regard immersions as frivolous, useless or voyeurism, and/or feel personally threatened by them. There are normal pressures of work and other perceived priorities. Bureaucratic culture looks inwards and upwards, not downwards and outwards.

A third force is rhetoric about development relations. For staff of lender and donor agencies, there has been the convenient political correctness of government ownership. For international NGOs there has been increasing reliance on the insights of partners who are supposedly close to poverty. To seek direct personal experience through immersions could then be thought of as untrusting and interfering.

These personal, institutional and rhetorical forces combine. Any organisation or individuals who want excuses for not pressing for immersions have no difficulty finding them. It is not difficult, then, to understand why until recently effective demand for immersions has not been strong.

Why now?

The case is stronger now than ever for three reasons.

First, the conditions, awareness, priorities and aspirations of poor people are changing faster than ever before. There is a continuous and intensifying challenge to policy makers and practitioners to keep in touch and up to date.

More educational than a powerpoint?

Second, a new simplistic certainty has been infiltrating development thinking and practice. The downside of the Millennium Development Goals and of the inspiring movement to Make Poverty History, has been the belief that ‘we know what needs to be done’ (especially on the part of non-Africans about Africa) – and that the solution is more money. The issues are not so simple; nor in most cases are the solutions. Immersions provide one means of checking against the complex and diverse realities of poor people.

Third, the grip of the urban offices, capital traps and elite activities has tightened – for government, aid agency and NGO staff alike: more and more emails, meetings, negotiations, reports, often with fewer staff; participation in the pandemic of incestuous workshops, many of them about poverty; donors’ budget support, sector-wide programmes, and harmonisation on policy issues, all of this in what Koy Thomson calls our “self-referential universe.” Qazi Azmat Isa speaks for other agencies too when he notes that ‘increasingly World Bank staff are confined to government departments in capital and provincial cities, removed from the reality of poverty and from our ultimate clients – the poor of the country’.

Immersions are means to offset these biases and trends: to keep up to date; to be in touch; to escape the self-referential trap. It is fitting and fortunate that they are rising fast on the agenda. They are now better understood, more talked about and easier to arrange. More organisations – EDP, SEWA, ActionAid International, Praxis, Proshika – are providing them for others. More people and more organisations are setting them up for themselves. The increasing numbers of those who have experienced immersions and the conviction, commitment and authority with which they can speak, encourage others. We appear to be approaching a tipping point of a critical mass of stories, buzz, communications and enthusiasm.

What would those living in poverty want us to do? Would they, as Koy Thomson has asked ‘express their amazement that people who are experts in poverty don’t even bother to spend time with them’. As he observes ‘For a development organisation to see four days simply being with people living in poverty as a luxury is a sign of pathology’. The question is not whether the direct experiential learning of immersions and reality checks can be afforded. It is whether anyone in any organisation committed to the MDGs, social justice and reducing poverty, can justify not affording and making space for them.

Well that was written five years ago, and there doesn’t seem to have been an immersion tipping point since then. Any thoughts or personal/organizational experiences from readers?

Provocations for Development: Superb new collection of Robert Chambers’ Greatest Hits

This is not an impartial review – Robert Chambers is a hero of  mine, part development guru, part therapist to the aid community. His ideas and phrases litter the intellectual landscape. Or ought to: if you don’t recognize some of his major contributions to the development lexicon – ‘hand over the stick’, ‘uppers and lowers’, ‘whose reality counts?’, participatory research methods or seasonality, (there are dozens of others) you have seriously missed out, and Provocations for Development, a greatest hits collection of his speeches, writings, reflections and one pagers should definitely be on the top of your reading pile.

Chambers is also playful. ‘Fun is a human right’ he announces in the foreword, and the book duly starts with a beginner’s guide to bullshit bingo, that essential way to survive particularly mind-numbing meetings. He even provides handy photocopiable bingo tables for you.

His more serious intent in this first section is to highlight the power of words in development. Treacherous, slippery jargon that embodies and transmits certain views of the world, power relationships etc, often subliminally (think about the implied power relationships in the phrase ‘capacity building’). He sees words as being used to legitimize actions (‘partnership’), maintain dominance through obscure jargon (‘disintermediation’, ‘conditionalities’), camouflage realities (‘defence spending’; ‘donors’ rather than ‘lenders’) or sanitize, stereotype and stigmatize (‘freedom fighter’ v ‘terrorist’).

The book’s second section covers perhaps his most significant contribution to development thinking – participation. Robert’s work was central to developing methodologies such as Participatory Rural Appraisals (PRAs) and Poverty Assessments (PPAs) and highlights the uncomfortable reality of the disparities in power within the aid industry. ‘Whose reality counts?’ Aid workers are in a position of power, but they can do something about it, starting by ‘handing over the stick’ to poor people (to point out things on a blackboard, not to beat each other). I have consciously had to tell myself to hand over the stick on numerous occasions (and I’m still rubbish at it – handing over the powerpoint is even harder).

One of his major contributions was through his involvement in the ‘Voices of the Poor’ study, a watershed piece of World Bank research in the mid 90s, led by Deepa Narayan, which interviewed thousands of people in dozens of countries to try and grasp the complex multidimensional nature of poverty as experienced by poor people themselves (rather than defined by outside ‘experts’) (see diagram – compare that to the empty precision of $1.25 a day).

Chambers’ work has certain recurring themes, in addition to the power of language. Turning the tables (as in the subtitle of one of his most influential books, ‘Putting the Last First’); a complete absence of cynicism (even his bullshit bingo is somehow turned into a positive learning experience); an unquenchable curiosity about the lives of poor people; the use of visuals, diagrams, do it yourself methods with stones and sticks to reflect those lives; an honest appraisal of the lives, work and career paths of development professionals – he’s one of the few to address how people actually feel when they are ‘doing development’. And he is relentlessly quirky – the one part of his work that I really struggle with is his fondness for dashing off some pretty dire rhymes about the aid business.

Chambers’ abiding interest in excrement, for example (graphic accounts of his first disastrous encounter with a high tech Japanese toilet – he pressed all the buttons) has found its outlet (sorry) in his most recent enthusiasm, the Community-Led Total Sanitation movement. It’s a brilliant participatory, human, low tech response to the all the high tech magic bullets that hog the headlines.

Given all this, Chambers could be forgiven for being a bit pompous, but he isn’t. Not even slightly. He’s great company, a mischievous zen master to the aid community. I could go on. And I will – I’ll post on a couple of particularly resonant chapters in the next few days.

What have the MDGs achieved? We don’t really know… Heretical thoughts from Matthew Lockwood

A second instalment in Matthew Lockwood’s series of valedictory boat-rocking blogs (his first was on fossil fuel subsidies) as he leaves the IDS Climate Change team for a new role in the UK energy sector. This time, he asks why the results agenda often stops short of being applied to the big picture stuff like the MDGs.

One of the interesting things about having come back to the international development field after some years away is the greatly increased emphasis on results, across all areas of activity, including not only projects and programmes, but also policy making, research, and advocacy.

Many people and organisations are interested in the results agenda, including the big foundations such as Gates, influential bloggers like Owen Barder, my boss Lawrence Haddad, and DFID’s Secretary of State, Andrew Mitchell. In his first big speech in office, in Washington in June 2010, Mitchell said “we’re also fundamentally redesigning our aid programmes so that they build in rigorous evaluation processes from day one.”

Like many others, I think aspects of the results agenda are important, reasonable and politically wise, although there are also some interesting critiques of the approach. But I also think that, if you really take it seriously, it throws up some challenges and dilemmas.

For me, this is clearest in the case of development’s big frameworks and policy directions. One prime example is the UN Millennium Development Goals (MDGs) and their proposed replacement with more development goals after 2015. As most readers will know, the MDGs are a set of human development goals, with subsidiary targets and indicators, formally adopted by the UN in 2000.

There is pretty broad agreement that progress towards meeting the MDGs is partial and uneven – some of the goals have been met or look very likely to be met, in some countries, while other goals (such as the target reduction in maternal mortality) may not. Asia, especially East Asia, has done better than Sub-Saharan Africa.

However, applying the results agenda to the MDGs is not simply a matter of asking whether the goals will be met. Rather, it is about asking whether the goals have been met as the result of the MDGs having been adopted. The purpose of having high level goals, including any that come after the current MDGs, is to create political will, the mobilisation of resources, policy change and delivery, all of which should bring about a positive change relative to what would have happened in their absence.

Many in the aid world would say that, of course, the MDGs have had a major impact, and that it is absurd to even raise the question. However, a rigorous assessment of the evidence suggests that it is actually quite hard to make a strong case.

First, the evidence that the MDGs may have made a difference is, at best, mixed. The most comprehensive and rigorous independent assessment is by Andy Sumner and Charles Kenny for the Center for Global Development. They look for significant differences in outcomes and impacts before and after 2000, when the MDGs were adopted.

The clearest effects were on aid levels (which are not an ultimate impact but an intermediate outcome). Compared with the previous decade, official aid increased in the post-2000 period, but not as a proportion of rich country GDP. More aid went to the poorest countries, including to Africa. There was a small shift in the share of aid going to the social sectors, on which the MDGs tend to focus, and this happened soon after 2000.

There is plenty of evidence of the influence of the MDGs on policy discourse, if this is measured by mention of the goals or their presence in donor policy documents, PRSPs and developing country government goals. However, the effects on actual policy change are less clear. Sumner and Kenny find it “hard to detect a trend” in low income country government spending on health and education. They also find no trend in the quality of developing country policy making, as measured by the World Bank’s Country Policy and Institutional Assessment ratings.

On the actual impact indicators themselves – such as income poverty, malnutrition and mortality rates, educational enrolment etc – Sumner and Kenny’s most relevant assessment is whether progress was faster pre- or post-MDGs, and whether progress post-MDGs has been faster than what would have been expected based on past trends. Again, results are inconclusive. The data “suggest that in no case is there an obvious sign of a significant break towards faster progress since 2000. Nonetheless there has been somewhat faster global progress on income, primary completion rates, child and maternal mortality over the post-Declaration period”. A study by Fukuda-Parr and Greenstein of country level data gives a similarly mixed picture. The comparison with predicted rates of progress based on historical analysis implies slightly better than expected outcomes post-MDGs on primary education and gender equality in education, but worse on maternal mortality.

Second, there is the problem of attribution. As Sumner and Kenny put it, “even ignoring the very limited evidence of faster progress since 2000 in the average (unweighted) developing country, it is a considerable step from ‘more rapid progress’ to ‘the MDGs caused more rapid progress’”. In other words, bilateral aid may have increased somewhat, some indicators have improved, but how do we know that these changes are due to the MDGs, and not to some other factor?

It is not possible to know what would have happened in their absence. This is not a case of running randomised controlled trials across a number of interventions. And as Richard Manning points out, it is hard to separate out the potential effects of the MDGs from the environment that produced them.

In some areas, such as vaccination or primary education enrolment in sub-Saharan Africa, the links between the MDGs, the mobilisation

and focusing of additional aid, and subsequent impacts seem convincingly close. But in others, the links seem less plausible, especially where there are also good alternative candidates that may explain changes in indicators better than the effect of MDGs. Poverty reduction in Asia, for example, is more likely to have been driven by the extraordinary period of sustained economic growth in China, than by a set of UN targets. It is also plausible that China’s growth will have pulled along a number of countries in its wake, including commodity exporters in Africa. The rapid reduction of poverty in Brazil is due in part to the development of social safety nets such as the Bolsa Familia. When I recently asked Romulo Paes de Sousa, Brazil’s former Deputy Minister for Social Development, and closely involved in the design of the Bolsa, whether it was the result of the MDGs, he dismissed this immediately, saying it was the outcome of a domestic debate that emerged from the minimum wage.

Yet despite the lack of clear, strong evidence of the impact of the Goals, and the difficulties of attribution, the MDGs are routinely hailed as a success. Most importantly, this success is asserted in the context of discussion about a new set of post-2015 development goals. When it was announced that David Cameron would be co-chair of the UN High Level Panel on post-2015 goals, Andrew Mitchell hailed the “huge progress that has been made through the Millennium Development Goals” and “the successes of the current goals”.

When challenged with the point that attribution is often difficult in cases such as these, and that you can’t compare counterfactuals, many proponents of the results agenda recognise the problem. However, their argument is that, in such circumstances, it is the duty of those proposing any particular approach to be explicit about their “theory of change” – that is, be explicit about the full chain of causal linkages you think is going to run from your intervention (here adopting international goals) and the impacts you hope for. Identify your assumptions. Assess the evidence for and against those assumptions, and weigh up the risks.

If done properly, this wouldn’t be just about ticking a box. The point of such an analysis should be to help understand how to make such goals more effective. It should look at why some goals were easier to meet than others (gender equity on education as opposed to access to clean water or reductions in maternal mortality) and in some countries than in others. It should look in a systematic and rigorous way in how the goals were used (or not used) and where there is evidence that they failed to lead to a result, explore alternative, potentially more effective “pathways to impact”.

The point here is not that the MDGs are somehow a bad thing, or that there should not be a new set of goals. In any case, it is not seriously in question that there will be further goals post-2015, of some form. Too much political capital has been invested in them for this to be the case, regardless of the ambiguity of the evidence base. The results revolution will not change the reality that some policies and initiatives are often inevitably driven by more than evidence, and that politics plays a major role.

Nor am I advocating a view that we should not try to measure impact or wrestle with the problem of attribution. What I am saying is that I think the example shows that really, really applying the agenda of results and evidence-based policy consistently and rigorously can be more difficult than the current discourse acknowledges.

Matthew Lockwood is a Research Fellow at the Institute of Development Studies at the University of Sussex. From October 2012 he starts work on a four year project on innovation and governance in the UK energy sector. 

Three x 4 minute videos for World Humanitarian Day (that’s today)

Three thought-provoking short pieces from the slightly Orwellian-sounding Security Management Initiative in support of today’s UN World Humanitarian Day

Access and Acceptance

Risk

Principles and Pragmatism

Horizon 2025: the future of aid (and a potentially epic nerdwar on poverty numbers)

I’ve been starting to feel like an unpaid publicist for the Overseas Development Institute recently. It’s not my fault – ODI keeps publishing really interesting stuff (and anyway, I’m not always nice about it). You’re likely to hear a lot about their latest paper, Horizon 2025: Creative Destruction in the Aid Industry, by Homi Kharas (Brookings Institution) and Andrew Rogerson (ODI), so I urge you to read it, rather than just winging it based on this blog (not that you would dream of it, of course).

The paper first takes the ‘MICs v LICs’ discussion on where poor people actually live, which Andy Sumner has been leading for the last couple of years, and projects it forward to 2025. Futures work usually offers several scenarios because the future is so difficult to predict, but Horizon 2025 opts for a single ‘base level 2025 scenario’, extrapolating recent trends as follows:

• High per capita income growth and falling population growth in large, dynamic, middle income countries (MICs) shrink the global poverty pool drastically.
• Income stagnation and high fertility rates in selected low-income and fragile countries re-establish them as the main locations of global poverty.
• Growth in emerging economies dominates global growth and they account for most new trade and foreign capital flows to poor countries, along with sizable increases in aid-like flows, competing for influence with traditional aid donors.
• Availability of public and private resources for development, coupled with the fall in global poverty, imply that dramatically more funding is potentially available for each poor person.

This is all good stuff but is arguably skating on fairly thin ice as the ‘baseline’ is at the optimistic end of possible scenarios. It uses IMF growth projections and average expenditure per person from GDP data (known as the national accounts), rather than the now burgeoning amount of household poverty survey data, and assumes that inequality remains constant as countries grow (when it often rises). That means that the 2025 baseline effectively maximises per capita income and minimises poverty, compared to other estimates such as that by Karver, Kenny and Sumner for CGD (here) and Andy for IDS (here).

Andy’s now ubiquitous killer fact is that three quarters of the world’s poor – well, 79% or 2 billion poor people (by $2 poverty) – now live in MICs. You hear this quoted everywhere, with the implication that internal politics and distributive struggles (rather than aid) hold the key to future development. Kharas and Rogerson argue that the MIC thing is actually a blip (OK, a twenty year blip, but still). By 2025, growth in most MICs will have lifted almost all their poor citizens above the $2 a day poverty line (I said they were optimists). Instead, the vast majority of the world’s remaining poor will live in FRACAS – fragile and conflict affected states (see chart 1): the long-term dividing line will between between fragile v stable states, rather than LICs v MICs, which chimes nicely with my own work on effective states. They will also largely live in Africa, which should please Paul Collier, as it endorses his Africanist ‘Bottom Billion’ thesis.

Using a different source of data – household poverty surveys and multiple scenarios – Andy Sumner finds a quite different pattern. Across a range of scenarios, there is a 50/50 LIC/MIC split in poverty in 2020 and 2030. Given that some of today’s LICs will be MICs by then, it’s possible that just one-third of the world’s poor will be in the remaining LICs. Meanwhile, estimates for total global $2-a-day poverty by Andy’s method in his paper with Charles Kenny and Jonathan Karver at CGD (see here) run from as low as 600m to as high as 1.6bn (there’s also multiple scenario data for other poverty measures such as malnutrition there too).

All in all, I’d say that Horizon 2025 is not the last word: there’s plenty of room for doubt and arcane methodological debate over the numbers, with significant consequences for the way we think about the future of development. I see a nerd war approaching – wonderful!
But whatever the final outcome of that war, it seems to me that Horizon 2025 is really onto something. In terms of reducing poverty, the focus in most MICs will be domestic policy, not aid. Donors and INGOs may play a minor supporting role in those domestic struggles, but the core aid business is likely to retreat to fragile states, (where, incidentally, we will witness a titanic clash between the chaotic, unpredictable nature of fragile states and the increasing demands for measurable, attributable impact).

Where Andy and Kharas and Rogerson do agree is on the affordability of ending poverty. Andy provides a set of estimates that suggests the costs of ending poverty might well be affordable (as a percentage of GDP) for most MICs by 2020 and certainly by 2030, leaving just 16-28 LICs in 2030 needing foreign aid to end poverty.

Kharas and Rogerson calculate the costs of eradicating poverty in a similar rump of fragile states, assuming a certain amount comes from domestic sources and the rest from aid. Their argument is that by 2025, mobile phone-based banking will be ubiquitous so it really will be possible to deliver cash direct to the poor, however dysfunctional the state. As poverty falls and GDP grows, a tech-based $2-a-day safety net becomes increasingly affordable, accounting for just a fraction of 1% of global GDP (see chart 2).

Next, the authors think through a further set of issues on the future role of aid. They look at three intriguing and believable ‘disruptors’ to the traditional aid system:

1. New channels for aid via philanthropy or ‘disintermediated’ direct giving
2. The rise in South-South cooperation, and with it, the blurring of distinctions between aid and cooperation based on mutual interest
3. Climate change and the rise of climate finance (both for mitigation and adaptation) either in parallel to, or as a competitor to, poverty-focused aid

Going into any detail would be too much for an already overl-long blogpost (you really should read the paper), but they run a stress test against existing aid agencies to see who is best/worst prepared for this 2025 world. On the resulting ‘traffic light’ bar chart (below), red is the worst prepared. Bad news for Spain, but Canada (for once) is deemed to have got it right, and they’ll be dancing in the corridors of GAVI and the Global Fund.

I suspect this may not be the last post on this.