How to end foreign aid and avoid a punch-up

An edited version of this piece appeared on the Guardian’s Comment is Free site on Saturday

The spat between South Africa and Britain over ending its (very small) aid programme has sparked another round of debate about whether British aidshould be going to middle income countries (the last round was over aid to India, which seems to particularly rile the Daily Mail).

But whatever the rights and wrongs of ending aid to South Africa (whose economy is growing slowly, but with sky high levels of inequality and 10 per cent of all the world’s people living with HIV and Aids), aid agencies are inevitably going to have to shift money around as the world changes. Countries rise and fall, aid priorities change and new opportunities (like the opening up of Myanmar) will arise, to which aid should of course respond.

That churning process is accelerating as more and more countries reduce their dependence on aid thanks to economic growth, rising revenues from oil and gas and surging remittances from their migrant workers overseas. Those upward curves contrast with falling aid volumes.  Total global aid flows have been falling for the last two years as, with the sole glorious exception of the UK, the ‘Austerians’ in what are sometimes known as ‘the formerly rich countries’ take the axe to aid spending.

What’s more, many developing country governments can’t wait to end aid – it affronts the dignity of the Big Men (as they usually are) in charge to be seen as asking for handouts. As two Ugandan ministers once proudly told me ‘when 60% of our revenues came from aid, we had to go to the World Bank on both knees. Now we’ve got it down to 30%.’ (Unfortunately, I blew it by joking ‘ah, so do you just go on one knee now, then?’ They were not amused.)

So the issue is not whether aid partnerships sometimes have to end, but how. Oxfam has some ideas on that – several pages of guidelines on ‘exit planning’ in fact. They are about ending funding to particular grassroots organisations, but they offer useful lessons for DFID and others contemplating exits from entire countries. Talk to the partner from the outset, help them fill any organisational gaps and weaknesses before you go; agree (don’t impose) a clear timetable for phasing out funding and crucially, agree what kind of relationship you want once the cash tap has been turned off. Because as one Oxfam partner from Pakistan put it, ‘We want to be treated as a “relationship” and not just as a “partner”’. Partnerships are for projects, but relationships go beyond that. We should have a relationship in the future, even if there is no project money.’‘ Sadly, the unpleasantness over DFID’s exit from South Africa means that remaining relationship has got off to a rocky start.

I imagine DFID has similar guidelines, but those can’t legislate for ministers shooting from the hip (or if conspiracy theorists are to be believed, throwing a bone to the Tory right ahead of local elections last week). Still, if civil servants in the land of ‘Yes Minister’ can’t manage their political masters, what are we coming to?

Joking aside, this matters. Aid agencies need a clear understanding of what constitutes a responsible exit. After all, aid should bequeath a legacy of trust and friendship between the UK and the rising powers of Africa and Asia, (who incidentally, we are going to need in future), but in this case that legacy has been (temporarily, I hope) squandered. Ending the aid relationship should be a moment of mutual celebration, not public mud-slinging.

Update: and here’s a new report on the subject of responsible exit

What can DFID learn from Chinese and Brazilian aid programmes?

IDS researcher Henry Tugendhat (right) wonders whether UK aid is following in the path of China and Brazil

Two weeks ago at the London Stock Exchange, Justine Greening announced her new policy of supporting UK businesses to invest in developing economies for the mutual benefit of both sides. According to the UK’s Secretary of State for International Development: “This is good for investors, who earn a financial return… [and] good for the poorest, who receive jobs and support”.

New as it may sound in UK development circles, this strategy sounds an awful lot like the “win-win” sound-bites we’re increasingly used to hearing from China among the other BRICS countries, who are meeting in Durban, South Africa this week. China and Brazil have both made efforts to leverage private sector investments as part of their aid/South-South Cooperation agendas, but problems have already arisen which the UK could usefully learn from.

Alignment of aid and investment efforts is a particular challenge. Business seeks profit, while aid is traditionally geared to humanitarian objectives, and they don’t always match up.

One example is the disconnect between the promotion of food security in Africa, and the agricultural interests of investors. The Chinese state has pushed for a serious engagement with food security issues affecting African countries. This is seen in everything from meetings in the Forum on China-Africa Cooperation, to Agricultural Technology Demonstration Centres. Equally Brazil has committed to supporting agricultural mechanisation and technical support in a number of countries.

All such efforts are envisaged to link to private sector activity: to provide tractors, take over farms, or offer technical support. However, Chinese and Brazilian investors in Africa have predominantly invested in cash crops and large-scale farming, and even when smallholders are targeted, in many cases they have been those that are better off and already more commercially oriented.

New evidence from the Future Agricultures Consortium generated by research in Ghana, Ethiopia, Mozambique and Zimbabwe, shows that when Chinese companies do invest in local agriculture, it is predominantly in crops such as cotton and tobacco. Otherwise, additional investment has focused on surrounding infrastructure such as irrigation schemes and roads. Brazilian investments have predominantly focused on sugar-cane and ethanol production. However, when private sector investment has targeted food staples necessary for food security, there has been either little success, as in the Xai-Xai project in Mozambique; or only small-scale investments, as in the small farms in Ethiopia that supply local Chinese restaurants.

Adding to this disconnect between Chinese and Brazilian government statements and the behaviour of their companies, even Chinese state-owned enterprises at times flex their independence from state interests, as Lucy Corkin describes for Angola. But of course, Chinese companies in particular may be subject to more pressure than UK companies to invest in areas aligned with state interests abroad, as they are often supported by state policy and finance. Which begs the question: when financial returns are less obvious, what makes Justine Greening more able to leverage investment for development interests, where China and Brazil have failed?

The second issue is that many investments in Africa have led to harmful practices over the years, whatever the nationality of the investor. Even if investors are introduced based on the principle of “development assistance”, profits are still paramount. This can lead to unintended consequences. For example, Chinese and Brazilian companies have got mired in controversy in recent years, including accusations of ‘land grabbing’, even when such claims have been refuted.

One of the currently most contentious investments is the vast ProSavana project spearheaded by Brazil in a triangular agreement with Japan and

Mozambique. The aim is to bring about agricultural development in Mozambique’s Nacala corridor, transforming it into a bread-basket on the scale of Brazil’s Cerrado savannah region, the site of a “green revolution” transformation which began in the 1970s. Brazilian private investors are already, as well as Japan, and a ‘Nacala Fund’ expected to mobilise US$2 billion has been set up. Already described as an example of ‘Brazil’s neo-colonialism in Africa’ in the Mozambican press, the project has come under heavy fire from local civil society groups that criticise the secrecy surrounding the project and the risk of evictions of local smallholder farmers.

So will ‘land grabbing’ become an issue that UK businesses face if they follow Justine Greening’s recommendations? In her speech, she recognised this challenge, arguing that there is a need for transparency on investments, so as to expose “those who acquire land unfairly”.

As a final thought, we should also question whether this new policy could be a subtle re-introduction of tied aid practices, despite Justine Greening’s

strong assertions to the contrary. The political reality is that this would no doubt be an appealing means of defending the aid budget against current cuts, so it must be properly scrutinised to avoid to a return to inefficient aid practices.

Ultimately, overseas investment by private sector actors is open to many challenges and problems. As the UK adds this new facet to its aid programme,
lessons from Brazilian and Chinese experiences in Africa should definitely be heeded.

The evidence debate continues: Chris Whitty and Stefan Dercon respond from DFID

Yesterday Chris Roche and Rosalind Eyben set out their concerns over the results agenda. Today Chris Whitty (left), DFID’s Director of Research and Evidence and Chief Scientific Adviser and Stefan Dercon (right), its Chief Economist, respond.

It is common ground that “No-one really believes that it is feasible for external development assistance to consist purely of ‘technical’ interventions.” Neither would anyone argue that power, politics and ideology are not central to policy and indeed day-to-day decisions. Much of the rest of yesterday’s passionate blog by Rosalind Eyben and Chris Roche sets up a series of straw men, presenting a supposed case for evidence-based approaches that is far removed from reality and in places borders on the sinister, with its implication that this is some coming together of scientists in laboratories experimenting on Africans, 1930s colonialism, and money-pinching government truth-junkies. Whilst this may work as polemic, the logical and factual base of the blog is less strong.

Rosalind and Chris start with evidence-based medicine, so let’s start in the same place. One of us (CW) started training as the last senior doctors to oppose evidence-based medicine were nearing retirement. ‘My boy’ they would say, generally with a slightly patronising pat on the arm, ‘this evidence-based medicine fad won’t last. Every patient is different, every family situation is unique; how can you generalise from a mass of data to the complexity of the human situation.” Fortunately they lost that argument. As evidence-informed approaches supplanted expert opinion the likelihood of dying from a heart attack dropped by 40% over 10 years, and the research tools which achieved this (of which randomised trials are only one) are now being used to address the problems of health and poverty in Africa and Asia.

The consequences of moving from expert (ie opinion-based, seniority-based and anecdote-based) to evidence-based healthcare policy, far from being some sinister neocolonial experiment, have been spectacular. To quote a recent Economist headline, ‘Africa is currently experiencing some of the fastest falls in childhood mortality ever seen, anywhere’. It is a great example of the positive side to modern Africa the current excellent Oxfam publicity campaign (right) is all about. This success is based on many small bits of evidence, from many disciplines, leading to multiple incrementally better interventions. Critically, it also involves stopping doing things which the expert consensus agreed should work, but which when tested do not. It is no accident that one of the most evidence-based parts of development is also one where development efforts have had some of their greatest successes.

Proper evidence empowers the decision-maker to be able to make better choices. This is a good thing. In every discipline, in every country, where rigorous testing of the solutions of experts has started, many ways of doing things promoted by serious and intelligent people with years of experience have been shown not to work. International development is no different, except that the communities we seek to assist are more vulnerable, including to our bad choices.

Much of what we all do in international development has very limited evidence that it does any good  (in this it is no different from many other policy areas) – which is not the same as saying it is pointless. Rather we don’t know what is pointless. Some of our actions will work better than we think, much of it will work much less well than we hope, and some of it will be damaging the poorest without us realising it. In the evidence-light areas we just don’t know which are which.

We must have the humility to accept that we are all often wrong, however reflexive the practitioner, however deep their reading and experience and passion to do good. Evidence-based approaches are not about imposing a particular theory or view of the world. It is simply about taking any opportunity to test our own solutions in the best way available, using evidence honestly when it is available to inform (note the word) decisions, and when the facts change, changing our minds.

This honesty includes saying to decision-makers when evidence is methodologically weak, mixed or missing so they know they are on their own, unable to rely on (or make a claim on) the evidence. The worst possible solution, which we know Chris and Ros would also deplore, is using the social power of the ‘expert’ to imply we know the answer when we actually have no solid evidential basis for our opinion or prejudice.

A few false assumptions about evidence-based decision making

Some of those who express unease about evidence-based policy and practice seem to assume that it is always based on randomised trials and quantitative methodologies: not so. Methods from all disciplines, qualitative and quantitative, are needed, with the mix depending on the context. Randomised trials are one tool amongst very many, although a good one in the right setting. The argument that evidence-based approaches can “only apply in cases of individual treatment and not the wider community level” ignores over 30 years of methodology which has done exactly that, with very convincing results.

A sterile argument  between people who are on the one side believe that a  randomised trial can answer any question (they can’t), and people who do not appear to be aware of any  methodological advances since the 1970s except in their own narrow field is a depressingly familiar experience. We know this does not apply to Rosalind and Chris, but listening to people passionately critiquing methodologies they have not taken the trouble to understand does no good to anyone. This applies both to a randomista who seems to believe that all there is to social research is a few focus groups and in-depth interviews, and to people from a more qualitative social science background who would have trouble explaining the difference between cluster randomised and step-wedge design but assume both are irrelevant to social research anyway (both can be used to measure societal rather than individual effects).

It is tempting to take every point the authors make where we have concerns about their factual basis and logical framework but we will take just three.

“Evidence-based approaches are pre-occupied with avoiding bias and increasing the precision of estimates of effect”. On less bias – generally true. Please complete the sentence ‘More biased research is better because…’. On precision – no, incorrect, the range of situations where a more precise answer is a better answer is small.

One statement we would like to address head-on starts “Evidence-based approaches became linked to value for money concerns to deliver ‘results’…”. We agree- and this is a good thing. Doing a pointless thing, professionally delivered and passionately believed in, is always going to be poor value for money. Testing what works and what does not therefore is essential to value for money. More importantly, doing pointless things diverts very limited human and financial resources, in an ocean of need, away from those who could best use them- not what any of us are in international development to do.

Is it “technical approaches” on the one hand, and “power, political economy” analysis on the other?

Rosalind and Chris’ key criticism is that evidence-based approaches “deflect attention from the centrality of power [and] politics […] in shaping society”, and they offer “power analyses” as an apparent alternative to assessing rigorously what works. This creates a false dichotomy, as if a choice has to be made between a “technical, rational and scientific approach to development” and an approach that recognises politics and the role of power. It is easy rhetoric, but troubling and, if taken much further, even dangerous. Understanding power and politics and how to assist in social change also require careful and rigorous evidence, and again, results are not simply what experts would have expected a priori. Recent studies on the positive impacts of female leadership quotas in rural India are for many of us rather surprisingly good news, even if one can fairly worry about its applicability in other settings, while the struggle to find systematically a positive impact of decentralisation and community-driven development programmes is important to internalise in our actions for change, and highlights the importance of understanding contexts and politics. In these cases, it is not a matter of just RCTs, but of rigour, and of combining appropriate methods, including more qualitative and political economy analysis.

Strong analysis of politics and power without offering much in terms of what can be acted upon is similarly unhelpful. They criticise an evidence-focused agenda by stating that “to act ‘technically’ in a politically complex context can make external actors pawns of more powerful vested interests and therefore by default makes them, albeit unintentionally, political actors.” But all actions by external actors will interact with political forces and vested interests. In many of the settings where development actors want to make a difference, power and political institutions are biased against the poor. Being able to act on strong evidence of what works in constrained political settings is crucial.

A reductionist and misinformed view of evidence as purely ‘technical’ or as being only about “what works” is unhelpful – it is also about generating evidence and understanding (and learning) on why interventions and approaches may work, including understanding the social, political, and economic factors that may enable or constrain success of different approaches. Far from the search for evidence pushing us in a ‘technical’, apolitical direction it has reinforced the importance of understanding and trying to tackle the underlying causes of poverty and conflict. There is agreement on the importance of politics and institutions in shaping growth, security and human development. However, the ability of external actors to influence institutions is much less clear and this is where DFID research is now focussed. Ros and Chris have misread the context – the commitment to evidence has opened up the space fundamentally to challenge conventional, technical approaches to aid.

Why it matters for international development

There are large areas of international development where decision-makers are largely flying blind – forced to make decisions purely on gut feeling and ideology not because they wish to because they have no option. Try making difficult decisions in education policy compared to health policy and the difference in usable evidence is dramatic – yet both are complex, social and context-dependent parts of human life. It is always puzzling when people say airily ‘health is easy’- it is not, and is an intensely political and social subject requiring interventions at societal level.

Today we can eradicate rinderpest in cattle and build bridges over the Zambezi based on rock-solid evidence from many disciplines, but do not have anywhere near as clear an idea how to reduce violence against women or tackle police corruption. All are great challenges with social dimensions but in two of them people have set about finding and testing solutions in a systematic way over many decades.

Having robustly tested evidence-based solutions certainly does not eliminate politics: the decision whether to build a bridge, what sort and where, is an intensely political choice – but at least those making the choice now have a fair assumption it will stand up- based on hundreds of years of incremental evidence. The evidence-barren areas in development are a collective, and in our view shameful, failure by us all in the academic and practitioner community. We should never excuse them with the feeble assertion that it is too difficult or complicated. Development is difficult and complicated – but the bases for making decisions will gradually improve if we are serious about improving it.

In conclusion, we collectively have the capacity to be able to give to our successors in every continent a far better basis on which to make their decisions for their lives than our generation have. To imply it is not worth trying to provide the best and most rigorous evidence to those who need to make difficult decisions because they will have other influences as well is like saying to someone going for a walk in dangerous mountains that they do not need a map because there will be many other factors that will determine where they go. That is true – but they are still less likely to fall off the cliff if they have one.

Where evidence is clear-cut we should be making that plain to decision makers – and where it is not we should say that as well, be honest about what is there and try to get better evidence for the future. That, in essence, is what evidence-based decision making is about – and all it is about. If the academic community is serious about trying to assist those working in the field (including in Oxfam), and above all empowering the most vulnerable communities to make the most informed possible decisions available for their own development, we should be putting our greatest efforts into supporting decision-makers to use the best evidence, and finding better methodologies in areas where we currently have very weak evidence. There are many, and this should be tackled as a matter of urgency.

Tomorrow, Chris Roche and Rosalind Eyben respond

What do DFID wonks think of Oxfam’s attempt to measure its effectiveness?

More DFIDistas on the blog: this time Nick York, DFID’s top evaluator and Caroline Hoy, who covers NGO evaluation, comment on Oxfam’s publication of a set of 26 warts-and-all programme effectiveness reviews.

Having seen Karl Hughes’s 3ie working paper on process tracing and talked to the team in Oxfam about evaluation approaches, Caroline Hoy (our lead on evaluation for NGOs) and I have been reading with considerable interest the set of papers that Jennie Richmond has shared with us on ‘Tackling the evaluation challenge – how do we know we are effective?’.

From DFID’s perspective, and now 2 years into the challenges of ‘embedding evaluation’ in a serious way into our own work, we know how difficult it often is to find reliable methods to identify what works and measure impact for complex development interventions.  Although it is relatively well understood how to apply standard techniques in some areas – such as health, social protection, water and sanitation and microfinance – there are whole swathes of development where we need to be quite innovative and creative in finding approaches to evaluation that can deal with the complexity of the issues and the nature of the programmes.  Many of these areas are where NGOs such as Oxfam do their best work.

So we would really like to welcome and applaud Oxfam’s new Effectiveness Reviews, which adopt a clear and practical framework for assessing what difference it is making, through its partners, in the development process. It is a big step forward for them – and it would be great if it also inspires other organisations to develop new and interesting approaches to measuring results and undertake rigorous analysis of what works.  Clearly this needs to be done in a way which each organisation can afford and resource – things need to be done in a proportionate way – but the Oxfam initiative shows some of what is possible.

They have chosen quite a practical strategy – picking out a random sample and then probing more deeply and using different techniques to measure impact or use well-tried monitoring of performance indicators.

Of course there is one potential drawback – random sampling may mean there are gaps in what you can say, if key areas don’t happen to have been sampled this time.  Oxfam also notes that the reviews do not necessarily enable full understanding of why a programme is successful (e.g. in Pakistan) and that they now need to go back and undertake some more work.   One way round this is more purposive sampling – we don’t know if this was considered –  or identifying priority themes up front based on what the organisational objectives are, and focusing on them in some depth.  The key challenge is finding a strategy for using the limited resources for evaluation and data collection in a targeted way that gives a nice balance between extensive coverage and intensive analysis.

Another challenge is maintaining the independence and integrity of those carrying out the evaluations.   Finding impartial observers – given that many people and experts have worked for years in these areas and know each other well – can be difficult.

The very interesting study of policy influencing by Oxfam’s partner in Bolivia, Fundacion Jubileo, is worth looking at in some detail.   It made a good case that the grantee was really having an impact on some key aspects of social change in Bolivia The evaluators clearly applied the process tracing technique skilfully and identified the most significant changes – but it must have been difficult to stay objective when doing the interviews, working with the grantee and identifying who was really influencing whom.  Howard White and Daniel Phillips’s paper on ‘small n’ techniques talks a lot about the biases that one needs to avoid in using these techniques.  The appendix to the study provides an excellent and useful set of reflections on the use of the process tracing methodology and what the evaluators learned.

One key assumption is that by doing more work and collecting more data (e.g. from comparison sites in Zambia and the Philippines), they will be able to understand and demonstrate impact.   Actually, based on discussions we have had with Michael Woolcock recently in DFID, we have started to ask a different sort of question.    In some types of programmes, more data and more work may not be the solution - more innovative methods and approaches to understanding impact can be required and if the programme itself develops as you implement it then the goal posts are continually shifting too.

Looking ahead, and thinking about the next stages of this agenda….first, we would encourage others to share their approaches and experiences in the way that Oxfam has done.   Second, it would be great to see Oxfam and other NGOs sharing resources to develop better methods across the sector, given their common challenge of demonstrating results and the limited resources.   The results agenda is particularly challenging for smaller organisations, whose inputs are increasingly recognised – so can we ask if Oxfam sees itself as in a position to demonstrate leadership in linking with such organisation to jointly share and explore results?

Nick York is DFID’s Chief Professional Officer – Evaluation and Caroline Hoy, its Results and Evaluation Specialist, Civil Society Department.

Can theories of change help researchers (or their funders) have more impact?

Got dragged into DFID this week for yet another session on theories of change. This one was organized by the DFID-funded Research for Development (R4D) project (sorry, ‘portal’). A lot of my previous comments on such sessions apply – in DFID the theories of change agenda seems rather dominated by evaluation and planning (‘logframes on steroids’), whereas in Oxfam, it is mainly used to sharpen our work in programmes and campaigns. But the conversation that jumped out at me was around ‘how do we influence the researchers that we fund to use theories of change (ToCs) to improve the impact of their research?’

It’s risky to generalize about ‘academics’, but I’m going to do it anyway. Let’s apply some ToCs thinking to academia as a target. Applying ToCs to try and understand why academics don’t use ToCs may feel a bit weird (like the bit in Being John Malkovich where Malkovich enters his own brain), but bear with me.

Let’s start with the 3i model – processes and decisions are influenced by institutions, interests and ideas. Because academia is largely non-profit making, institutions and interests are pretty much the same thing, and come down to incentive and career structures. Here I think DFID has a problem in getting researchers to be more concerned with impact - whatever favourable ideas are around in terms of academics wanting to change the world are likely to be neutralised by the institutional culture:

Career progression takes place largely through peer approval rather than through any ability to influence the world outside (in fact, being dubbed a ‘media don’ can damage your promotion prospects).

One of the big risks for an academic is being rubbished in public for being wrong, naive or insufficiently nuanced – academics love snark and gossip (not like NGOs then…) and that kind of kicking can damage your reputation for years. So there are strong disincentives to set out clearly your assumptions about how change happens (especially if they’re really naff, like ‘all you need is robust research to convince grateful-but-dim policy makers to change their misguided ways’, which I suspect is actually the theory of change behind a lot of research).

That fear of clarity may explain why when I worked as a publisher, I watched how perfectly good, clear writers started a PhD and were lost to me, entering into several decades of inaccessible post-modern gibberish before emerging blinking into the light as self confident, respected professors once again able to communicate in normal English (e.g. talking to a potential young author on Mexico. Me: ‘so who has the guns then?’ Author – light dawns after baffled look – ‘Oh, you mean the repressive apparatus of the state!’)

What other ideas might ToCs suggest? That you need to reward and build alliances among the drivers of change (eg encouraging young Blattmanesque bloggers who ‘get’ communications and influencing, while doing your best to neutralise ‘blockers’ – custodians of the peer-reviewed flame, perhaps?).

Or that you need to spot and capitalise on windows of opportunity, since change is seldom smooth and continuous. In the UK, one such window of opportunity is the new version of the Research Assessment Exercise (RAE), the enormously influential scheme by which UK universities are assessed for state funding. The next round of the RAE, now renamed the ‘Research Excellence Framework’ concludes in 2014. Importantly, it will allocate 20% to impact, defined as ‘reach and significance’. Could DFID and other funders pick that up and use it in their own assessments?

How else could DFID help turn this around? It has a lot of clout, largely coming from its sizeable research budget (about £200m a year last time I looked). Here’s a few ideas, in no particular order and mixing up sticks and carrots.

How to get researchers to understand the minds and lives of the non-researchers they hope to influence? How about insisting that any recipient of a DFID research grant not only identifies the non-academic targets of their research, but gets credit if they manage to arrange to shadow these targets for a few days to find out how they absorb and use information (I learned more about advocacy from shadowing a UK Development Minister for a day than from dozens of workshops).

Publish (and require recipients to publish) stats on blogging, citations in the media (not just journals) and any other indicators of communications and/or impact, by named academics, in order to generate some positive competition. Let the league tables commence….

Start ‘a window of opportunity fund’ that specifically excludes new research in favour of funding previous or actual research recipients to rapidly repackage existing research in response to major new opportunities in terms of demands for new thinking – e.g. change of leadership in target institution, scandal, external shock etc.

In funding applications, insist on a proper power analysis/theory of change, including which target institutions are to be influenced, what the opportunity timetable looks like (eg new legislation or drawing up manifestos). If anyone limits their ToC to ‘changing the discourse’, they should probably be taken out and shot (unless they can plausibly suggest how they aim to achieve that).

Ask researchers to explain how they will involve both influencing targets and communications people in the governance of their research from the outset (rather than completing the research and then saying, ‘oh blimey, how do we communicate this to keep DFID happy, we’d better organize a seminar and send a copy to the Minister’).

There are also risks here – people are sometimes scarily ready to blur/erase the boundaries between advocacy and impartial academic research – more on that to follow.

I’m sure there are lots of other ideas – please send them in

Previous thoughts on getting research into policy here and here.

Other thoughts from the workshop here.

Theories of change = logframes on steroids? A discussion with DFID

‘Theories of Change is just the latest attempt to shine a light on what lies behind, what makes everything work or fail. We constantly reach for new tools, but we keep alighting on small islands and losing the big picture.’ Jake Allen, Christian Aid

I recently spoke at a half-day DFID seminar discussing a draft paper by Isabel Vogel – ‘Review of the Use of Theories of Change in international development’. The draft is here (keep clicking) – Isabel wants comments by this Friday 18 May, either on the blog, or emailed directly to info[at]isabelvogel.co.uk. She is particularly looking for examples of documented theories of change (ToCs) originating in developing countries (as opposed to donor-funded programmes).

The level of interest was impressive – 40 DFIDistas in the room, plus 7 country teams via videocon and sundry NGOs and consultanty types. My overall impression was that Monitoring, Evaluation and Learning (MEL) is driving the ToCs discussion in DFID, and not always in a good way. So in my allotted 5 minutes, I stressed that ToCs should not become a ‘logframe on steroids’ (a phrased nicked from Alfredo Ortiz) and the importance of power analysis and ToCs as a permanent aspect of the planning cycle - and not just for programmes but for policy and campaigns work.

Plus their usefulness (albeit in different ways) in all 4 quadrants of the Cynefin framework(Simple, Complicated, Complex, Chaotic – see graphic), rather than just in the simple/complicated quadrants preferred by development types. I also said we should throw away those horribly complicated ToC diagrams once we’ve finished them (lest they terrify those that follow).

The discussion confirmed these concerns. Lots of people (including many of the measurers) are fully aware of the risk and want to avoid it, but are struggling against powerful incentive structures that make it happen anyway (principally the results agenda, but also the difficulty of using non-linear ToCs in practice). Hivos, a wonderfully cerebral-but-practical Dutch NGO that has done a lot of thinking on this, talks about a broader range of ‘ToC thinking’ as a useful way to prevent it all being turned into just another toolkit (‘ticking the ToCs box?’). Rick Davies recalled that the logical framework approach was originally a separate exercise to filling in the logframe table, but they collapsed/reduced into the table due to the structure and working practices of the aid business. Might the same fate await ToCs?

What of the benefits? In addition to those discussed in previous posts, Joanna Monaghan of Comic Relief (a funder), sees ToCs as making explicit the hypotheses underlying funding decisions – ‘the rules of thumb we all carry around in our heads’. That allows partners to challenge them, if they think the funder has (gasp!) got it wrong.

People also saw ToCs as making people look at the evidence and identify what is known/unknown (that rather alarmed me – what were they doing before?), but also helping programmes adapt more quickly as new evidence emerges. From the MEL end, an explicit ToC also allows a discussion with beneficiaries on what indicators to measure progress against (rather than the funder just imposing them from outside).

ToC challenges
When it came to the challenges of implementing ToCs, the big headache is how to balance donor accountability (reflected in the pressure for measurement and results, and holding partners to account against pre-agreed plans), and the ability to use ToCs intelligently to learn and adapt to changing environments.

ToCs are about people engaging intelligently with the complexity and nuance of context and process. But how do you rigorously assess the quality of people’s thought? The development community usually focuses on process and outcomes, whereas ToCs may demand something more like academic assessment on how deeply people are thinking about things. ‘Accountability has to be about trying hard enough. We never ask questions about critical thinking, only about delivery on a set of results which 5 years ago we thought we would be able to achieve.’ Stand by for quasi-professorial marks for project proposals (‘beta minus, must try harder’).

The more practical types worried over how you can balance constantly revisiting/revising a ToC with the need to get on and actually, you know, do something. One answer: pre-agree circuit breaker reviews at e.g. one year, two years into the project, when everyone knows the ToC is up for grabs; another – test (and fund) a series of ToCs in a pilot stage before deciding on a final ToC – a bit like the DFID-funded research programme consortia, which include (and finance) an ‘inception phase’ during which the recipient is allowed to test and finalise their research plans for the subsequent years. Perhaps there also needs to be a clear process for designated people to have access to a ‘red button’ change of direction in response to major contextual shifts that require a rapid revision of the plan (‘Mugabe dies’).

If failure is indeed a source of ideas etc, we need to create a safe environment to recognize, communicate and learn from it. That requires a shift in culture and incentives – e.g. circuit breaker reviews must have a convincing discussion on failure and what we’ve learned – if a project can’t demonstrate failure as well as learn from it, it probably isn’t trying hard enough.

Another plea from the practical peeps – can we separate out communities of practice from communities of theory, otherwise the practitioners are cowed into silence by the theory wallahs sounding off (who could they be thinking of?)

One final random thought: Is this (i.e. funding projects with plural ToCs, greater appetite for risk of failure etc) a suitable role for philanthropic foundations who are more able to take risks on failure than publicly funded donors?

Responding to emergencies: is the centre of gravity shifting away from the UN towards national goverments?

Ed Cairns, Oxfam’s Senior Policy Adviser on this kind of thing, explores the shifting sands of humanitarian aid policy

After a flurry of reviews and new policies on all aspects of aid, the UK put out its new humanitarian policy, Saving Lives, Preventing Suffering and Building Resilience, a couple of weeks ago. Not every word is new – it owes a lot to the review DFID commissioned from Paddy Ashdown, former High Representative in Bosnia, earlier this year.

More importantly though, it’s rather good – in some parts even brave. If the UK really means to implement its new policy, we’re going to see some improvements. Not that the UK is a bad humanitarian donor. Far from it. It’s been the third largest humanitarian donor, but, according to the independent Humanitarian Response Index, the UK ranks only 8th among OECD donors for spending its money according to agreed international principles. Even in humanitarian action, the UK just can’t shake off the suspicion that it’s sometimes peddling its political or security interests. A lot of this stems from the UK’s continuing closeness to the US. In 2010, it was the US and UK, according to UN insiders in Pakistan, that tried to pressurise humanitarian agencies to make use of NATO’s ‘air bridge’ to fly in flood relief – an initiative that had more to do with promoting NATO than creating an effective disaster response.

Now though, DFID’s new humanitarian policy speaks passionately about ‘preserving the civilian nature of humanitarian assistance’ and the threat that ‘access may be compromised when humanitarian aid is perceived to be linked to political or military goals’. Quite so.

More fundamentally, the new DFID policy puts building long-term resilience to disasters at its very heart – ‘a core part of DFID’s approach in all the countries where we work’. As we see in the Horn at the moment, terrible disasters will always be with us. There will always be a need for relief and response. But DFID gets the big point: that a much greater focus of humanitarian action should be on helping countries and communities build up their own strength to cope with disasters – which means that the long-term development and humanitarian enterprises are not so different after all. We need not only the leadership – often lacking – to ‘call’ a disaster at the first warning signs, but the far longer-term work of making people less vulnerable to disasters forever.

Where DFID is quite brave is in criticising the UN – though still too mildly, in my view. In the humanitarian field, ‘the quality of UN leadership is not consistently good’.  DFID is British after all; it does understatement, but over the last couple of years, DFID has seemed significantly more critical of UN performance than when, in the middle of the last decade, it encouraged the UN’s Humanitarian Response Review, and put its faith in the ‘cluster system’ and other conclusions from the review to deliver results. In sorrow rather than anger, most humanitarians now see those results as modest and disappointing. In the bigger picture, this December will mark 20 years since the UN General Assembly set up what is now OCHA, the UN Office for Coordination for Humanitarian Affairs, to ‘make more effective the collective efforts of the international community, in particular the UN system, in providing humanitarian assistance’. We have come a long way since the 1980s, but after overstretched responses to Haiti’s earthquake or Pakistan’s flood in 2010, who now believes that UN humanitarian performance has improved as much as it should have in those two decades?

Indeed, it is perhaps that very overstretch – an international humanitarian system incapable of responding to the world’s increasing humanitarian need – that DFID does not yet quite grasp.  For all its many strengths, UK policy still looks like a document written in a Western-centred world, in which international humanitarian action is the most important thing.

Think again. International aid reached only half the 14 million people affected by last year’s Pakistan floods. An enormous proportion of assistance to disaster victims already comes from within their communities, local authorities and Southern-based organisations. That proportion is likely to expand. While it is shocking how little international donors give in humanitarian aid – around $60 for each person in need – nobody believes that in the current economy that amount will substantially increase to meet the demand from a rising tide of often climate-related disasters.

DFID is, more than anything, a donor and it inevitably sees the world through a donor’s lens. If there’s something missing in its new policy, it’s the sense that the world is moving from a Western-inspired, UN-centric model of humanitarian response, to something much more diverse, localised and sustainable. And that means that donors should think more – even more, to be fair, than DFID already does – about substantially increasing their support to disaster-affected countries to build both resilience, and their immediate capacity to respond.

Does India need aid and if so, what kind?

Thought provoking piece on India and aid from Andy Sumner, whose paper showing that 72% of the world’s poor now live in middle income countries (MICs) caused a big stir last year. Andy has now become a contributor to the Global Dashboard blog.

The UK is currently tying itself in knots over whether to give aid to a country that has a third of the world’s poor people, is the biggest recipient of British aid over the last decade, but also has $300bn in foreign exchange reserves (400 times more than the UK’s aid contribution), and a space programme. Tricky, eh?

Andy proposes three ways DFID and other donors could do things differently:

“i. Focus on poor people, not poor countries If the focus of aid is poor people not poor countries then the low income/middle income country way of looking at the world needs a rethink. DFID could switch its aid allocation metrics to fit DFID’s mission – ie from poor countries (low or middle income countries) to poor people.

ii. Think beyond traditional aid Maybe aid is no longer about money transfers with MICs. There is the ‘do no harm’ agenda – designing favourable and coherent development policies on remittances and migration, trade preferences, climate negotiations and climate financing, as well as tax havens – and a case for making aid increasingly about global public goods (and the importance of MICs support to these) and multilateralism, especially in middle-income countries and where aid might be channelled through the United Nations Children’s Fund, for example, or a new global fund for cash transfers to households as a direct poverty and redistributive measure.

iii. See equity and shared prosperity as a global and donor concern not just a domestic issue More equitable countries reduce poverty faster, and stubborn asset, gender or identity inequality (ie caste systems) might begin to explain persistent poverty amid wealth in MICs. This entails some thinking on what aid is for and it’s role perhaps in supporting political voice of the poor and marginalized in policy processes. Any attempt to discuss inequality will be viewed as an infringement on political sovereignty but is domestic inequality solely a domestic issue if it hinders the effectiveness of aid?”

I think (ii) – do no harm – makes absolute sense, but have my doubts about the other two.

What does focusing on poor people mean? If it’s aid donors delivering services themselves, or subsidizing the Indian government to do so, then it just seems to let the state off the hook (for example by letting it carry on subsidizing the private sector, rather than public services). The standard counterargument is that some Indian states are as large and as poor as many African countries, and so have the same claim on aid, but that strikes me as rather apolitical – Indian states like Bihar are not independent countries, but part of a large, rising power and that changes the equation.

There is another way to focus on poor people that makes more sense – helping them organize. Development happens through the construction of effective states and active citizenship, and the nature of the interaction between the two. The problem in India is not policy (there’s lots of them, many excellent, on everything from employment guarantee schemes to the right to education), or cash, (remember those reserves). The problem is implementation, and the political priority accorded to poor people. A study in Bihar showed that teachers teach on average only every second day and there are no sanctions. In the same state, one of India’s poorest, 80% of the food that is supposed to reach poor people through the distribution system goes missing (sold by corrupt bureaucrats, etc).

So one role for donors is to support poor people and their organizations in demanding accountability in the implementation of policies. The question then is how far such support can go before provoking accusations of interference in domestic politics, and whether the boundaries on such actions are even tighter for foreign governments giving aid than for multilaterals like the UN or international NGOs. Go too far and it might well be that the Indian government gets tired of the aid donors even before the donors give up on India. India’s finance minister has already dismissed aid as ‘peanuts’, and the government dislikes the aid agencies’ public narrative of 450 million poor Indians, child malnutrition, maternal mortality and so on, rather than its own preferred story of India as an emerging power.

Another option might be for aid donors to pick political winners – spotting and supporting promising exercises in state-building – states or bits of the official system at either federal or state level that are clearly committed to reducing poverty and inequality. Donor cash or technical assistance might have more impact if targeted in this way.

The bottom line is that if the Indian government really wanted to end poverty, it could do so ($300bn in reserves works out at $650 per poor Indian). Whether it does will depend much more on the evolution of domestic politics and the continued mobilisation of the Indian people, rather than whether or not DFID continues to give aid. No easy answers – as Madeleine Bunting wrote in a follow-up piece to Andy Sumner’s, some experts admit ‘changing their minds from backing to ending aid to India every other day.’

I’m with them – if you’re not confused, you aren’t thinking hard enough…….

Where have we got to on fragile states and what comes next?

Another week, another conference. This time it’s hosted by the UK development ministry, DFID, which among other things, has an impressive track record of funding research on development issues (declaration of interest – I worked for DFID for a year in 2004, and sometimes advise them on research issues). This week’s gabfest is called ‘The Politics of Poverty: Elites, Citizens and States’, and it is being held to pull together the findings of four large DFID-funded projects over the last ten years. So in terms of epistemic communities, we’re talking political scientists – the conversation so far has been pretty much an economics-free zone.

The synthesis paper of all that research is on DFID’s Research for Development website, along with links to the four research centres involved in the work.  It shows what can be achieved by a sustained research funding on a particular topic and  I’ll try and summarize/pick out highlights at some future date. For now, here are my notes on a great presentation from Tom Carothers, of the Carnegie Endowment for International Peace. Tom’s a guru on democracy and development. Sorry for the long post, but he’s worth it.

First, how did the debates stand ten years ago, when this research project got under way? Tom took note of the ‘huge movement towards ‘politics matters’’ in the development aid community, one that started in the 1990s and gathered considerable steam in this decade. The evolution of the governance debate in the 1990s meant that donors felt that they understood what policies were required to trigger development, but were frustrated by states’ incapacity to implement them. This led to a focus on state-building, but when the first efforts led to technocratic failures, aid donors began to think that political process must hold the answer, sometimes to the extreme that ‘politics became a metaphor for everything we didn’t know – everything that wouldn’t fit in the equation.’

Secondly, the rise in importance of post conflict situations meant that in countries like Mozambique, Angola, Liberia, and Cambodia, politics was ‘hitting them in the face’. Over the course of the decade, this concern with post-conflict morphed into the concept of fragile states. The focus on fragile states gained so much attention that it sometime overshadowed consideration of governance issues in non-fragile states.

Thirdly, there has been a big response to the question ‘Can we prove that good governance leads to better development outcomes?

How far have we got on these three points?

1. We have broken politics apart into more useful/researchable questions:

-         Understanding the incentives of elites and how they shape politics

-         Mapping the different forms of state-society relations

-         Understanding institutions in a political light, eg when do governments introduce progressive or substantial taxation systems?

-         Grasping the role of informal institutions

But Tom saw a deeper question: maybe having an ‘ideal’ for state-citizen evolution is neither necessary nor ideal. Are we getting to a world where we have no template, a ‘deeper anarchism’ in which we don’t know what the end of state-building should be, given that our ‘ideal forms’ seem unrealistic and many non-ideal forms are producing results in some areas?

2. Violence: three areas of progress

-         we have to understand ‘political settlements’ as the glue that keeps states together

-         there are certain core elements of the first stage of state-building: security, revenue collection etc

-         the importance of inclusion and inequality as drivers of conflict

3. He finds the question of the evidence about the value of good governance for development ‘surprisingly unsettled’, although he sees a ‘rich vein of argument and discussion’. It has been easier to prove impact at micro level, but proof at the national level has been very elusive. ‘It is impossible to explain why some regions do well without good governance, or that if good governance is introduced, it leads to growth’. That’s a pretty big hole.

Has all this research been translated into practice? Citing a recent article by Sue Unsworth in the Journal of International Development as a rich source of analysis on this question, and drawing on his own experiences, Tom pointed to DFID’s research on ‘drivers of change’ in Nigeria, where what is now called ‘political economy analysis’ helped DFID make some basic changes in its work to reflect a new understanding of the importance of coalitions as potential agents of change in a context where the entrenched elites show no incentive for developmental reforms. So yes, it has had an influence, but only on some governments and projects. Why has it been so hard to introduce political thinking more generally into donor behaviour?

-         Aid organizations have a huge historical investment in technocratic approaches – that is very hard to change

-         This knowledge (on complex political interactions between states and citizens) is hard to know what to do with as an outsider. Research always stresses ‘there is no recipe’, because of complexity and context – not much help for policy makers.

-         It involves acceptance of much more limited goals for development actors – a very difficult lesson for the development industry.

More heretically, he thinks there may be a fundamental conflict between the Paris and Accra principles on aid effectiveness and taking politics seriously, which requires ‘infiltrating aid much more intensively in local political realities.’

Finally, he stood back and gave us the really big picture: these discussions are rooted in a post-Cold War interregnum characterized by a ‘certain benignness in international relations’. That assumed

-         we knew the direction of history and economic policy

-         the international community was involved in mopping up old Cold War conflicts

-         donors could get more involved in politics as governments around the world accepted the benefits of higher levels of pooled sovereignty.

But in the period that we’ve been trying to understand the political dimensions of development, these assumptions have become invalid, as part of a larger international shift away from the post-Cold War period into something different. There is much less confidence on what constitutes ‘sound economic policy’; it has become clear that conflicts are metastasizing and multiplying – Thailand, Mexico, Kyrgyzstan are not cold war hangovers. Finally, there is a ‘huge backlash’ against political intervention by outsiders, especially aid donors

So although we’ve moved far, the international context, and therefore the research questions we’re asking are changing underneath our feet. What new questions should we be asking?

And there frustratingly, hemmed in by his 20 minute limit, he left it. What are the equivalent questions on the politics of development for the next decade? Given what we now know, what political or other interventions in fragile states can an economically ailing West realistically undertake, with a decent chance of success? Answers, as ever, in the comments, please.

Postscript: see here for Foreign Policy magazine’s 2010 ‘fragile states index’, out this week, topped by Somalia, Chad and Sudan, with interactive map.

How has DFID’s thinking evolved since 1997? Four White Papers provide a rough guide.

Last month, the UK’s Department for International Development, (DFID), published its fourth White Paper since it was created as a fully fledged ministry by the incoming Labour Government in 1997. So what?  Well, it sets out the thinking for one of the more cutting edge bilateral aid organizations, so it’s at least worth a skim.

White Papers are a big deal when you work in government (as I did briefly, at DFID – declaration of interest). They provoke a cathartic and exhausting mix of debate and turf wars as all the relevant government departments seek to thrash out a common line, usually to a ridiculously short timetable, and subject to vigorous lobbying from those outside government (I’ve been on that side of the table, too). Once published, they largely disappear from public view (unless they lead directly to new legislation), but live on in the interstices of the bureaucracy as a weapon that civil servants can brandish in their internal battles.

Looking back over the four papers, you get a glimpse of how DFID’s thinking has evolved. All four papers have the title ‘Eliminating World Poverty’; their subtitles are more interesting.

‘A Challenge for the 21st Century’ (WP I,1997) sets out the stall of the new department, pledging to ‘refocus our international development efforts on the elimination of poverty and encouragement of economic growth which benefits the poor’. Out go tied aid, aid as a vehicle for foreign policy and a bunch of other dodgy practices (at least on paper).

‘Making Globalization Work for the Poor’ (WP II, 2000) picked up on the ‘globalization is good for you’ rhetoric of the turn of the millennium (the WTO’s Doha Round was launched shortly after), marking the high point of Washington Consensus thinking in DFID and triggering a (very) heated debate on the pros and cons of trade liberalization. But it also enshrined the International Development Targets (shortly to be renamed the Millennium Development Goals) as the ‘focus of all our development effort.’

‘Making Governance work for the Poor’ (WP III, 2006) amounted to saying ‘whoops, we forgot about politics’. However, since politics is a dirty word, it was sanitised as ‘governance’ – politics minus power – at both national and international level. The paper’s headline message ran:

‘Whether states are effective or not – whether they are capable of helping business grow, and of delivering services to their citizens, and are accountable and responsive to them – is the single most important factor that determines whether or not successful development takes place. Good governance requires: capability – the extent to which government has the money, people, will and legitimacy to get things done; responsiveness – the degree to which government listens to what people want and acts on it; and accountability – the process by which people are able to hold government to account.’

Despite the blind spot on power, this clearly represented a big step forward in DFID’s understanding of the political and institutional underpinnings of development, compared to the largely ‘economicist’ focus of previous white papers.

And what of Number 4? Subtitled ‘Building Our Common Future’, it reads like a stock-take of current development thinking, and rather a good one at that:

Power and politics regain their rightful place: ‘The UK will increasingly put politics at the hearts of its action. We need to understand who holds power in society, so we can forge new alliances… In the future, understanding political dynamics will shape more of our programmes’.

An interesting reframing of the debate on access to justice: ‘The UK will treat security and access to justice as a basic service, on a par with health and education, and a fundamental right.’

Putting money into supporting citizens, media and parliaments to hold governments to account – 5% of all future budget support funds will go to strengthen such efforts.

Recognition of the central role played by domestic taxation in the construction of effective states (and funding for a new International Tax Centre)

Unequivocal stance against user fees in education and health: ‘User fees in education stop children from going to school. Health costs deter families from seeking care or can plunge families into debt…. The UK favours the abolition of user fees.’

New commitments on social protection (extend SP programmes to 50 million more people in 20 countries over next 3 years)

A significant increase in attention to environmental issues, especially climate change (‘a climate crisis is already upon us… climate change will take centre stage in the UK’s international development efforts’), and the promise of extending the idea of ‘Advance Market Commitments’ to promote new low carbon technologies.

Perhaps the most innovative section of the paper is on fragile and post-conflict states. Clare Short, who was DFID’s first and most high profile minister, reckons that restoring aid to Rwanda was her greatest achievement, (at the time, aid donors shied away from the country because of the 1994 genocide) and led to some highly effective state-building. Now DFID is promising to focus more of its work and money (50% of all new bilateral country spending) on fragile states – a brave pledge, given that they are where it is hardest to ‘do’ aid, and where things are most likely to go wrong.

There is however some concern that the White Paper’s commitment to ‘integrating civilian and military efforts’ in conflicts may further blur the boundaries between humanitarian and military actors – not good news if you’re a humanitarian worker in Afghanistan.

There are of course some things you can disagree with in the paper and even more that were either weak or absent (cue interminable NGO wish list), and relations between UK NGOs and DFID get heated at times. But I can’t help feeling that DFID and its achievements may be one of the lasting legacies of the Labour Government, and would like to hear from non-UK readers about how their own aid departments measure up against the White Paper.

For more views, read Alison Evans, ODI’s new director here or Alex Evans (a former special adviser to Hilary Benn when he was development minister) here. That’s enough Evanses for now.