Aid and complex systems cont’d: timelines, incubation periods and results

I’m at one of those moments where all conversations seem to link to each other, I see complex systems everywhere, and I’m wondering whether I’mstarting to lose my marbles. Happily, lots of other people seem to be suffering from the same condition, and a bunch of us met up earlier this week with Matt Andrews, who was in the UK to promote his fab new book Limits to Institutional Reform in Development (I  rave reviewed it here). The conversation was held under Chatham House rules, so no names, no institutions etc.

Whether you work on complex systems or governance reform or fragile states, the emerging common ground seems to be around what not to do and to a lesser extent, the ‘so whats’. What can outsiders do to contribute to change in complex, unpredictable situations where, whether due to domestic opposition or sheer irrelevance to actual context, imported blueprints and ‘best practice guidelines’ are unlikely to get anywhere?

In his book Matt boils down his considerable experience at the World Bank and Harvard into a proposal for ‘PDIA’ – Problem Driven iterative adaptation, which I described pretty fully in my review. The conversation this week fleshed out that approach and added some interesting new angles.

PDIA needs funding, but not big million dollar cheques that come with all the paraphernalia of targets, milestones, logframes etc that are more likely to kill thought than promote experimentation and learning. Instead, it needs a trust fund approach – lots of small grants that allow incubation of local solutions to a given problem while ‘avoiding a premature results agenda’.

But does that mean that institutional reform should avoid the big aid dollars altogether? Matt thought not – he portrayed PDIA as a new and extended incubation phase, which can then take the homegrown solutions that emerge and move into the more traditional aid world of large scale, large budget programming. So the challenge for aid agencies is how to create, fund and protect a space within their institutions for small budget experimentation and incubation, sitting in parallel with the big stuff.

Timelines emerged as a useful, but undervalued tool. But these are timelines of what has actually happened in the past, not the imaginary future timelines of funding applications. Matt reckons any project seeking funding should start by building a 20 year timeline of what has happened on that issue/in that locality. If done properly, the exercise of reconstructing the timeline using documents and interviews will reveal overlapping interpretations of what actually happened and recover the kinds of knowledge and experiences that all too often go missing in Aid World as staff leave and projects are wound up. We need a decent timeline methodology – Matt uses the work of Peter Hall at Harvard but it also sounds a lot like process tracing, something our MEL team uses.

The issue of narratives is central – it lies at the heart of the response to a reductionist results agenda that privileges pseudo medical trial data over real experience. Claire Melamed likes to say ‘the plural of anecdote is not data’. True, but I think that a well researched anecdote rapidly becomes a ‘narrative’, and the plural of narrative can definitely be evidence, if not data. Matt, ODI and Oxfam are all separately thinking about the need to build a collection of rigorous, nuanced narratives on stories of power and change – we’ll be swapping notes and hopefully coming up with some ideas for working together on this.

There was a good discussion on what constitutes ‘results’. Good PDIA-type work in developing countries requires a rapid feedback loop of results, but of a different kind to those typically demanded by the aid business. Developing country politicians want to know what’s happening with their money, what has been learned, what has worked and what hasn’t, and how the project has responded. They don’t need the (often bogus) certainty and data demanded by aid planners.

I do find this all slightly baffling – politicians intrinsically know how to navigate in complex environments, respond to shocks and opportunities, using trial and error, instinct and rules of thumb. They make decision on partial information and change direction if things don’t work. That’s what politics is about. But then they become aid ministers in donor countries, and suddenly buy into a paraphernalia of logframes and a particular understanding ofresults that in some other part of their brains they must know has huge limitations in the real world. How to get ministers to think more like pols and less like aid bureaucrats?

All fascinating and thanks to Matt for kicking off and CGD Europe for organizing the discussion (am I allowed to say that under Chatham House rules? If not, please ignore). I’m thinking of writing a paper on the ‘so whats’ of complex systems, but will first wade through the draft of Ben Ramalingam’s forthcoming book before deciding whether it’s necessary.

Citizens Against Corruption: What Works? Findings from 200 projects in 53 Countries

I attended a panel + booklaunch on the theme of ‘Citizens Against Corruption’ at the ODI last week. After all the recent agonizing and self-doubt of the results debate (‘really, do we know anything about the impact of our work? How can we be sure?’), it was refreshing to be carried away on a wave of conviction and passion. The author of the book, Pierre Landell-Mills is in no doubt – citizen action can have a massive impact in countering corruption and improving the lives of poor people, almost irrespective of the political context.

The book captures the experience of the Partnership for Transparency Fund, set up by Pierre in 2000. It summarizes experiences from 200 case studies in 53 countries. This has included everything from using boy scouts to stop the ‘disappearance’ of textbooks in the Philippines to introducing a new code of ethics for Mongolia’s judiciary. The PTF’s model of change is really interesting. In terms of the project itself:

-          Entirely demand led: it waits for civil society organizations (CSOs) to come up with proposals, and funds about one in five

-          $25k + an expert: the typical project consists of a small grant, and a volunteer expert, usually a retiree from aid agencies or governments, North and South. According to Pierre ‘the clue to PTF’s success has been marrying high quality expertise with the energy and guts of young activists’. (I’ve now added ‘Grey Wonks’ to my ‘Grey Panthers’ rant on why the aid world is so bad at making the most of older people).

-          The PTF is tapping into a zeitgeist of shifting global norms on corruption, epitomised by the UN Convention Against Corruption (2003). The idea that ‘they work for us’ seems to be gaining ground.

-          The PTF prefers cooperation to conflict – better to work with champions within the state (and there nearly always are some, if you can find them), than just to lob rocks from the sidelines (although some rock-lobbing may also be required).

-          It also prefers action and avoids funding ‘awareness-raising’, ‘capacity building’ and other ‘conference-building measures.’

So what works? On the basis of the case studies (chapters on India, Mongolia, Uganda and the Philippines), and his vast experience of governance and corruption work, Pierre sets out a ‘stylized programme’ for the kinds of CSO-led initiatives that deliver the goods:

1. Nail down the problem: use surveys, focus groups, right to information laws where they exist
2. Come up with (and implement) an action plan: get people involved with community report cards, community radio, public hearings and other approaches
3. Propose ideas for ways to reform the system or reduce the opportunities for corruption, drawing on the results of (1) and (2)
4. Discuss the ideas with stakeholders and amend
5. Campaign to persuade officials and politicians to adopt the ideas
6. Once you’ve won (bit of a leap, that – see cartoon) monitor the implementation of any measures introduced to reduce corruption.

This may look like a bit of a blueprint, but actually it isn’t – the PTF fits the model of how to work in complex systems pretty well. It acknowledges that outsiders can’t possibly understand the labyrinths of formal and informal power, or identify potential allies and windows of opportunity. Those have to come from within. By breaking funding down into small grants, and using only volunteer experts, it tries to keep power away from the consultancy/donor complex, and stay true to being country-driven. At the ODI, Pierre described the underlying theory of change as ‘the aggregation of millions of actions to reach a tipping point.’

He also expanded on the problem of aid institutions. Anti-corruption campaigning is often long-term, over 25-50 year time horizons. That means aid donors can support particular phases, but if they don’t have the staying power to see the work through, they need to avoid trying to control it. Unfortunately, ‘politicians and officials who think they can make their mark are the biggest menace for this work’.

Despite this critique, the book is a pitch for funding from the aid agencies, although Pierre believes that in the long term CSO anti-corruption work will have to find alternatives sources.

Which all sounds great, but the results debate is obviously getting to me, because I did have some sympathy with DFID’s Mark Robinson, who said at the ODI that although the UK Government (which has been a core funder of PTF) ‘is increasingly persuaded about the value of citizens’ transparency and accountability initiatives’, we really can’t be expected to judge PTF entirely on the uplifting case studies and stats collected by, errrm, the PTF.

I raised another issue: the rhythm of civil society action is almost always episodic – long periods of tranquillity (people getting on with their lives), punctuated by episodic spikes of protest. Attempts to turn this dynamic into some kind of permanent state of mobilization are probably destined for frustration and failure. Between spikes, the long term work of renewing or changing social capital, social norms and values etc takes place in the more permanent ‘grains’ of civil society – trades unions, neighbourhood associations, religious communities – that endure between spikes. It wasn’t clear that PTF understands and works with this – it seems to have permanent mobilization as its underlying model of how civil society works.

PTF seems to belong to a family of ‘post supply side’ approaches to governance, which also includes the International Budget Partnership, the research of Matt Andrews or the Africa Power and Politics Programme, as well as Oxfam’s own work on governance and accountability.

What they have in common is the need to move from ‘best practice’ to ‘best fit’, to identify and support locally driven initiatives, and to support coalitions between champions within the system and those outside. Where they seem to differ is on the prominence of civil society in these discussions – at one end of the spectrum is PTF’s perhaps excessive glorification of its role; at the other the APPP’s rather contemptuous dismissal of civil society as irrelevant to the ‘real’ Paul Kagame world of big men and decent chaps sorting out political settlements (’citizen pressure is at best a weak factor and at worst a distraction from dealing with the main drivers of bad governance.’) I would love to see APPP’s David Booth and Pierre Landell-Mills go head to head on this.

To be continued, I suspect (not least because Matt Andrews is in London this week).

How to Plan when you don’t know what is going to happen? Redesigning aid for complex systems

They’re funny things, speaker tours. On the face of it, you go from venue to venue, churning out the same presentation – more wonk-n-roll than rock-n-roll. But you are also testing your arguments, adding slides where there are holes, deleting ones that don’t work. Before long the talk has morphed into something very different.

So where did I end up after my most recent attempt to promote FP2P in the US and Canada? The basic talk is still ‘What’s Hot and What’s Not in Development’ – the title I’ve used in UK, India, South Africa etc. But the content has evolved. In particular, the question of complex systems provoked by far the most discussion.

I started off with the infamous US military mindmap of Afghanistan. Although ridiculed at the time, the map looks like a genuine and nuanced effort to understand the country and is probably fairly typical of the complexity of power and relationships in any given country. The point is that such a system is complex, not complicated. Complicated means if you study it hard, you can predict what happens when you intervene. In contrast a complex system has so many feedback loops and uncertainties that you can never know how it will react to a stimulus (say $100m in aid, or an invasion….).

The crucial point is that most political, social and economic systems look like the map. Yet the aid business insists on pursuing a linear model of change, either explicitly, or implicitly because a ‘good’ funding application has a clear set of activities, outputs, outcomes and a MEL system that can attribute any change to the project’s activities – a highly linear approach. Other organizations – say forest fire managers, or the military, seem more able to cope with complexity, although I found out from a woman in one seminar who had served in Afghanistan that the power map was actually drawn up by a consultant, who was promptly sacked after showing the slide to General Petraeus, so maybe the soldiers aren’t so comfortable with complexity after all.

In denying complexity is obliged either to seek islands of linearity in a complex system (vaccines, bed nets), which may not always be the most useful or effective places to engage, or to lie – writing up project reports to turn the experience of ‘making it up as you go along’ that epitomises working in complex systems into the magical world of linear project implementation, ‘roll out’, ‘best practice’ and all the rest. That not only wastes a lot of staff time and energy, it also reduces the ability to learn about how to work best in complex systems.

So how should the aid system change? Overall, we need to think though ‘How to plan when you don’t know what is going to happen’ (my best effort at explaining complexity without resorting to jargon). Here are my bullet points, and brief explanations:

Fast feedback: if you don’t know what is going to happen, you have to detect changes in real time, but also have the institutions to respond to thatinformation (as was not the case recently in the Sahel).

Focus on problems, not solutions: Drawing on Matt Andrews’ work, the role of outsiders is to identify and amplify problems, but leave the search for solutions to local institutions. (At the World Bank, Shanta Devarajan pointed out the contradiction between this approach and NGOs’ preference for big, simple solutions – end land grabs, no to user fees. Ouch.)

Rules of thumb, not best practice toolkits: I am told that the US marines do not go into combat brandishing Oxfam toolkits and online resources on best practice. They operate on rules of thumb – take the high ground, stay in communications and keep moving. They improvise the rest. Aid workers on the ground operate far more like this than our project reports admit. If we were honest about it, we could have a better discussion on how to improve those rules of thumb.

Some possible approaches that spring to mind (and I would love to hear examples of others)

Work on the ‘enabling environment’ rather than specific projects: things like norms, rights or access to information

Evolutionary/Venture Capitalist approach: run multiple experiments and then zero in on what seems to be working best. Example, the Chukua Hatua project in Tanzania

Convening and Brokering: Get dissimilar local players together to find solutions – the outsiders’ job is to support that search, not do it themselves. Example, the TAJWSS water project in Tajikistan

But any attempt to move in this direction raises some fundamental challenges to the current structures of the aid industry:

Results for grown ups: The current approach to measuring results favours linearity. But rejecting results altogether is the wrong approach – both because even those who recognize the central role of complex systems still want to know if they’re doing any good, and because the results people control the cash. No results, no funding. We need to get much better at ‘counting what counts’, and reclaim the idea of ‘rigour’ for qualitative and other methods better suited to complex systems.

Who to employ? Risk-taking, entrepreneurial, maverick searcher types have a hard time in an aid business dominated by bureaucratic procedures and risk aversion. Moreover, working in complex systems requires deep local knowledge of formal and informal power maps, something expats on a two or three year posting are unlikely to acquire. How do we turn the tables to attract and retain searchers, and value locally embedded knowledge?

Short Term v Long Term: Funding and project cycles are short term, change in complex systems is often long term. How can we bridge the gap, for example by combining good, plausible stories about the short term, with more rigorous impact assessment in the long term (how often do we go back and study the effects of an intervention 10 or 20 years after the funding has ended?)

How to keep/build political support given that working in complex systems means acknowledging a lack of control over what takes place and limits to attribution (no you can’t ‘badge’ the Arab Spring as created by Oxfam, USAID or anyone else, sorry). It also means greater tolerance of failure – a venture capitalist approach means accepting 9 failed start-ups for every 1 big success, but imagine what aid critics would do with a 90% failure rate. And how do we communicate and sell this approach to the public after systematically dumbing down the aid and development story for decades? (From buy a goat and save the world, to a post-goat narrative….)

Ben Ramalingam has been thinking about this for years, and writing about it on his Aid on the Edge of Chaos blog. His book of the same name is due out later this year, so let’s hope it can settle a lot of these issues (and doubtless raise many more).

Impressions of North America’s aid and development scene: the good, the bad and the ugly

Just got back from a two week immersion in the US & Canada aid and development scene (well, the East Coast version, anyway). Boston, New York,Washington and Ottawa, talking at universities, NGOs, multilaterals and aid agencies and experiencing a wonk version of groundhog day + powerpoint, brought on by giving the same presentation 16 times (I’m getting pretty good at it now).

Overall impressions?  Lots of really smart and committed people caught between what Oxfam America’s Greg Adams calls the ‘high and low politics’ of aid. High politics is about policy – thoughtful discussions of how to make aid better; low politics is fending off the ‘aid doesn’t work/charity begins at home’ counter attack from right wingers and fiscal conservatives.

In Canada, it felt like low politics is in the ascendant – the aid community seems besieged as the government takes the axe to a number of institutions, including ‘merging’ CIDA with the Department of Foreign Affairs and International Trade (feels more like an acquisition than a merger).

The US felt more finely balanced – lots of good reform proposals coming from the Administration, and a really interesting discussion with USAID on how to move from funding relationships to partnerships like its triangular relationship with Brazil, where USAID and Brazil jointly support aid programmes in Lusophone Africa. They’re wondering how to expand that approach as more middle income countries set up their own aid agencies.

For all my admiration for their blogging, I found my day or so at the World Bank pretty depressing in terms of politics and policy. The Bank seems stuck in a ‘technology + private sector = solution to everything’ mindset. I’m not against either, but you have to take politics, power, institutions etc at least as seriously.

I’ve already covered my exchange with Marcelo Giugnale. At my staff talk, Bank uberblogger Shanta Devarajan stated ‘poverty is a series of government failures’ and came out with examples where ‘governments intervene, but make people worse off.’ Unfortunately his conclusion seemed not to be that the Bank should help strengthen states, but that it should bypass governments/find private sector solutions to everything. An approach that is unlikely to reduce inequality and has little historical foundation, I fear.

As for the evidence debate, Shanta reckoned ‘results always have to be relative to a counterfactual – that’s what they’re about’. So how do you assess things with no counterfactual, like the fall of apartheid, or the invasion of Afghanistan? Or the impact of international conventions on the rights of women?

[update: Shanta says I got him all wrong - see his comment below]

Meanwhile a discussion with the team producing the forthcoming World Development Report on Managing Risk suggested that the Bank still cannot get past its traditional technocratic approach of ‘if a state wants to improve, here are some suggestions’. On fragile states, what if a state isn’t interested in solutions? Reply – private sector + foreign investment. Oh dear. No theory of change for how fragile states turn around, finding nuclei of good governance in otherwise fragile states, building coalitions of civil society, faith-based institutions, media, academia, traditional authorities, shifting norms in the next generation. Nope, just a fairly barren state v private sector dichotomy. Still, these were rushed conversations, and I’d be delighted to be proved wrong.

Other impressions? Great intellectual capacity at the UN, frustrated by the lack of clarity and political constraints of the system. A professor who still remembers her first class with Robert Chambers. Robert had pinned up a map of the world, with the North at the bottom. Then he just sat in a corner as his new students filed in and commented that he’d put the map up upside down. You can imagine the rest. Genius.

And a great suggestion from someone (sorry can’t remember who) – a ‘voices of the activists’ study on lightbulb moments: what were the life-changing experiences that set you on your present course – a meeting with an individual? A personal experience of violence or injustice? A seminar (hey, it happens)? Something you read? That is research I would love to read.

Dogs that didn’t bark? Surprisingly little discussion on the rise of China, depressingly little on climate change. Otherwise, my over-riding impression of the trip is the network of smart, committed people who read this blog, comment, think and argue with passion. Thankyou – you have definitely renewed my commitment to keeping this forum going, even though it can be daunting when (as this morning) I wake up jetlagged, with nothing ready to post. Normal service will be resumed tomorrow.

Update: here’s the video of one version (at Oxfam America) of my ‘what’s hot and what’s not: new thinking in development’ presentation.

Is power and politics a massive distraction? Crossing swords with the World Bank.

This post is written on the hoof, dashing between presentations, so please pardon the rough edges.

Yesterday I shared a platform with Marcelo Giugale, the World Bank’s Africa Director for Poverty Reduction and Economic Management (right). We were coming from very different places, some might say different planets, which is always stimulating. I did my standard power and politics spiel, focusing on multidimensional poverty, inequality and complex systems and their implications for aid agencies (more on that to follow).

Marcelo responded by saying that this was all a massive distraction, and that we should keep our eyes on the prize of ending poverty. And on this he was relentlessly upbeat, optimistic and pretty apolitical. ‘We can end poverty without blasting the system… we have the technology’ he said.

Marcelo argued that six key developments have made this possible:

• We will know the poor by name, individually. Thanks to a combination of technology and the widespread introduction of cash transfers, governments are increasingly registering all their poor citizens (the mega example being India’s biometric identity card programme – below, left). This allows them to scale up transfers rapidly in the event of shocks.
• We can determine impact, not just outcome. He defined impact as ‘that subset of outcomes that would not have happened without the intervention’ and pointed out that many of them are negative. Eg aid agencies give aid for education, so the education budget is redirected to something less worthwhile.
• ‘The time has come to link people with their natural resources.’ The World Bank seems to be getting behind the ‘doing an Alaska’ proposal to distribute natural resource revenues straight into the hands of poor people. Interestingly their power analysis suggests that the most likely way to overcome domestic political barriers (politicians not wanting to give up their slush funds) is by persuading ‘desperate oppositions’ who do not expect to win to adopt it as a last throw of the dice. Something a bit similar led to the introduction of India’s renowned Rural Employment Guarantee scheme. They think early adopters will ease the political logjam and increase pressure on neighbouring countries to follow suit.
• Equity not Equality: the way to steer a course through the politically polarized terrain of inequality is to focus on children. Hence the Bank’s new Human Opportunity Index, which asks ‘how important are a child’s  personal circumstances  over which he/she has no control or responsibility (e.g., gender, family income, skin colour, birthplace, etc), to his/her probability to access the services without which he/she can’t succeed in life (things like completing 6^th grade on time or having potable water in the first two years of life)?’ I’m not sure about this – is it a way to get at the real causes of inequality, breaking the transmission between generations that has grown so much more rigid in recent years. Or is it a convenient way of dodging politically contentious issues of distribution and redistribution, kicking the can down the road with a new version of the kind of ‘equality of opportunity’ approach (aka the American Dream), which I thought we had left behind?
• Focus on non-cognitive skills, such as punctuality, respect and dedication to understand the reasons for success. Why? Because they are important and becoming more measurable.
• A proliferating set of ‘standards’ for public expenditure will help governments to introduce results-based payments and budgeting.

Most of this is taken from his (freely downloadable) 2010 book The Day After Tomorrow.

Several things struck me about his presentation. Firstly, the overwhelming can-do optimism is very seductive. And the emphasis on technology neatly avoids any difficult political decisions. This is a happy technocratic world of win-wins. In contrast my presentation was all about difficult politics – I’m not sure I had the best tunes.

But in the end, I didn’t buy a lot of it – by invoking the use of ‘we’, as in ‘we can end poverty, by fixing X or Y’, he reminded me of Pierre Jacquet’s great question – who is we? And why assume that ‘we’ have a common agenda?

Marcelo has a remarkably outsiderish view of the ‘we’ – in a follow-up email he defined them as “All those that care about ending poverty, not just 19^th Street, but NGOs, advocacy groups,  faith-based organizations, the college kid that spends a year in a developing country giving a  hand, etc”.

In contrast, I would argue that these are all bit players: the key ‘we’ is within developing countries – political actors, civil society organizations, faith leaders and the rest. There, assumptions of a common agenda are likely to prove unfounded. That’s why we need to go back to school on power and politics. Which all reminded me of Matt Andrews’ critique of the World Bank’s efforts to ‘roll out best practice’ on institutional reform, including the institutions needed to introduce these new technologies.

Today I’m launching the book at the World Bank at 12.30, so expect the debates to continue……

How to end foreign aid and avoid a punch-up

An edited version of this piece appeared on the Guardian’s Comment is Free site on Saturday

The spat between South Africa and Britain over ending its (very small) aid programme has sparked another round of debate about whether British aidshould be going to middle income countries (the last round was over aid to India, which seems to particularly rile the Daily Mail).

But whatever the rights and wrongs of ending aid to South Africa (whose economy is growing slowly, but with sky high levels of inequality and 10 per cent of all the world’s people living with HIV and Aids), aid agencies are inevitably going to have to shift money around as the world changes. Countries rise and fall, aid priorities change and new opportunities (like the opening up of Myanmar) will arise, to which aid should of course respond.

That churning process is accelerating as more and more countries reduce their dependence on aid thanks to economic growth, rising revenues from oil and gas and surging remittances from their migrant workers overseas. Those upward curves contrast with falling aid volumes.  Total global aid flows have been falling for the last two years as, with the sole glorious exception of the UK, the ‘Austerians’ in what are sometimes known as ‘the formerly rich countries’ take the axe to aid spending.

What’s more, many developing country governments can’t wait to end aid – it affronts the dignity of the Big Men (as they usually are) in charge to be seen as asking for handouts. As two Ugandan ministers once proudly told me ‘when 60% of our revenues came from aid, we had to go to the World Bank on both knees. Now we’ve got it down to 30%.’ (Unfortunately, I blew it by joking ‘ah, so do you just go on one knee now, then?’ They were not amused.)

So the issue is not whether aid partnerships sometimes have to end, but how. Oxfam has some ideas on that – several pages of guidelines on ‘exit planning’ in fact. They are about ending funding to particular grassroots organisations, but they offer useful lessons for DFID and others contemplating exits from entire countries. Talk to the partner from the outset, help them fill any organisational gaps and weaknesses before you go; agree (don’t impose) a clear timetable for phasing out funding and crucially, agree what kind of relationship you want once the cash tap has been turned off. Because as one Oxfam partner from Pakistan put it, ‘We want to be treated as a “relationship” and not just as a “partner”’. Partnerships are for projects, but relationships go beyond that. We should have a relationship in the future, even if there is no project money.’‘ Sadly, the unpleasantness over DFID’s exit from South Africa means that remaining relationship has got off to a rocky start.

I imagine DFID has similar guidelines, but those can’t legislate for ministers shooting from the hip (or if conspiracy theorists are to be believed, throwing a bone to the Tory right ahead of local elections last week). Still, if civil servants in the land of ‘Yes Minister’ can’t manage their political masters, what are we coming to?

Joking aside, this matters. Aid agencies need a clear understanding of what constitutes a responsible exit. After all, aid should bequeath a legacy of trust and friendship between the UK and the rising powers of Africa and Asia, (who incidentally, we are going to need in future), but in this case that legacy has been (temporarily, I hope) squandered. Ending the aid relationship should be a moment of mutual celebration, not public mud-slinging.

Update: and here’s a new report on the subject of responsible exit

The Limits of Institutional Reform in Development: a big new book by Matt Andrews

There’s nothing like an impending meeting with the author to make you dig out your scrounged review copy of his book. So I spent my flight to Boston last week reading Limits (sorry the full title is just too clunky).  And luckily for the dinner conversation, I loved it.

Limits is about why change doesn’t happen, and how it could. It synthesizes the ‘groundswell’ of disquiet about the failure of the governance and institutional reforms that have been promoted for many years now by aid agencies like the World Bank. And it’s not just a whinge – there are plenty of ideas for how aid agencies can do better. The book is particularly useful for those working on fragile states – lots of the positive examples (as well as some failures) come from Afghanistan, Ivory Coast and elsewhere, although there is a bit of ‘why can’t everywhere be more like Rwanda?’ in there too.

Overall, the approach reminded me of Dani Rodrik’s great book, In Search of Prosperity, and Matt says Rodrik (a fellow Harvard prof) was influential in pushing him to nail down the always-elusive ‘so whats’.

Limits summarizes research and thinking from disparate disciplines, with lots of fascinating case studies (he’s put in the legwork to build a serious empirical basis for his conclusions). His big idea is captured in a new acronym, PDIA (Problem-Driven Iterative Adaptation), which, as he pointed out, is similar to the Participatory Institutional Appraisal idea I raised in a recent blog. I’m not sure if PDIA will catch on – it could have done with a snappier title, as could the book – but the content is really important if you are interested in aid, institutions or governance.

So what does it say? Firstly, that we have a big failure on our hands. The spate of projects and programmes around institutional reform has at best a mixed record of success; in many countries institutions have actually deteriorated in terms of effectiveness, corruption etc.

Limits argues that governments’ real motive for committing to reforms is often not about improving performance, but is actually about ‘signalling’ a willingness to ‘modernize’ (which usually means move power from state to market, deregulation and privatization, increase budget controls and  accountability and reduce debt). It often involves ‘isomorphic mimicry’ – if poor countries mimic the institutions of rich ones, then – voila! – they too will become rich. The trouble is that the current aid system rewards such signalling. When the reform fails, a new government typically introduces a new round of signalling and off we go again.

Uganda is the Daniel Day Lewis of isomorphic mimicry: according to the think tank Global Integrity, it has the best anti-corruption laws in the world, (it scores 99/100), but came 126^th in the 2008 Transparency International Corruption Perceptions Index. Oops. More generally ‘developing countries are now more likely than developed countries to boast systems that resemble international best practice.’ So if laws and best practice were decisive, Uganda would rapidly be overtaking Norway.

Such reforms as do take place happen on the fringes of real power ‘in areas that are externally visible and where reform is influenced by concentrated sets of reform champions.’ Eg the ‘ceremonial’ world of Poverty Reduction Strategy Papers (PRSPs). Or perhaps (at the risk of sounding like a bad loser) the MDGs…..

Aid agencies often focus on identifying and supporting a small number of champions, but Limits debunks such ‘decent chap-ism’ as an ‘illusory promise’. He quotes Brecht’s ‘Life of Galileo’: ‘Unhappy is the land that has no heroes….. No. Unhappy is the land that needs heroes.’

If not single heroes, then what kind of leadership is needed for genuine reform? ‘Institutional entrepreneurs’ are essential, but there’s a paradox – those in power benefit from the status quo, so are unlikely to support change. That can change ‘when something creates a bridge between these highly embedded agents with power and low embedded agents with new ideas.’ And they often need convenors and brokers to help them overcome barriers of distrust and status.

But there’s a further group – the ‘distributed agents’ that are required to implement what the entrepreneurs come up with. And for ownership and relevance, they need to be engaged from the outset, not as ‘adopters’.

Otherwise, ‘reforms often progress well when under the control of champions in concentrated agencies directly involved in designing change, but falter when deconcentrated agencies must implement what these agencies design.’

The book examines the broader contexts for institutional reform, pointing out that there are always ‘multiple logics’ that govern how people think and act. Sometimes one logic is dominant, at other times there are strong competing alternative logics. The job of change agents, whether internal or external, is to back the good guys when there is genuine competition, but otherwise incubate alternative logics to challenge a damaging status quo. Either approach needs a deep understanding of what is there, rather than an imported blueprint for best practice.

Matt recognizes that shocks are important drivers of change, but the argument goes into much more interesting terrain than the standard spiel. Shocks disrupt, weakening the dominant logic and testing the viability of alternatives. That creates the conditions for change, but the change process itself needs to be broad and incremental – how do discontinuity and gradualism fit together? I think the idea is that shocks create the conditions for reform, but reform itself can’t be sudden.

But there may be trade-offs, as the appetite for reform may fall away soon after a shock, so the question (which the book doesn’t answer) is what do reformers need to put in place before the window of opportunity closes, to pave the way for that longer, more inclusive process? I’ve got a horrible feeling the rise of Thatcherism may provide the perfect case study here…..

What happens after shocks is a five stage process (this is new to me, from the literature on institutional change):

• Deinstitutionalization: encourage the growing discussion on the problems of the current model
• Preinstitutionalization: groups begin innovating in search of alternative logics, involving ‘distributive agents’ (eg low ranking civil servants) to demonstrate feasibility
• Theorization: proposed new institutions are explained to the broader community, needing a ‘compelling message about change.’
• Diffusion: as more ‘distributive agents’ pick it up, a new consensus emerges
• Reinstitutionalization: legitimacy (hegemony) is achieved. We all go off to the pub.

As to what outsiders can do, again he has some sensible recommendations, while desperately trying to avoid creating a new blueprint of his own:

• Focus on identifying, highlighting and exploring problems, but leave solutions to local players. Accept that this process may take time
• Provide opportunities for local actors to reflect on problems – convening and brokering
• Focus on clearing out the obstacles to new approaches (deinstitutionalization)
• Fund flexible learning-by-doing approaches to finding solutions

Specific suggestions include Cash on Delivery Aid, stringent tests for all ‘manifestations of good, better or best practice’ and creating institutional reform trust funds that can disburse smaller grants fast in response to evolving local processes.

At those happy moments when governments buy in to the need for reform (he cites Rwanda’s decentralization and Indonesia’s Corruption Eradication Commission (right) as examples), Andrews proposes ‘purposive muddling’ – slow, experimental and incremental approaches. Outsiders can contribute by exposing decision makers to experiences elsewhere, helping them develop hybrids best-suited to local contexts, and then test them. They can also capture and publicise successes to build momentum and buy-in.

Outsiders should also look beyond champions in positions of authority, and try and cultivate ‘mobilizers’ who connect different constituencies and spread ideas. An interesting survey of those involved in 12 different reform processes showed that leadership was far more dispersed than is customarily assumed – multiple leaders, often non-usual suspects (no-one in Afghanistan cited the president), such as those behind the scenes who brought people together and acted as catalysts.

The survey did identify external agents like aid agencies as important leaders, but only their locally-based staff, who are embedded in national contexts; no-one cares about visitors from HQ. Outsiders are more important at the start of reform processes (their influence tends to diminish after that). Not surprisingly, providing funding is their key role, with the key proviso that the funding is open ended and flexible, not tied to the ‘roll out’ of ‘best practice’. Overall however, outsiders are bit parts in the reform drama.

Discussing all this over dinner, Matt thinks we have arrived at a ‘moment’ – a coming together of dissidents from numerous disciplines to reject the logframe/best practice culture and push for something more rooted in reality. Political science, complexity theorists, aid veterans, Cash on Delivery proponents, the Development Leadership Program, the Africa Power and Politics Programme and many more are all challenging linear/blueprint thinking and proposing new and (hopefully) better alternatives.

In a nice twist, he applies PDIA to the task of persuading the aid agencies to adopt, erm, PDIA. He thinks the level of disruption to the signalling model is high, driven by growing evidence of failure. I’m not so sure. To steal from Robert Chambers ‘whose reality counts?’, for many aid donors right now, reality feels like political and financial siege, and that is fuelling the pursuit of a divisive emphasis on ‘results’. I’m not sure there will be much appetite for a movement, however well grounded in evidence, which says that the way to achieve change is to make it up as we go along (a sceptic’s version of PDIA) rather than to pursue short term, attributable results.

And (and this gets politically tricky for me), both the volume of aid and its management may also be obstacles to realigning it. Matt cites the World Bank’s ‘Learning and Innovation Loans’, which have been largely ignored, mainly because they are too small – an average of $5m, compared to $150m for other investment projects. As long as Bank staff are promoted on the basis of banking-style rules that reward the volume of aid they move, who is going to waste their time on LiLs? Then of course there is the ‘pre-programming’ model epitomised by detailed logframes and other project documents that require a pretence of predictability and linearity – all of it toxic to a PDIA approach. The increasing influence of governance indicators like the

CPIA that themselves enshrine ‘best practice’ at the heart of what we measure closes the conceptual circle and makes it even harder to conceive of new approaches.

As you may have realized from the quotes, the book’s language is pretty dense and technical. That, plus being published as an academic hardback, could easily reduce the book’s audience and impact. Any publishers willing to back a more popular version should beat a path to Matt’s door.

Finally, there is lots of overlap with my own work on power and change – the importance of power analysis/understanding local context, seizing critical junctures, convening and brokering rather than trying to go it alone, evolutionary learning-by-doing rather than single grand plans. Over dinner, we kicked around some exciting plans for working together in future – watch this space.

Matt is launching the book in the UK (London – ODI and CGD – and Manchester) from 20-22 May. Details here.

Is it time for a rethink on the definition of aid?

Crushed by my humiliation at the hands of Claire Melamed, it would just make matters worse to come back for another round of post-2015 jousting, so let’s move on.

I actually quite like blogging about meetings held under Chatham House rules, as they allow me to write about the discussion without worrying about who said what. And to take the credit for anything clever, of course.

So last week, I found myself in a heated debate on the future of aid, with a bunch of NGOs and aid boffins. The topic was ‘is it time for a re-think?’ Why? Because the aid world is changing:

-          New donors, such as foundations, philanthropists and emerging economies such as China and India are starting their own aid programmes, often outside the traditional donor club of the OECD DAC

-          Increasing diversity of sources of ‘financing for development’, from domestic taxation to remittances to private investment

-          Austerity driving many traditional donors to cut aid, either overtly or sneakily, by trying to count lots of non-aid flows as aid, or both (see FT letter here). A reminder that in terms of its increasing aid budget, the UK is really an outlier these days – ‘we are talking in the vicarage, here’.

-          The post-2015 discussions raising lots of questions about sustainable development goals and collective action on everything from climate change to tax havens, which have been traditionally fenced off from the aid discussion.

Underlying all this was a sense that the definition of aid corresponds to an old order (rich northern countries give cash for big push in the South to get public services functioning and the economy humming). That world has little to do with many of the preoccupations of modern development – fragile states and conflict, climate change, leaky financial systems, migration etc etc.

But does that mean aid needs to be overhauled? All were agreed that the current levels of aid, running globally at around $130bn a year, are a precious achievement, the only flow of resources aimed specifically at helping poor people, with a reasonably tight definition, making it easier to defend from dilution. Lots of talk of not throwing babies out with bathwater. (And tanks on lawns, heads in sand – mixed metaphors threatened to get seriously out of control.)

Which brought us to the political context – the march of the Austerians means that any decision to open up discussions on the definition of aid (which governments such as Netherlands and Germany are already doing) is much more likely to lead to a watering down/dilution of aid, with lots of other stuff being included – I pointed out that, in contrast to Pandora’s Box, the nasties will fly in when this one is opened.

Broadly, aid donors will want ‘what allows you to reach your aid target without spending any more money’, while aid recipients will want to keep everything separate, so additional cash for things like climate finance is not counted as aid. One old hand said ‘and the donors will win.’

Which made me line up on the ‘conservative’ side of the table – the risks are largely downside, so try and resist efforts to redefine aid and defend what you’ve got. Others felt that the debate was already happening, and we had no option but to engage.

Everyone was for improved data and transparency (who isn’t?) on non-aid flows, so that donors, governments and others could see what is already happening before allocating their cash (lots of praise for the new DFI/Oxfam Government Spending Watch database of how much poor countries are spending on the MDGs, with seasoned aid officials saying they had spent years trying to get this data out, without success). Another piece of good news is that Development Initiatives are working on an annual report on Investments to End Poverty, which documents all resources available for poverty eradication – watch out for it in September and see some of the material here.

Lots of discussion on the 0.7 target, with the technocrats seeing it as arbitrary and weird, and the advocates seeing its use in driving government action, even in countries that haven’t endorsed it, like the US. Interesting suggestions that 1% of government spending (a penny in the pound) might make a more sensible and communicable target than 0.7% of Gross National Income.

As for the new southern aid donors, the wonks reckoned that they are not interested in targets, but are interested in what counts as aid – one cited Turkey which, when obliged to count it, found it was giving much more aid than it had realised, partly because it had assumed a narrower.

Other interesting discussions on ‘fair shares’ – how you could modify the 0.7 target to take account of a country’s stage of development, perhaps using the UN formula for assessing members’ contributions to its budget. Anyone done that?

Overall, I did feel that there is an institutional problem here – at some point the aid discussion needs to be taken out of the OECD, even though it’s been doing a pretty good job so far. Otherwise, it risks being seen as a project of the declining North, with minimal buy in from others. But would the UN (the obvious alternative) do a better job?

My conclusion? At this political moment, I think there is a real danger in trying to stretch the debate on aid to include everything that contributes to development (we wonks always like to do this – look at post-2015). Right now the test of any proposal should be ‘what is most likely to increase rather than reduce funds going from rich countries to poor countries for good purposes?’ For example, stretching ‘aid’ to include most peacekeeping fails that test badly  -   irrespective of all the good sense about security and development reinforcing each other. Better to try and keep the aid definition (and debates) tight and work on the rest in other fora – Government Spending Watch, tax havens, climate change etc. We won’t win them all – for example there is clearly substantial overlap between climate finance and aid, so insisting on ‘additionality’ is very unlikely to succeed, but I see little benefit in helping others prize open the Pandora’s Box of aid.

Post-2015 wonkwar continued: Claire Melamed on why it’s a Good Thing + your chance to vote

Claire Melamed responds to my ‘bah humbug’ opener on post-2015

I spend most of my working life thinking about post-2015 so this is a slightly nerve-racking experience.  What if Duncan convinces me?  Let me first respond to his arguments, then set out what I think is to be gained from the post-2015 circus… and then we’ll see if I’m still working on post-2015 at the end of it.

I’ll start with the magical thinking.  Yes a lot of what’s being said in the name of post-2015 is a bit ‘if everything was nice everything would be nice’.  But think of it this way: people everywhere, not just wonks like us – are getting involved in serious debates at national, regional and global level, about poverty, about politics, about economics and about the environment.  We don’t know where it will lead yet.  Some of it will lead nowhere.  But don’t write off all that energy and commitment because it’s a bit unfocused, rather celebrate the fact that so many people want to get involved in political debate and action (even be, um, active citizens….).

In any case, that is about the campaign and the public debate, not the goals, and the two shouldn’t be confused.   If the outcome is important, being annoyed at the tone and strategy adopted by campaigners has to be a reason to get in there and change that, not to walk away.

So is it important?  Will a new agreement have any effect? It depends on what.  If it’s a specific change – say a new law on land rights, or criminalisation of gender violence – in a given country you’re after, quite obviously you don’t work on any multilateral process.  You work through national politics, if you’re a local organisation or in solidarity with those local organisations if you’re outside the country. Many organisations and individuals do just that, brilliantly.

But that’s not what we’re trying to do here.  Both Duncan and I, and millions of other people over the years, have also taken part in campaigns, research and advocacy dedicated to improving the global context for those national politics – for example by improving global trade rules or forgiving debt.  This is one of those.  Multilateralism will never be the fastest or most certain route to national change, but it’s a contribution.  Even if the changes are marginal in any given country, taken in lots of countries together that can add up to something quite big.

Of course you can’t know in advance, for any agreement or institution, how those global changes are going to work out in any given situation.  But you have to take a punt on the basis of (almost always partial) evidence, and go for it.

And so to the evidence.  Should, as Duncan suggests, post-2015ers have considered all the available options for multilateral instruments before embarking on this particular course?  Well, the next time someone asks me to design a multilateral system from scratch, then of course I will do that research.  Maybe we can do it together.

But this is not about fantasy multilateralism.  Yes post-2015 is about goals, because that’s what’s on the table.  That’s what governments, in the UN, in regional organisations, in bilateral forums, are negotiating.  Other instruments are available – if it’s laws you want, have another shot at the WTO, or if it’s league tables, there’s always the HDI.  They exist, they have an impact, and plenty of people work on them.  But the political opportunity of post-2015 is about goals, not any of those other instruments.

So why do I (and, by the way, a large number of the world’s governments and the whole UN system, not really the ‘sidelines’) think it’s worth working on goals for post-2015? Apart from the impact on aid, which everyone seems to agree on – there are at least three other reasons to think that the current MDGs have done some good in the world, and therefore why it’s worth investing in a new agreement.

More and better information.  The MDGs, and in particular the indicators linked to each goal and target, created a huge global effort to assess progress on the basis of commonly agreed metrics.  Information has improved in every way since then. The common set of indicators agreed as part of the MDGs allowed us to compare countries to each other and over time.  They created incentives to invest in data, and, probably, reduced the tendency to reach for GDP alone as the all-purpose indicator for human progress.  And more data improved advocacy, policy making , and sometimes led to a race to the top between governments  – all ways that this particular multilateral agreement has an impact at national level.  A new agreement could do this for information on gender violence, or on employment, to take two examples of very important things on which the data is terrible.

More campaigning.  There was campaigning before the MDGs and there would have been campaigning in their absence.  But the combination of goals and targets have been used as an extra bit of ammunition for national campaigns – and again, been one small part of changes in national politics and policy.  Advocates for education and for health services have probably been the keenest users of the MDGs.  It’s rare to read a description of the campaign for free universal primary education in Kenya, for example, that doesn’t mention the education goal as one of the factors that helped push the politics in the right direction (pdf).  Without the MDGs they would have had one fewer stick to beat governments with, and progress may well have been slower.  Campaigners for universal health care, for example, think a goal or target on this would be helpful as they try to push policy in that direction in particular countries.

More and better consensus.  In the 1980s and 1990s growth was king, income was the only thing that mattered, and, according to some of the architects of structural adjustment programmes, it was justifiable to actually make poor people’s lives worse in the short term in the name of ‘development’. The MDGs were the moment that the world agreed that this was not ok, and that social development, as defined in the goals, should be an equal priority for international efforts.  A new agreement can make a move to achieving a similar consensus on inequality, for example, or on the need to make sure that we keep to within environmental limits.  This stuff matters – look at how norms, and then laws and actions, on human rights have changed in the last 20 or 30 years.

A post-2015 agreement is not going to change the world overnight.  Nothing, sadly, will do that.  We can’t know in advance exactly what changes it will bring, and to who, and how.  It may all end horribly and pointlessly, and even if we get a good agreement, it will be a big and unwieldy thing, with an impact that’s felt through many channels over many years.  But within the range of global processes that it’s currently possible to influence, this seems to me to be pretty good investment of my time.  A thousand words later, I’m still convinced.  You?

So over to you for the inevitable poll. As on the results one, I couldn’t think of suitably nuanced revealing questions, so let’s just see if you agree more with Claire, me both or neither. And I think I can assure you, the result will have absolutely no influence over the post-2015 process!

Anyone fancy a post-2015 wonkwar? Me v Claire Melamed on the biggest development circus in town

I’ve been good friends with Claire Melamed for ages, but recently we’ve found ourselves on opposite sides of the post-2015 debate. As ODI’s growth and inequality supremo, Claire is deeply immersed in the ever-proliferating discussions, whereas I decided early on that I had massive reservations about the whole process. So for your amusement (and who knows, perhaps enlightenment), we’ve decided to air our differences in public. I’ll kick off,

Claire responds, and we hope that will produce a load of comments and a life and death struggle for the last word (which I shall of course win, because it’s my blog).

What’s my beef? The post-2015 discussion typifies the kind of ‘magical thinking’ that abounds in aid circles, in which well-intentioned developmentistas debate how the world should be improved. These discussions and the mountains of policy papers, blogs etc that accompany them, are often based on what I call ‘If I ruled the World’ (IRW) thinking. IRW, then I would do X, Y, Z – Rights for (disenfranchised group of your choice)! More Infrastructure! Better Data! Jobs!

The high/low point of this for me came last year, when I had to MC an interaction between 250 civil society lobbyists and the High Level Panel on post-2015 – we managed to squeeze about 80 interventions into the allotted hour of consultation, which produced a Christmas Tree (Claire’s term, much copied) of issues that had no chance of making it onto the final post-2015 agenda.

But in any case, so what if they do? Because what is missing from this is any consideration of power and politics. What, after all, is the point of the post-2015 process, beyond creating (another) international forum for debating development? The MDGs were primarily about improving the quantity and quality of aid, and arguably they were quite successful in this. What is much less certain is the extent to which they influenced government policy (as in, persuaded governments to do things they wouldn’t have done otherwise). Rich country governments have systematically ignored MDG8 (the one on global partnership), while the evidence of ‘traction’ on developing country governments is really rather flimsy (more on that here).

In particular, I was astonished to find that there is no rigorous research comparing the traction exerted on national decision-making by the various different kinds of international instrument (laws, conventions, regional league tables, norms, academic exchanges). So the post-2015 circus is busily debating what ‘should’ happen without first establishing whether/how its conclusions will affect national decision-making. And this blind spot is massive – you can go entire days in the bubble of post-2015 discussions without ever hearing anyone mention any other international instrument on development or rights.

When I raised this at a recent OECD post-2015 conference, Claire wearily replied ‘There isn’t an answer – there is no single thing that we can say ‘if you do it like that, it will have traction’. It is very hard to predict beforehand which mechanisms for any given agreement will get traction.’ So that’s a relief then, can we just ignore these annoying questions about actual impact and get back to decorating the Christmas Tree?

That really isn’t good enough. It is certainly possible to know much more than we do about attribution through more rigorous qualitative research. For example, in-depth interviews with policymakers could investigate the traction exerted by a range of external and domestic forces on their decisions. I have yet to locate such research. (And rocking up and asking developing country ministers leading questions like ‘how have the MDGs affected your decision-making?’ most definitely does not constitute rigorous research.)

So if it can’t generate national traction, what could the post-2015 process achieve?

-       Aid still matters, albeit to a diminishing group of countries, and post-2015 could bolster the case for aid (under siege from the Austerians), and continue to improve its quality

-       Intellectual hegemony matters, so general debates on development are always good (hey, they’re my bread and butter)

-       It may help break the logjam on collective action on everything from climate change to migration (but don’t hold your breath)

But by ignoring the primacy of national politics and avoiding serious political economy questions on traction, it feels like the post-2015 process hasperhaps inadvertently relegated itself to the sidelines – a bit player in a drama that is increasingly national and beyond the reach of the aid industry.

Over to you Claire (and for the sake of my peace of mind, and a natural urge to run away and joint the post-2015 circus, this is one argument I would really like to lose).