Learning the Lessons: Why is change NOT happening in the response to hunger crises?

I know I go on all the time about ‘how change happens’, but often in development the important question is ‘why doesn’t change happen?’, and we need to get better at answering it. On Tuesday Oxfam published Learning the Lessons, an analysis of the response to the 2012 Sahel food crisis, which affected some 18m people across 9 countries. It’s a serious piece of work, drawing on interviews with 30 external bodies – donors, governments etc, other published research, focus group discussions with affected communities and perspectives from civil society.

Compared to the ‘too little, too late’ response to previous crises in the region in 2010 and 2005, the report finds some improvements: early warning systems had improved and raised the alarm earlier, and governments in the region reacted in good time – Niger, for example, appealed for support six months earlier than it did during the 2010 crisis.

But there were still problems with governments, donors and the aid system. Governments in the region still lack the financial and technical capacity to really be able to lead. As for donors:

‘There was still disagreement about the likely severity of the crisis. Some donors, such as the European Community’s Humanitarian Office (ECHO), acted earlier than in previous years, but overall, donor funding was no more timely than before. By the beginning of July 2012 and the peak of the crisis, the UN appeal remained just under 50 per cent funded.’

This really matters: 5.6 million people didn’t get the seeds and tools they needed in time to prepare for the 2012 harvest cycle.

Interestingly, the report sees the roadblock as conceptual, and argues that this can be overcome by changing the way we think about such crises to emphasise the concept of ‘resilience’ – very much the current buzzword in a lot of development circles. ‘Learning the Lessons’ reckons a ‘resilience lens’ would allow donors to:

-          Develop a shared understanding of vulnerability to food insecurity so that support is targeted to the poorest and responses can be launched rapidly;

-          Break down barriers between humanitarian and development actors so that long-term and emergency programmes effectively support each other;

-          Invest in strengthening the capacity of national and local actors so that governments can deliver large-scale, sustained support to their citizens.

I must admit I was initially sceptical that the answer to such a profound failure is a new buzzword. But actually, I think the authors may be on to something.

The most useful framework I’ve found for understanding the roots of inertia (aka ‘why change doesn’t happen’) is the ‘3i’ model of ideas, institutions and interests. A combination of these three underlies the kind of paralysis we’ve seen in the Sahel response.

Institutions: there is still a deep division between the ‘humanitarian/emergency’ and ‘long term development’ wings of the aid business. This is reflected in funding structures, which are completely different for the two silos. The polarization makes it hard to take a long-term approach to reducing the vulnerability to the inevitable future crises.

Interests: if you work in an aid agency, there are clear risks to responding early to a crisis – what if the rains come, you are accused of crying wolf etc? In any case, your political pay masters often only start banging the table when the grim TV images start to roll (by which time it is often too late, and certainly much more expensive, to respond). There is also still something of a macho ‘I’m here to save lives, get out of my way’ approach to humanitarian work which can all too easily brush aside national governments and local knowledge that are crucial to understanding the long-term roots of crises, and building institutions to deal with future ones. National governments need to be at the heart of efforts to address food insecurity, but that is likely to threaten the power relations of the status quo,

Ideas: The institutional silos reflect a crippling conceptual dichotomy. Cyclical crises such as those affecting the Sahel really can’t be described as ‘emergencies’, in that they are predictable and regular. But the underlying thinking in the aid business is still ‘is that an emergency, or is it long term development? Do we send in the engineers or the economists?’

One of the things I’ve noticed about climate change is that it is a ‘disruptive idea’. Disruptive ideas can’t be fitted into existing entrenched mental and organizational frameworks, and so often prompt violent rejection, but also the possibility of paradigm shifts. Because climate change doesn’t ‘belong’ to any existing camp, it makes it easier to bring people together (development and environment, for example) to think differently about how we respond to it without prompting accusations of turf wars and interference. Is ‘resilience’ also a disruptive idea, with the potential to bypass the humanitarian/development divide?

I’m sure there is a big literature our there about the characteristics and impact of disruptive ideas. Any links appreciated.

Theory’s fine, but what about practice? Oxfam’s MEL chief on the evidence agenda

Two Oxfam responses to the evidence debate. First Jennie Richmond, (right) our results czarina (aka Head of Programme Performance and Accountability) wonders what it all means in for the daily grind of NGO MEL (monitoring, evaluation and learning). Tomorrow I attempt to wrap up.

The results wonkwar of last week was compelling intellectual ping-pong. The bloggers were heavy-hitters and the quality of the comments provided lots of food for thought. However, I was left wondering what it all meant for those of us who work in NGOs, trying to generate and learn from ‘evidence’ on a daily basis. I found myself unable to simply vote, so instead I blog….

The results and evidence agendas have brought some real benefits to NGOs in my view. First and foremost, it is important and right that those of us who claim to work in the interests of the poorest people in the world and are stewards of other people’s money, should set ourselves high standards for our own impact. In its simplest form the results agenda asks us to justify the trust others have placed in us, by demonstrating whether we are actually bringing about positive change. In Oxfam GB, accountability has long been held as a core organisational value. It is not the results and agenda that has got us thinking about how to capture and communicate our effectiveness, but it has provided a helpful additional push.

A further positive is that space has been created both within our own organisations and in the wider sector, to stop, listen and learn. MEL-istas (as Duncan calls us) 5 years ago struggled to get the ear of senior managers (let alone Ministers). But the results agenda has increased the stakes around MEL – encouraging organisations not only to increase investment, but also to listen to the findings coming from our own data gathering and analysis.

However, it has also increased the demand and the expectation, which are not easily met by all NGOs. In Oxfam GB the investment in MEL has increased over the last couple of years, undoubtedly, but still it is a real stretch to deliver the ever-more ambitious demands from donors, to develop tools to tell the story of our broader organisational impact, and to ensure that we are developing innovative ways of measuring cutting-edge programming areas, such as resilience, enterprise development and influencing.

And we are one of the largest international development NGOs in the UK. How much more difficult for the smaller and niche NGOs, or those who lack the flexible financing that permits investment in MEL and innovation? We are conscious in Oxfam that we and other large NGOs need to guard against distorting the NGO market place by pushing the boundaries on MEL and impact too far, and thereby creating expectations that cannot be met by everyone. Somehow we all need to keep our sights on a proportionate approach.

It is not just important to generate evidence, but also to use it properly. There is increased demand for serious, evidence-based conversations about what works.  None of us can get away with decisions made purely on gut instinct, force of habit or ideological leaning. We are challenged by the ‘evidence’ question to collate and distil from the broad knowledge base we have at our disposal. And this has in some cases led to surprises. Rigorous studies, whether based on qualitative or quantitative methods, can challenge our preconceptions – showing us impact where we were not optimistic, or the opposite. The test, of course, comes when new programmes are designed. Will the body of evidence be applied – will we be able to find it for starters (in our often not-so-state-of-the-art knowledge management systems), and will it be politically acceptable in our own organisations to apply it to practice?

So, how can we use the results and evidence agendas and make them useful to us as NGOs?  We need to do this in a way that a) is true to the actual work we do (which in the case of Oxfam includes a great deal of work that drives for political change and influencing) and b) does not distort decision-making away from the right decisions (i.e. what most suits the specific needs and opportunities of each context) in our efforts to be able to measure and communicate what we are doing.

One of the concerns raised in last week’s blog was that in some institutions, evidence becomes synonymous with impact evaluations, and even specifically with Randomised Control Trials. As all the bloggers agreed, the default use of one research method for interventions of all types is simply nonsensical. You only have to look at the enormous variety of the things we do in international development (from campaigning for policy change to delivery of bed-nets, from building of bridges to raising awareness of the rights of citizens) to realise that one approach is just not going to cut it.

Another challenge is that so much of what we do in international development is extremely hard to measure. How can we trace the input through to impact chain and clearly demonstrate the ‘on the ground’ changes we have brought about in people’s lives when the investment is in budget support or core funding?  How can we reduce the process of a community standing up against acts of violence against women to a Value for Money calculation? The ethical dilemmas and practical difficulties wrapped up in measuring and ‘evidencing’ many of the processes we are involved in are huge. And, as Eyben and Roche point out, much of what we engage with in international development is messy and political. We need to make sure that the tools we have at our disposal for evidence generation are sophisticated and nuanced enough to acknowledge this messy political reality, and that we are sharing ideas on how to do this in a practical and affordable way.

The push for evidence should go hand in hand with a more entrepreneurial approach to development, opening up space for honest

MEL that - US military mindmap of Afghanistan

reflection on both success and failure. That is the theory. But, of course, there are obstacles to this becoming a reality. Our systems in large institutions, including NGOs, are designed to demonstrate success. We all have our logframes and our KPIs, and we want to be able to put a tick in the box. No-one wants their project to be the one famous for not achieving what it set out to do, even if the real story is that it helped enormously to generate learning for future projects. Complexity thinking is having some influence right now, which helps to raise the right questions about process and incentives. However, we have a long way to go before even in the most reflexive learners in NGOs and other development institutions want their project to be hailed as the great failure.

So, we proceed with caution – welcoming the increased space the Results Agenda provides to consider ‘what seems to work’, and the profile it gives to the need to take a thorough and transparent look at the information coming out of our programmes. But, wary of the dangers of distorting what we do in order to make it measurable; of placing the MEL ‘bar’ for NGOs too high to reach; of the over-emphasis of certain methodologies; and of the danger of ignoring political realities in the work that we do. It is certainly helpful to keep reflecting and questioning, however, from all sides of the debate – so the wonkwar of last week was welcome.

Launch of ‘If’ – new megacampaign to tackle global hunger: how does it compare with ‘Make Poverty History’?

Sorry for a second post in one day, but the launch of If is a biggie

Ah the perils of age – am I becoming one of those annoying old guys who greets every new idea (however excellent) with a weary sigh and ‘we already did/discussed all that back in the 19XXs’? I ask because I have a distinct sense of ‘here we go again’ as today, a smorgasbord of 100 NGO logos will adorn the press releases for the launch of ‘If’, a big campaign to tackle global hunger. Logotastic, lots of killer facts, a smart video (below) and, wait for it, white wristbands! Yep, it feels a bit like a rerun of Make Poverty History (2005, for the younger readers). I may blog about this properly when I’ve had time to gauge the debates around the launch, but initial impressions are:

What’s the same as MPH?

Northern focus, pegged to this year’s UK presidency of the G8 (although the G8 is not the global steering committee it was (or at least thought it was) back in 2005).

The wristbands and celebs, which should take development debates outside the usual circuits (a good thing, in case more wonky readers are in any doubt).

The big coalition of NGOs managing the tensions of any alliance in terms of pushing their particular priorities while maintaining a clear enough message to get media ‘cut-through’. More subtly, they also have to balance the dangers of over-hyping impact, ‘make poverty history’ style, with the risks of disappearing into an academically rigorous but entirely incommunicable message of ‘hey everything is context-specific, and there are enormous limits to the efficacy of international action, but we think this would probably help a bit.’

The focus on aid – this is a big year, with UK government becoming the first G8 country to meet the international aid target of 0.7% of national income, even as other governments are tearing up their aid promises under the weight of economic crisis.

What’s different

We didn’t say ‘cut through’ back in the day.

Many more technological options for viral campaigning – twitter (#If) being the most obvious. Linked to that is a much greater focus on transparency (helpfully, if clunkily, translated as ‘seeing clearly’ in the campaign literature). And a seriously funky website (left).

If reflects the shifting development agenda: in come tax dodging, biofuels, agriculture and nutrition, out go trade (Doha round going nowhere) and debt (successful cancellation in dozens of countries). More of a focus on the rich countries putting their houses in order (tax, biofuels etc), which has to be a good thing (its lack was one of the main critiques of MPH by Dani Rodrik and Nancy Birdsall, among others). Climate change is one of If’s core issues, whereas in Gleneagles, it was put on the table by the British government, not MPH.

This one feels more UK-centric (at least for now).

No sign of Bob Geldof so far (but the year is young….)

So what do you think?

One other consequence of age: for my generation ‘If…..’ conjures up images of the 1968 film, which ends with a young Malcolm McDowell on a rooftop machine-gunning the parents and teachers of his posh public school (as we call private schools in the UK). It even has a memorable reference to Oxfam. Trust that’s just a coincidence.

Should men speak on all-male panels? Summary + time to cast your vote

Right, I have now waded through dozens of comments, tweets and my own tangled thoughts on Monday’s post. What stood out?

Boycott v constructive engagement: is it better to politely push conference organizers, suggest female panelists, and express ‘strong disinclination’ to take part in testosterone-fests, or to play hardball with a blanket ban? And is the crime less heinous for a three man panel than a six man one?

Should this approach be extended beyond gender, especially to having representation from developing countries?

Pressure during the event itself: questions from the floor and from panelists should ask organizers to explain themselves and/or panelists should make the effort to ask for female colleagues’ input to the debate and pass it on (duly credited).

Should we add a ban on all-female panels on gender issues?

Don’t blame the event organizers when the real problem is broader – the lack of women at top level in a number of development-related institutions (yes, but a combination of conscious effort and affirmative action by event organizers can be part of redressing the wider problem).

For the moment, I’m coming round to the following position: When asked to participate on a panel, right-thinking men  should

a)      Ask about the current make-up of the panel

b)      If it’s devoid of either women or people from other relevant population groups (depending on the topic), both express serious reservations and try and suggest some names

c)       If they think the organizers are not serious, they should decline, but if they seem to be really trying, it’s OK to say yes

d)      Before the panel, try and get input from colleagues to fill in any potential gaps in the panel‘s analysis due to its grotesquely distorted composition

e)      When speaking on the panel, mention the disparity, and try and ensure a fair spread of questioners (male domination applies to questions at least as much as it does to panelists)

And of course, none of this applies if the panelists are Barack Obama, Nelson Mandela and Jim Kim, but Christine Lagarde is not available that day.

There, that should ensure I never get invited to speak at a panel again. Result.

And since everyone likes voting (judging by last year’s top ten), please could you tell me whether I’ve got it right? The Poll will remain open for a few days. The question is

When asked to appear on panels, men should

a)      Simply refuse to appear on male-only panels

b)      Constructively engage with organizers in the way set out in the blog

c)       Just be grateful, say yes and abandon all this pitiful liberal self-doubt

Forget swimming pools and bra hunts, it’s time for the Great Intern Debate

It’s been a while. The issues that get a buzz going on this blog are often the internal-to-Oxfam debates, which  get Oxfamistas worked up while seeming to provoke a prurient curiosity in everyone else. Think swimming pools or bra hunts. There had been a bit of a lull on this front until I wandered unintentionally into another minefield – unpaid internships – during a recent post on how to get a job in development. Cue blizzard of comments, tweets, denunciations etc etc.

So I’ve read the comments, and Public World’s excellent briefing, and talked to those responsible in Oxfam, and here’s my take on the debate.

First, allow me a bit of self-defence. The post was intended as advice to young people trying to enter the development world as it currently

is, not as it ought to be. I realize that some people may take that as an implicit endorsement of the status quo (which it isn’t), but I still refuse to advise them to boycott the intern system and so reduce their chances of eventual success.

Now, back to the interns debate. The arguments against unpaid internships are several:

• They’re unfair, because (despite a few exceptions) they skew career progression towards those best able to work for free (middle-class kids, or people able to rely on spouses), and so introduce a class bias from the outset.
• They impose yet more financial burdens on students already staggering under their university debts
• By not putting a financial value on interns, they encourage sloppy or abusive management (interns making coffee, doing the photocopying etc). Abuses seem particularly bad in fashion and media – Oxfam has pretty strict guidelines to prevent this kind of thing and ensure that interns do actually get useful CV content out of their time – take a look, they’re pretty good.
• More systemically, they normalize the use of unpaid labour, and so undermine labour rights across the workforce.

But there are several arguments for unpaid internships:

• They’re not actually unpaid, in the broader sense that interns derive non-pecuniary benefits like skills and experience (which is, after all, why they do them)
• They provide a great way for employers to spot strong candidates for paid jobs (indeed, they’re a much better assessment than standard job interviews – maybe all job recruitments should include a week’s work for the shortlisted candidates, to test whether what they say in the interview is true?)
• If internships are paid, there will be fewer of them – I don’t buy the crude ‘NGOs can afford to pay’ – there is obviously some kind of elasticity of employment with respect to paying wages, the question is how great it is.

From the excellent comments, several additional issues emerged:

The role of government: Opinions differ on whether it is now harder or easier to claim benefits while interning. A lot of the commenters suggest the former, but our interns guru, Georgia Boon (herself a former Oxfam shop volunteer) thinks that for the bulk of charity volunteers, ‘claiming benefits became pretty much unrestricted under the Labour Government and hasn’t changed back’. Oxfam’s Intern Agreement includes guidance on how to claim benefits while interning, and its internship scheme tries to reflect that, for example by placing a ceiling on intern hours. Another key concern for interns is the cost of housing, and here there were some great suggestions for how we could make it easier to find cheap accommodation for interns, for example in unused university rooms over the summer.

Career ladders: I think there’s a much wider problem with NGOs and career ladders. Getting onto the first rung is hard enough, but then the path up the career ladder is also really difficult – overall, I think NGOs don’t invest as much as companies and governments do in areas such as graduate entry, fast track career progression etc, so perhaps it’s not surprising that we find real difficulty recruiting for more senior positions.

Economics v Mission: Various comments pointed out that diverting £8,000 per year into paying a living wage (£8 per hour for 20 hours a week) would mean £8,000 less spent elsewhere in Oxfam’s work. That has to be true (unless we could raise extra money to pay for interns – anyone want to buy their mum an intern for Christmas rather than a goat?). But there are wider considerations about what social justice and what NGOs are doing to ‘be the change you want to see in the world’. Or if you want to put that into management speak – the reputational risk of perceived double standards. We face the same cost v reputation balancing act on carbon emissions or sourcing fair-trade products, for example.

Legal concerns: There seems to be a lot of confusion over whether not paying interns will eventually be challenged under minimum wage legislation, or conversely, whether paying them a minimum wage will confer full employment status (and so effectively abolish internships). The key seems to be ‘if it looks and feels like work’, with regular hours, having to request time off, formal appraisals etc, then at some point a judge is going to say that it is work, with all that implies.

Complicating all this is the blurred boundary between interns and volunteers. Oxfam has thousands of great volunteers running our shops, campaigning etc, who donate their time in support of our work. Interns are doing something else, usually younger, and giving their time in exchange for some hope of career development. Given the criticism heaped on internships of late, we have considered changing the name of the scheme to something like ‘project volunteers’, but the interns objected – they see internship as a useful addition to their CVs.

The upshot of all this is that I am genuinely torn. I know from watching my kids’ generation emerge from university just how hard a struggle it is to find a job, and to survive as an intern. But I don’t completely buy the idea that paying a minimum wage or slightly higher would transform the class composition of entry level NGO staff – the barriers to diversity are more complex and pervasive than that (when Oxfam did briefly try a graduate entry scheme a decade or so ago, the applicants were less diverse than our normal volunteer intake). And I do think that paying interns would reduce the number of internships (though I have no idea by how much). So in time honoured FP2P fashion, I will stay firmly on the fence and let you, the readers, decide (see poll, right) albeit in a non-binding sort of way. But here’s the question I want you to vote on – please read it carefully before voting:

“In pursuit of fairness and a diverse workforce, NGOs should pay a living wage to interns, even if that means fewer internships are available, and some funds are diverted from other uses.”

Over to you

and thanks to those who responded to my twitter request to send in cartoons. Winning entry so far goes to Makarand for this Dilbert classic (but keep sending in your favourites):

How do we work out the returns to campaigning? Nice example from the Philippines

Like any campaigning organization, Oxfam has limited funds, and so needs to know whether its investment has paid off. The push from everyone and their dog to pursue a ‘results agenda’ and ‘value for money’ has added further momentum to that effort. That’s fine if you’re doing something that’s easy to measure, (say vaccinating kids, or cash transfers), and where attributing an effect to a particular cause is relatively straightforward, even if sometimes technical and expensive to establish. But what about influencing government policy, where there are dozens of voices, numerous events, and establishing any causal chain is both elusive and (inevitably) disputed (did anyone else grind their teeth watching Bono and Bob making poverty history the other night………?)

This matters because Oxfam increasingly sees a big part of its role as working with others to influence government policy, especially in developing countries, through programmes, partnerships and advocacy.

I got involved in a brain-bending conversation about this when trying to help out with a ‘killer fact’ on some smart campaigning by our team in the Philippines. At first glance, the success of the campaign for a ‘People’s Survival Fund’ was ideally suited to the task. Oxfam and partner iCSC (Institute for Climate and Sustainable Cities) commissioned research, and then launched a campaign in July 2010 calling on the government to set up a climate change adaptation fund. We did all the usual stuff – backgrounders for policy makers, popular mobilization, media work, celeb endorsements etc and (voila!) a US $25m a year People’s Survival Fund (PSF) was passed by the Philippine Congress in June 2012 after a two-year campaign. Result!

But was it value for money? At first glance it seems pretty easy to calculate the return on the money invested in the campaign – it’s just how much cash reaches poor people over a period of time, compared to the amount Oxfam spent on the campaign, corrected to take into account the fact that Oxfam wasn’t the only organization campaigning on the issue, and so shouldn’t take all the credit.

In mathematical terms, it’s even easier: Return to Campaign (RtC) = (AxBxC/D)

Where

A = The total new expenditure on climate change adaptation resulting from the PSF.

B= the proportion of that money that reaches poor people.

C = plausible % of attribution to the Oxfam campaign

D = Oxfam’s expenditure

We calculate the value for A, B, C and D as follows

A: P$1bn a year, taken over say a five year period, making it P$5bn (about US$125m).

B: If the money is equally distributed among all the people in the areas receiving PSF funds, some 45% would go to poor people (based on the 30-60% poverty rates in the relevant areas). But experience suggests that richer people may be more likely to get their hands on the cash. As we are looking for a conservative estimate here, we therefore assume that only 20% of the money would go to poor people

C: As the main funder, and lead agency in the lobby effort that led to PSF, it seems reasonable to take half the credit for the victory, so D = 0.5

D: Oxfam’s total expenditure over the three years of the campaign comes to P$7.4m

So using $Pm as the unit of calculation

Return to Campaign = (5000 x 0.2 x 0.5)/7.4 = 68

i.e. over a 5 year period, Oxfam’s campaign generated at least 68 times more resources for climate change adaptation than we invested in the campaign/for every $1 we spent we generated $60 for climate change adaptation for poor people.

Enter the nagging self doubt (otherwise known as Claire Hutchings in our monitoring and evaluation team). Every single one of those terms can be challenged:

A: assumes all the budget is disbursed and that none gets eaten up by overheads – any underspend or overhead costs would obviously reduce the amount available to reach poor people.

B: how do we know if that is a reasonable estimate of the proportion of the PSF that will ultimately reach poor people?

D: but what about all the other money Oxfam has spent globally and within the Philippines on raising awareness of climate change, supporting partners etc – didn’t that play a role in the victory?  What about cost of programming we’ve done in the Phillipines and other countries that have contributed to building the Oxfam brand, enabling us to ‘sit at the table’, participate in these conversations, influence etc.

And then we get to C: let’s assume for a moment that we can get an accurate costing of all the resources Oxfam has spent national and globally that have contributed to getting this issue on the agenda in the Philippines, and can reach a credible estimate of the proportion of PSF that will reach poor people.  The question remains how can we credibly attribute a % of any decision to the influence of the campaign?

For example, suppose years of global and national campaigns, by Oxfam and others, had got the issue to a tipping point, where only a small nudge was needed to persuade the government. Should the credit go to the patient slog of a multitude of actors, or the last minute glory-grabbing campaign (back to Bono and Bob)?  A light touch approach might be to ask people – staff, partners, government officials and perhaps most importantly, independent experts – to give us an estimate. But such questions risk being pretty leading (‘please attribute a percentage of attribution to the campaign’ is likely to get an inflated estimate), and open to bias. But doing something more rigorous, to investigate the main factors that contributed to the Parliament’s decision, would be expensive and still may not find the evidence needed to reach credible conclusions.  Now there’s a whole measurement challenge around evaluating campaigns and advocacy efforts, and through our Effectiveness Reviews we’re investing in trialing and refining an impact assessment approach for this work, one that builds from process tracing, to explore what it takes to reach credible conclusions about the contributions of our work to policy change (watch this space).

Let’s assume (for the moment) that such evaluations would allow us to credibly attribute our influence.  The fact is that these evaluations take time and resources.  Do we really need to commission an evaluation any time we want to talk about the resources that are being leveraged through our campaign work?  Or can we identify a rule of thumb, with all the necessary caveats and qualifications, that’s ‘good enough’, at least for cases that seem pretty clear cut.

What would be good enough in this case? Your thoughts please

How to get a job in development – an FP2P guide

There’s nothing like a lecture tour to bring home just how many bright young people are desperate to work in development, and how hard we make it for them (is this a deliberate form of institutional Darwinism, in which only the most determined survive?)  So I’ve gone back over a few previous bits of advice from me and others, to produce this revamped FP2P guide to throwing your life away getting a job in development.

I won’t give advice on what to study – if you’re reading this, it’s probably too late anyway. But in any case, qualifications are not enough – you need to get involved in organising relevant activities at your university, depending on your interests (e.g. Engineers Without Borders, Amnesty International, assorted International Development committees). You’ll learn a lot, make great contacts and friends, and develop skills that NGOs prize (organisational abilities, putting on events, writing, debating etc).

Then, decide what kind of work you are interested in. Research? Programme work on the ground? Emergencies (conflict refugees, disaster reconstruction etc)? Advocacy and lobbying? Public campaigning?

Next think what kinds of experience will help – experience often marks you out more than gaining another post graduate qualification, but you have to find some way to get over the inevitable first-rung problem of ‘how can I get experience when I haven’t got enough experience to land a job’ – it’s not easy, but it can be done.

For emergencies and programme work, try and get out there and get some experience in developing countries – it’s very hard to arrange that from this end, unless you have a particular network (eg a Church or university connection) that you can call on, so many people just try and sort something out on the spot. For campaigners, a record of activism at university or afterwards is always helpful.

For advocacy work, NGOs are often impressed by people who have worked in other sectors, especially the institutions we are keen to influence – governments north or south, multinational companies, aid donors. Many of them are much larger than Oxfam, and have good graduate entry schemes – a further advantage if you’re trying to get your foot on the ladder. Many are highly competitive, but check out the schemes for DFID, the World Bank, or the Overseas Development Institute. And if successful, make sure you get out before you get too comfortable, as NGOs are likely to pay less!

Show your face. Putting in a spell as an intern may not help with your student debt, but it enables you to make your mark and prove your commitment. It also enables you to apply for jobs that are only advertised internally, including short term jobs (maternity cover etc) that help you get on the paid employment ladder. But be choosy who you intern for, and what jobs you accept – even if there is no pay involved, you are offering skills and time to an organization, and should demand things in return.

And remember that research, advocacy and campaigning jobs are often the most sought after and competitive. It may be advisable to try to get a foothold by applying for more ‘corporate’ areas such as marketing, HR and finance, and then start from there.

Once you get an interview, follow all those useful guides to how to prep etc, but also, don’t hide your passion (even if you’re English). I’ve given jobs to interviewees because they were more passionate about development than the other candidates, (and never regretted it).

And here are some thoughts from a couple of other development bloggers

Alanna Shaikh

1. Get an office job while you’re still in school. Most development work is office work. You need to prove you can handle an office every day. Really, the only way to do that is to have an office job. Do it in the summers if you can’t hack it while in school. Office work is not the most profitable way to spend your time, but it will be worth it later.

2. Study something useful at university. For example, technical subjects like nursing and IT are useful. Epidemiology is useful. A master’s degree is more useful than an undergrad degree.

3. Learn to write. I don’t mean you need to be a novelist, but with practice everybody can write a clear, useful report at decent speed. Have writing samples to prove you can do it.

4. Study a second language. You don’t have to get all that good at it, but making the effort demonstrates you are willing to commit yourself to international and intercultural work. If you are already bilingual, you don’t have to learn a third language. People will assume you are good at intercultural navigation.

5. I think this is the hardest one: Have a goal for what you want to do, that’s specific but not too specific. “I am interested in food security and emergency relief” has a good level of specificity. “I want to work for UNDP” is too specific. “I am interested in women’s empowerment, reproductive health, and community development” is too vague. There is kind of an art to this; basically you want to give people a sense of who you are and what you want. Too broad and they don’t have any sense of you. Too narrow and you’ve ruled out too many jobs. If you’re having trouble with this, it’s a good thing to talk over with a mentor.

To which Chris Blattman adds

6. Be prepared to volunteer your first couple jobs

7. Pound the less-trodden pavement (e.g. try contacting program managers, country offices, etc. directly rather than applying through the front door)

8. Consider a private firm

9. It’s a numbers game (so understand that 50 emails will yield 45 non-responses, 3 immediate rejections, 2 interviews – and one job)

10. Be willing to go to uncomfortable places.

Alanna and Chris are both included in the exhaustive and excellent set of posts and links on Whydev.org

And once you get the elusive job, you can of course get your disillusionment in early by signing up to the excellent Stuff Expat Aidworkers Like blog. With that, good luck to all. Anyone know of any books I can recommend on this?

India’s Middle Class debate continued: should NGOs be looking in the mirror? Guest post from Bipasha Majumder

On my recent trip to India, I discovered some talented bloggers – here’s Bipasha Majumder, Oxfam India’s Communications Officer in Mumbai, writing in a purely personal capacity on the Great Middle Class Debate. She also writes a personal blog.

I have had discussions and I have had heated discussions.  Sometimes I have just let the question float in the air, sat back and observed what others had to say.

Whichever way you look at it, one thing is very clear. The great Indian rising middle class is just not bothered. They are largely happy and keen to contribute to the ‘growing’ economy. But when it comes to any kind of contribution to a cause, especially those related to poverty, there is a big wall of apathy around them.

As a friend puts it – there is a bubble around them…. a bubble of ipad, iphones, AC rooms and cars and what they have achieved in life on various levels. It’s not that they are ignorant of the problems afflicting the country. It’s just that it doesn’t affect them directly. If it doesn’t affect them, then there is no point bothering about it. It’s a problem which is out there and for the government to handle.  If prodded on charity, this group will probably turn around and say, ‘I can only donate when my needs have been taken care of – I still have  bank loans to pay, a second car to buy and a trip abroad to book.’

It’s not that the entire middle class is like this. There is a growing section of people who want to give back, to do something for somebody. And they do, mostly in the form of sponsoring the education of girl children or giving out books and old clothes or giving time by teaching kids in slums or orphanages. A few believe in collecting funds and giving it directly to individuals who need help, medical attention etc. Some would go and donate old computers to schools in villages. This set of people will willingly help as long as they know its directly benefiting somebody’s life.….a change that they can see and feel good about.

But this group is also very skeptical about donating to any NGO (not just Oxfam). With so many scams around politically-connected NGOs surfacing and lack of transparency in most, they don’t want their hard earned money to go into somebody else’s pockets. There is a joke that sums up this cynicism – if you want to own land, build a temple and if you want to earn money, open an NGO. During the rise in the global concern on HIV-AIDS, many such dubious NGOs surfaced, collected hefty foreign funds and then disappeared without a trace.

One can keep pointing fingers at the apathy of the middle class, but the situation within the social sector is not that rosy either. Apart from the fact that many NGOs (some known ones also) do have dubious financial histories, many others are mostly a one-man show working on the whims and fancies of this individual. (For example – a leading Indian NGO on education (better known internationally than within India) has no long term strategy or clear monitoring and evaluation system. If one fine day, the founder decides to work on improving the quality of teachers through training, then the whole system  starts working only around that. Yet, this organisation has won many awards). Quite a lot of others operate without any clear strategy or goals and work only in silos. Despite rising questions from the rest of the society, most of these NGOs are not ready to change or engage with the public (read middle class).…. which only feeds into the general distrust.

I recently met a 60 years plus Sarpanch (Head of the village local governance system or Panchayat) of Lata village in Uttarakhand (a newly created Himalayan state). This not so educated person told me, “Everybody in this country wants the others to change. We all want to earn quick money, we all tell lies and yet we look at others and say ‘they shouldn’t do it’. If you want any kind of change, you have to change yourself first.”

India’s middle class might be apathetic now, but they have a huge potential to change. But for them to change, the social sector needs to change first. A few scenes from my own experience:

• An NGO working on education cannot have volunteer teachers (or para-teachers) who spell BLACK as BLECK and teaches kids that only one Emperor penguin lays egg while the rest huddle to give warmth to that egg.
• A wildlife conservation NGO cannot have staff who throw plastic on the road or forests carelessly while stopping villagers (who they work with) from doing the same.
• An NGO working on public health cannot be successful if they do not teach people how to manage waste.

Even though I am now a part of this sector and hence reluctant to pass hasty judgments on NGOs, I find it difficult to trust organizations which clearly miss out on the basics. Dealing with the over critical middle class means looking in the mirror, as well as changing middle class attitudes to philanthropy.

Can you help promote ‘From Poverty to Power’? This won’t take long…..

OK, out of consideration for your sensitivities, I’m going to try and condense all the humiliating, grovelling self promotional authorial thing into a single post (OK, I’m lying, but the other promo will be less blatant). The second edition of From Poverty to Power is published on the 23^rd October, and as you doubtless know, there is nothing so craven as a writer desperate to promote their book. So let’s get it over with:

Presentations and lectures: Have Powerpoint, will travel. If you’re in the UK, I’m happy to add your organization to the launch roadshow, provided you can guarantee a reasonable turnout. Please contact chingley[at]Oxfam.org.uk to discuss dates etc. If you’re outside the UK, I’m still interested, but it may be more complicated (and expensive) – but please get in touch. The current list of launch events is here.

You can also follow event news on facebook and twitter

Buying it: You can order individual copies, or if you want to get a bunch of them, contact andrea.palmer[at]practicalaction.org.uk to discuss bulk discounts.

Reviewing it: Andrea is also the go-to woman for review copies

Translations: We’re keen to see translations, either of the whole thing, or in an abridged version. The first edition is already in Spanish, Italian, Portuguese and Korean, so you can cut translation costs in those languages (we can provide track changes versions). Contact rcornford[at]Oxfam.org.uk.

Blogs: I am generally keen to repost stuff as a guest on other blogs. My ideal model is the World Bank’s People, Deliberation, Spaces site, which regularly reposts, and is really easy to work with (no demand to post before/at same time as me etc). Anyone else out there interested? You don’t have to ask permission to repost, but if you let me know, I can alert you when suitable subject matter is on the way.

Free access: If you want a taster of the new edition we’ve opened up the ‘Food and Financial Crises‘ chapter ahead of publication to clelebrate World Food Day and Oxfam’s GROW week – Look inside now

Finally, here’s a promotional video, featuring me reading an autocue for the first time, which induces a strange zombie-like party political broadcast tone. Felt like having a lobotomy.

Phew, glad that’s done.

When we (rigorously) measure effectiveness, what do we find? Initial results from an Oxfam experiment.

Guest post from ace evaluator Dr Karl Hughes (right, in the field. Literally.)

Just over a year ago now, I wrote a blog featured on FP2P – Can we demonstrate effectiveness without bankrupting our NGO and/or becoming a randomista? – about Oxfam’s attempt to up its game in understanding and demonstrating its effectiveness.  Here, I outlined our ambitious plan of ‘randomly selecting and then evaluating, using relatively rigorous methods by NGO standards, 40-ish mature interventions in various thematic areas’.  We have dubbed these ‘effectiveness reviews’.  Given that most NGOs are currently grappling with how to credibly demonstrate their effectiveness, our ‘global experiment’ has grabbed the attention of some eminent bloggers (see William Savedoff’s post for a recent example).  Now I’m back with an update.

The first thing to say is that the effectiveness reviews are now up on the web.  Here you will find introductory material, a summary of the results for 2011/12, and some glossy (and hopefully easy to read) two-page summaries of each effectiveness review, as well the full reports. (You may not want to download and print off the full technical reports for the quantitative effectiveness reviews unless you know what a p-value is. With the statistically challenged in mind, we have kindly created summary reports for these reviews, complete with traffic lights….).  Eventually, all the effectiveness reviews we carry out/commission will be available from this site, unless there are good reasons why they cannot be publicly shared, e.g. security issues.

Plug over, I can now give you the inside scoop.  In the first year (2011/12) we aimed to do 30 effectiveness reviews, and we managed to pull off 26. Not bad, but our experience in the first year made us realise that our post-first-year target of 40-ish reviews per year was perhaps a bit overly ambitious.  We have now scaled down our ambitions to 30-ish, to both avoid overburdening the organisation and enable better quality control.

The issue of quality control, in particular, is critical because there are certainly opportunities to strengthen the effectiveness reviews, particularly in terms of rigour.  Currently, there is considerable interest in how to evaluate the impact of interventions that don’t lend themselves to statistical approaches, such as those that are seeking to bring about policy change (aka “small n” interventions).  See a recent paper by Howard White and Daniel Phillips.  We have attempted to address this by developing an evaluation protocol based on a methodology called process tracing used by some case study researchers.  However, we are struggling to ensure consistent application of this protocol.  Time and budgetary constraints, as well as inaccessibility of certain data sources, are – no doubt – key militating factors.  Nevertheless, we aim to improve things this year by more tightly overseeing the researchers’ work, coupled with the provision of more detailed guidelines and templates so they better understand what is expected.

While in no way perfect, we have perhaps had more success with the reviews of our “large-n” interventions, i.e. those targeting large numbers of people.  This is, at least in part, because we are directly involved in setting up the data collection exercises, and we carry out the data analysis in-house.  The key to their success is capturing quality data on plausible comparison populations and key factors that influence programme participation, and this has worked out better in some cases than in others.  We are also attempting to measure things that just aren’t easy to measure, e.g. women’s empowerment and ‘resilience’.  We are modifying our approaches and seeking to collaborate with academia to get better at this.  Despite their shortfalls, at £10,000-ish a pop (excluding staff time), we believe these exercises deliver pretty good value for money.

Humanitarian programming is not my thing, but I am particularly pleased with the humanitarian effectiveness reviews that critically look at adherence to recognised quality standards.  While there are some methodological tweaks needed here and there, the cohort of reviews presents an impartial and critical assessment of Oxfam’s performance and identifies key areas that need to be strengthened, e.g. gender mainstreaming.

So what do the effectiveness reviews reveal about Oxfam’s effectiveness?  While the sample of projects is too small to draw any firm conclusions, the results for this particular cohort of projects are – as one might expect – mixed. For most projects, there is evidence of impact for some measures but none for others.

There are, no question, some clear success stories, such as a disaster risk reduction (DRR) project in Pakistan’s Punjab Province.  Here, the intervention group reported receiving, on average, about 48 hours of advanced warning of the devastating floods that hit Pakistan in the late summer of 2010, as compared with only 24 hours for the comparison group.  Having had more time to prepare is one possible explanation why the intervention households reported losing significantly less livestock and other productive assets.  Oxfam’s research team is in the process of commissioning some qualitative research to drill down on this project to better understand what made it work.

Given Oxfam’s size and capacity to mobilise and make noise, it is no surprise that there is reasonably reliable evidence that many of the campaign projects have brought about at least some positive and meaningful changes, despite falling short of fully realising their lofty aims.  However, the results for several of the sampled livelihoods and adaptation and risk reduction projects are, quite frankly, disappointing.  Figuring out why these particular projects have not worked is just as critical for learning as is figuring why the Pakistan one did.

Whether their findings are positive or negative, I have to admit that I am impressed with how seriously the effectiveness reviews are being taken by senior management.  A management response system has been set up and embedded into the management line, where country teams formally commit themselves to taking action on the results.

That being said, the effectiveness reviews are in no way immune from internal controversy.  The random nature of project selection is perhaps the biggest sticking point.  While we do this to avoid ‘cherry picking’, inevitably some of the projects that are selected are small-scale and have little strategic relevance to the countries and regions.  Some are also concerned about how much time and resources the effectiveness reviews are sucking up.

We know that what we are attempting to pull off can be improved on a number of fronts, in terms of rigour, learning, and engagement and ownership of country teams.  And the good thing is that we are able to modify and improve things as we go along.  So any constructive criticism, advice, etc. is most welcome.