Are pro-poor renewables approaching a tipping point? Guest post by John Magrath

June 20, 2011

What does a theory of change look like?

June 20, 2011

How can research funders work better with international NGOs like Oxfam?

June 20, 2011
empty image
empty image

I spoke recently to a meeting of the UK Collaborative on Development Sciences. It’s a great initiative, bringing together 13 UK funders and stakeholders with an interest in international development research, but is ‘collaborative’ really a noun? Anyway, the topic was how research funders (mainly state funded) can link up more effectively with large INGOs like Oxfam. Let me talk you through the powerpoint……

First, why do INGOs do research? Above all, to improve impact of programmes and advocacy in three broad areas, according to a nice distinction made by my colleague Kimberly Pfeifer at Oxfam America: ‘tactical research’ (reactive to broader events and policy agendas); formative research (setting new agendas and directions) and evaluative research (MEL, learning lessons). INGO people are doers and activists, with little time for theorising – they think in terms of guidelines and toolkits. That is probably why UKCDS wanted to talk to me, because government is increasingly demanding that researchers demonstrate the impact of their research, rather than the beauty of pursuing knowledge for its own sake.

But what do we mean by the word ‘research’? For INGOs it is often much more about a clear narrative than about data. The risk is doing happiness v researchersviolence to a complex reality, but the upside is that we tell stories that stick in the heads of policy makers and others. There is also a priority on case studies and bearing witness – exploring how large scale phenomena (climate change, food prices etc) affect the lives of people living in poverty. What some academics dismiss as anecdotes are for INGOs (and most normal people) closer to reality than some massive number-crunching exercise (though we still need to be careful about correlation v causation and attribution). For some examples, check out the Oxfam publications website.

What does good policy research look like, from an INGO standpoint? A clear story, bringing together a decent review of the academic literature with those real life stories; preferably relevant to what is on the agendas of decision-makers over the coming months; drilling down into the issues of power, inequality and social relations that often go missing in conventional research. For impact it also needs a sprinkling of killer facts, an answer to the inevitable ‘what’s new in this research?’ question, and clear and convincing recommendations and solutions.

Are INGOs any good at research, thus defined? Generalizations are perilous, but here goes:

Strengths: at its best INGO research is rooted in real life, the experiences of partners and communities (e.g. our work on the impact of the global financial crisis, or forthcoming stuff on food prices); INGOs have been pioneers on participatory methods; the research packs a punch both in content and in the ability of INGO media teams to make a media splash that gets it noticed. And they have a global constituency and reach that many academic researchers can only dream of.

Weaknesses: often stronger on qualitative than quantitative; sometimes a bit cavalier on methodology (although we outsource a lot of research to academics which, if they’re any good, should fill that gap); weak systems of peer review (and some confusion over what constitutes a ‘peer’); suffers from short INGO attention spans, so few examples where research builds up over time; patchy links to developing country research institutions and always short of cash and capacity compared to the formal research institutions.

after-peer-reviewHow can funders improve the relevance and use of research by INGOs? Well, they could fund it directly of course, but that is often going to be difficult given the way they are set up, so here are some other ideas. Insist that research institutions work with INGOs to co-design research programmes (the norm is alas, for an institution to decide on a largely irrelevant agenda  and then approach the INGO as an afterthought to help with the communications, or ‘do the voices of the poor bit’.) Sure, we could (and do) take the initiative and approach research institutions with our own ideas, but the timescales, interests and approaches are often just too different to find common ground. Funders could provide incentives to help bridge the gap.

That means understanding what research INGOs are going to need over the next few years. Luckily the level of intellectual herding is pretty high, so if you get a bunch of them in a room, they will probably all come up with a similar set of priorities (current ones would probably include climate change, scarcity, food security, theories of change, measuring impact, multipolar world and the absence of gender and disaggregated data from most research questions).

And a few more specific suggestions for the higher education researches themselves, (and where prodding by funders can probably help):

If you want access to communities, the research had better be relevant to the people and partners (e.g. testing new approaches through action research). It needs to be properly discussed at draft stage and disseminated and discussed locally on publication. INGO staff time and direct costs (and those of their partners on the ground) should be properly funded. Finally, you need buy-in at country level, where harassed staff may have very different priorities from INGO HQ.

What is at stake is, I think, pretty important – building a regular and productive interchange between funders, higher education institutions, thinktanks and INGOs. Funders could help by creating incentives for better links between these groups, requiring researchers to demonstrate impact and relevance. They could also help create a space for collective reflection on research priorities among INGOs (that only happens in a very ad hoc way at the moment, for example by everyone commissioning papers from Alex Evans……) and build INGOs’ capacity to understand, commission and use existing research (as well as do a bit themselves).

Any other suggestions?

5 comments

  1. Good initiative, indeed, but one of the biggest problems to overcome I think is the disjunctures in a) timescales and b) (the associated issue of), research “aims” between research institutions and INGOs. Are these really compatible without some uncomfortable forcing together?. Or does the new focus on impact give an opportunity to find a “sweet spot”?.

    I’d distinguish 3 research timescales. Timewise there is some compatability around lengthy longitudinal research over 5, 10 or even more years (and INGOs should be better at getting involved in it). But, there’s disjunctions between much NGO research that is for 1-2 years (which is that length of time because it is very much related to change goals, which in turn are more related to advocacy timetables/opportunities than to non-advocacy aspects of programming), and timetables for much “academic” research that is 3-5 years long (and the funding system for academic research is geared to that timetable, as is publication discipline).

    Maybe now with the emphasis on “impact” there are opportunities to “compromise” – I don’t mean by that necessarily coming to some middle ground solely on timing (3-4 years anyone?) – but rather to find ways to give more depth to 1-2 year NGO research by incorporating academic insights, methodology and critiques (in a lite form), and sharpen up 3-5 year academic research with NGO insistence on the relevance of research aiming for real change impacts.

  2. This is a very important and interesting discussion. I think there is often a rather ‘snippy’ and unhelpful discussion that goes on between different people on this topic that does no one any favours – academics, NGOs or government.

    We need to be more much more realistic and honest about (1) the value of research in society, (2) about the different kinds of research and research-related activities that take place, and (3) about the roles of different kinds of people involved and their different priorities.

    For the first, research is about chipping away at a coal face – it’s important, but it’s often boring, and you often do not know what you are going to find. It is important for its own sake (despite Duncan’s scepticism) because you never know when you are going to need it. Building up a store of useable dynamic knowledge is a vital asset for any society, a public good, and it is very important that there are always people who are plugging away at it.

    For the second, research is not just about the design of studies, the gathering of data or the writing up and dissemination of results. It is also about other associated activities – synthesising, bridging different audiences, renewing ideas, networking, building convincing stories and meanings, and creating public policy debate. But we should be honest about the fact that research hardly ever influences policy decisions in direct or significant ways, though it might help to influence the wider climate of thinking in which decisions are made or influenced. It is only ever a tiny part of the bigger picture of politics and power.

    For the third, different people clearly have different interests in research. Academics tend to be interested in research as something to be done carefully and well, and in line with wider disciplinary debates and traditions. They see research as a specialised form of practice (like NGO staff, academics are ‘practitioners’ too!) But I think most academic researchers in the social sciences interested in development issues are very concerned that their research engages with the public and makes a difference. Policy people and NGOs tend to be far more interested in research in ‘instrumental’ terms – they want research findings that help them to take forward the agendas they have identified. They want research stories that help justify the decision they have decided to implement, or show the importance of the issues they believe in or are responsible for. They work within shorter time frames, they are impatient with complexity. Finally, government is now more concerned with research in terms of ‘value for money’. It also views research increasingly instrumentally, and this is not unreasonable given that it is accountable for how it spends tax payers’ money. However, it now wants researchers to construct somewhat artificial stories of ‘pathways of impact’ to show how the research they fund impacts upon the world. Many academics are encouraged by the pressures to improve relevance and engagement, but object to the idea of ‘impact’.

    What is to be done? We need to recognise the different – equally valid – interests various people have in research, and show more respect to each other. We need to build more conversations around the different kinds of research taking place, synthesise it so that it can be influenced and accessed by different groups for both direct and indirect uses.

  3. I wonder if we need to be more honest about agendas as well? It is not just timescales that are sometimes different. INGO’s do need good research to tell them what is going well, or badly, what they need to do more of, less of etc. But also (and increasingly) they need targeted research to help locate themselves in the good books of donors and demonstrate that they are value for money. this often means validating the status quo. Researchers on the other hand may be looking to find a new angle, move a debate on and (perhaps too cynical?) make a name for themselves amongst their peers. These agendas can be complimentary, but where things have gone wrong in my experience it is often because there wasn’t enough up front honesty about expectations and ambitions

  4. Another interesting post, Duncan. Thanks.

    With a note to disclose my bias (I am part of one of the project teams doing the work), the Committee on Food Security’s High Level Panel of Experts has a mandate along these lines – to write reports that reflect more than one kind of knowledge system and that creates a space for non-academic learning. It isn’t easy. You have to write in English. You have very little time to do the work (though that might improve, I hope, over the years). You have to cite “credible” sources. But I think the HLPE reports could prove an interesting experiment over time. Where the report prepared for the G20 Agriculture Ministers’ summit from the international institutions does not even mention the fact that the Doha Agenda has collapsed, the HLPE report is free to note this reality and move on. My hope is we will generate some refreshing debate in government circles. Fingers crossed!

  5. Really interesting post and comments. I like your suggestions for academic researchers wanting to work with NGOs. I came up with a few recently, based on things I saw while working in NGOs. At the risk of completely alienating myself, I’ve copied them below. Some overlap with yours, and some more serious than others:
    · Check to see whether what you think is new and exciting is actually stuff NGOs/governments have been doing for years.
    · Don’t just revel in exciting ideas about how difficult it is to predict or control anything – tell us how to apply it.
    · If you’re invited to present/speak at a workshop, find out what they (i.e. the practitioners) want, don’t just talk about your current interest.
    · If the practitioners’ time will be needed in the research, get this agreed (and how much, when, ground rules around communication etc) at the start (and get managers’ buy in).
    · Recognise practitioners’ strengths and knowledge (try to tone down the academic arrogance that sometimes appears).
    · Don’t just criticise and point to gaps – understand the reasons for them (gaps in both NGO programmes and their research capacity).
    · If you say you’re going to provide a report on the findings, please do so.
    · Be open to collaboration on writing/joint authorship (and build in time to support this).
    · Give credit to the practitioners who have helped with/funded your research when you’re presenting it (within confidentiality limits, obviously).
    · Be in it for the long-term – help in trying to find and implement solutions (but don’t take over).

    I’m sure there are lots of academic horror stories about trying to work with NGOs (I’ve heard a few) – maybe we could get their views on what NGOs can do to ease relationships?

    On the point about NGO research being rooted in real life: there have been concerns in the past that NGOs’ policy/advocacy research is sometimes distant from their service delivery/grassroots work. I found it was often a struggle to get the information needed for policy research from the programmes (e.g. due to different reporting requirements, general busyness of field staff). Do you think this is still an issue, or are NGOs managing to bridge the programme-advocacy divide in their research?

Leave a reply Cancel

Translate »