Should poverty be defined by a single international poverty line, or country by country? (and what difference does it make?)

This guest post comes from ubercrunchers Ugo Gentilini (World Food Programme), left and Andy Sumner (Institute of Development Studies), right

International poverty lines are calculated by the World Bank: $1.25 per day per person is said to represent the ‘absolute poverty line’, below which a person can hardly survive.

This is calculated from the mean of the national poverty lines for the poorest 15 countries. A slightly higher line, set at $2 per day per person, is the average of the national poverty lines for all developing countries.

To date, these lines have been accepted as the universal poverty metric, underpinning global goals such as the MDGs – at least MDG 1a – and discussions on how the world is doing in reducing poverty. But that is increasingly coming into question. One problem is that poor people increasingly live in middle-income countries (see here), and so are not represented as long as the extreme poverty calculation sticks with the poorest 15 countries (which are actually only home to about 10% of the world’s absolute poor).

Another problem is that while international poverty lines allow us to compare like with like in monetary terms, at national level, all countries define poverty for themselves, often using different approaches.

Poverty levels in OECD countries, for instance, are often defined relative to median income (e.g. below 60 percent of the median for the EU).  By contrast poverty lines in other countries are “absolute” – that is, the income required to meet basic needs (e.g. Afghanistan).

Unsurprisingly, given these issues, the recent update of global poverty estimates by $1.25 and $2 poverty measures (see here) has had a few critiques (see here), while alternative multidimensional poverty measures are also – rightly – attracting attention as they gather speed and are taken up.

So what difference does it make if instead, we look at how many poor people there are in the world, based on how poverty is defined in the countries where those people live (rather than by international poverty lines)?

To answer this, we added up all the country-level poverty data based on national poverty definitions to produce a new and different perspective on global poverty, based on national measures from 160 countries.
 
What did we find? Three things:
 
First, there are 1.5 billion people living in nationally-defined poverty, a billion of whom are in middle-income countries (MICs). This corroborates the view that global poverty has increasingly become a middle-income country phenomenon, although much of this is down to just five countries – the PINCIs – Pakistan, India, Nigeria, China, and Indonesia (see here and here and here).

Second, when poverty is defined nationally, one in ten (170m) of the world’s poor live in high-income countries. (Of course one could question comparability- absolute and relative poverty – so when we present the global poverty data we do so with and without high-income countries). By its own definition, the US has 45m poor people.

Third, while global totals are the same, the overall number conceals some big national variations in poverty numbers depending on whether national or international lines are used (see bar chart). In Mexico and Bolivia, for example, poverty rates according to national lines are more than 40 percentage points higher than those based on the international $1.25/day measure. In Africa, by contrast, for various countries (e.g. Uganda, Tanzania, Liberia, Burundi, Nigeria and Malawi) poverty rates resulting from international lines are much higher than from national measures (e.g. about 35 percentage points higher in Tanzania and 20% in Malawi). And in India 45m people are missing in national poverty estimates that would be counted by international poverty measures (see here).

Why might a focus on nationally-defined poverty be useful? Three reasons:
 
First, such a focus might fit better with the domestic task of forging national social contracts, as poverty increasingly becomes about national inequality (see here). National poverty measures tend to be what matters most to policy makers in-country. However, those domestic measures are moving closer to the international $1.25 line in some countries – for example in China (see here).

Second, expressing poverty in national terms implies a greater degree of involvement of national actors in defining “what is poverty?” in a given context.That has practical consequences: in a number of cases, countries are testing how to better connect national measures with eligibility for domestic social protection programs.
 
Third, the recognition that poverty (relative or absolute) exists everywhere entails a shift in poverty thinking – framing poverty as a universal issue relevant to all countries, rather than a ‘them and us’ question.

more than just a number

As the discussion on poverty measurement and classification evolves, it might be interesting to broaden the range of countries, so that highly-populated MICs, or at least the 10-20 countries where most of the poor live, become the basis for the calculation of international standards for absolute poverty.

This could entail establishing global lines (e.g. current $1.25/day) not on the average of lines of the poorest countries, but on the average of the countries with the highest numbers of poor people – 80% of the world’s poor live in just 10 countries and 90% in twenty populous countries (most of which are not currently part of the $1.25 calculation).

Such a process would perhaps better synchronize global measurement to the shifts in global poverty.

Can we demonstrate effectiveness without bankrupting our NGO and/or becoming a randomista?

Back in March there was a fascinating exchange on this blog between Ros Eyben and Claire Melamed on the role of measurement in development work (my commentary on that debate here). Now one of Oxfam’s brightest bean counters (aka ‘Programme Effectiveness Adviser’), Karl Hughes, explains where Oxfam has got to on this:

Eric Roetman, in a recent 3ie working paper, A can of worms? Implications of rigorous impact evaluations for development agencies, tells a provocative tale of the experiences of International Child Support (ICS) in Kenya carrying out randomised control trials (RCTs) in partnership with several  world-renowned quantitative impact evaluation specialists.  ICS saw itself evolve into a “development lab”, where the bulk of its staff became devoted to supporting the organisation’s research, as opposed to development, operations.  Given ICS’s desire to revert back to its roots, it eventually opted to get out of the RCT business.

ICS’ story relates directly to issues further explored in another recent 3ie working paper I recently co-authored with Claire Hutchings, another one of Oxfam GB’s global MEL advisers, entitled Can we obtain the required rigour without randomisation?  Oxfam GB’s non-experimental Global Performance Framework.  The central issue is this: We in the international NGO community are all too aware of our need to up our game in both understanding and demonstrating the impact – or lack thereof – of the various things we do.  But what really baffles us is just how to do so without going down the “development lab” route.  (This is not to imply that “development labs” are bad; in fact, the more their findings inform our programming, the better.)

The bottom line, as outlined in our paper, is that evaluation is research, and, like all credible research, it takes time, resources, and expertise to do well.  This is equally true no matter what our epistemological perspective – positivist, realist, constructionist, etc.  This is perhaps why, rather than using  those offered by mainstream academia, we as a sector are so quick to experiment with seemingly more doable alternatives such as Most Significant Change, social return on investment (SRI), outcome mapping, and participatory M&E.  They’re all very well, but those of us who feel a need to go further find ourselves at a loss.

One popular way of attempting to demonstrate effectiveness, being pursued by several international NGOs, which we comprehensively bash in the paper, is dubbed “global outcome indicator tracking.”  Here, the organisation in question gets all its programmes/partners  to collect common data on particular outcome measures, e.g. household income.  All these data are then aggregated (only the gods know how) to track the welfare of global cohorts of programme “beneficiaries” over time.  If there is positive change in relation to the indicator from time 1 to time 2, the organisation can boast about how much impact it is generating.  Aggregation complexities aside, the underlying foundations of this approach are inherently precarious.  In general, outcome level change is influenced by numerous extraneous factors, e.g. rainfall patterns in rain-fed agricultural communities.  Consequently, even if we are able to capture reliable data on a decent outcome indicator, its status will go up and down and all around not matter what our interventions are and/or how well they are implemented.  Any consideration of attribution is entirely absent.

But what of the fact that donors have been encouraging us to pursue outcome indicator tracking for decades now through instruments such as the logframe, as part of ‘good practice’?  In a paper entitled, The Road to Nowhere, Howard White argues that the United States Agency for International Development (USAID) identified the futility of the outcome indicator tracking strategy some years ago and, consequently, abandoned it.  I worked on a USAID funded orphan and vulnerable children (OVC) programme from 2005 to 2010, and yes we were only required to report on outputs, so perhaps this was the consequence of this realisation.  (Incidentally, USAID also came to the realisation that there was no evidence-base established on what works and what does not in OVC programming after all the billions that it spent and seems to regret not having supported the rigorous evaluation of key OVC care and support interventions.)  To what extent have the other donor agencies recognised the fallibility of outcome indicator tracking?  Sadly, there is plenty of evidence to suggest that many are still operating in this outdated paradigm.

So where does this leave us as NGOs?  While Oxfam GB has not come up with a panacea, it is attempting to pursue a strategy that is reasonably credible.  Each year, we are randomly selecting and then evaluating, using relatively rigorous methods by NGO standards, 40-ish mature interventions in various thematic areas.  The causal inference strategy differs depending on the nature of the intervention.  For community-based interventions, for instance, where we are targeting many people (aka large n interventions), we are attempting to mimic what RCTs do by statistically controlling for measured differences between intervention and comparison populations.  Evaluating our policy influencing and “citizen voice” work (aka small n interventions), on the other hand, requires a different approach.  Here, a qualitative research method known as process-tracing is being used to explore the extent to which there is evidence that can link the intervention in question to any observed outcome-level change.

It is not that the above approaches are free of  limitations.  In the case of large n interventions, for instance, given that programme participants have not been randomly assigned to intervention groups, coupled with the conspicuous absence of proper baseline data, we cannot absolutely guarantee that any observed outcome differences are the result of the workings of the intervention in question.  The  process tracing approach is also retrospective in nature, when ideally the research should take place throughout the life of the advocacy or popular mobilisation initiative.  But, hey, what we are doing is not too shabby, especially considering that we are no “development lab.”  Moreover, every evaluation design –  even the golden RCT – has  inherent limitations.  Nonetheless, if anyone has any suggestions on how NGOs in general and Oxfam in particular can do a better job at both understanding and demonstrating impact, I’d love to hear them.