At last, a sensible suggestion for post2015

After my ‘bah humbug’ paper on post2015, I’ve been largely avoiding the subject as a monumental timesuck. However, a combination of Sabina ‘multidimensional’ Alkire and Andy ‘bottom billion’ Sumner is an unstoppable force, so I’m making an exception for their new paper, Multidimensional Poverty and the Post-2015 MDGs, which is worth a skim.

What Sabina and Andy do is use her previous work for UNDP on multidimensional poverty indicators (MPIs) to square an important circle. They suggest building an ‘MPI 2.0’ based on whatever combination of issues is finally agreed in the post2015 discussion. This would produce a single number that summarizes a country’s overall progress towards the post2015 goals.

That in turn would allow the post2015 process to generate more traction on national governments (the lack of which is the subject of my paper) through league tables. Imagine if every year, all countries (including the rich ones) are ranked on a comprehensive human development table that (unlike the Human Development Index and other similar efforts) has buy in and recognition from across the international community. Each annual report would pick out the countries that have risen/fallen relative to the others. Regional tables could compare India and Bangladesh, or Peru and Bolivia, to generate extra public interest and pressure on decision makers. Within countries, an MPI could highlight regional disparities (see map).

A particular advantage of the approach is speedy feedback for policy makers: The MPI reflects effective social policy interventions immediately. With measures of income poverty, a positive social change – for example in schooling or clean water – may not be reflected for a number of years.

One of the lasting institutional legacies of the MDG process is the investment in better quality data needed to assess progress – this proposal would build on that.

One suggestion though – MPI 2.0 is a dreadful name. Why don’t we just call it ‘poverty’ and argue that it should replace $1.25 a day as the international standard?

And here’s me at a recent IDS seminar explaining why I’m so underwhelmed by the general post2015 debate.

Why do some (better) alternatives to GDP get picked up, while others sink without trace? Useful new study on political economy of indicators.

Took me a while to overcome my reluctance to read a document subtitled ‘Deliverable 1.1’ on the front page (yuk), but I’m glad I did so. The paper ‘Review report on Beyond GDP indicators: categorisation, intentions and impacts’ (cracking title too…..) is published by BRAINPOoL – ‘Bringing Alternative Indicators into Policy’ (is that the sound of teeth grinding?).  The authors come from Eurothinktanks CUEC (Czech Republic) and the new economics foundation (UK).

But enough of the snark, because the paper is really good. Rather than add yet another indicator to the smorgasbord of measures that go ‘beyond GDP’, it tries to understand why some of these indicators get picked up, while others sink without trace. Asking the right question, as Einstein said, is a lot more than half the battle.

The researchers sought to answer that question with a mix of desk research, interviews with indicator nerds, and media analysis. First they tried to categorize the ever-growing list of alternative indicators. They came up with:

• Level of impact – international, national, local
• Indicator domains – environmental, social, economic
• Indicator approaches – subjective, objective
• Indicator types – single indicator, set/dashboard, aggregated, composite
• Envisaged users – politicians/policy makers, public, experts
• Link to GDP – adjusting GDP, “replacing” GDP, supplementing GDP

Media analysis showed that single figure indicators get far more pick up than complex dashboards, as do those that enshrine ‘simple and meaningful concepts’ like the Human Development Index or Ecological Footprint, which lead the pack by some way (see chart).

But a media splash, while useful, is not the main purpose of the indicators studied, which was overwhelmingly to influence policy. When have they achieved it and why?

• The most basic influences noted were transmission to or reference by a decision maker.
• The next step up from this is use of Beyond GDP indicators in assessment. Examples of this include GPI, ISEW and QUARS being integrated into regional or local official assessment frameworks in the US, UK and Italy. Also noteworthy is the Ecological Footprint being used to set environmental impact targets in several national governments.

• Actual policy changes connected to indicators were also reported. These include (i) the implementation of schemes to improve habitat for farmland birds after changes to the UK Defra Sustainable Development Indicator set (ii) the investment of $15 billion by the UAE government to develop alternative energy sources spurred on by a poor ranking in the Ecological Footprint.
• The most prolific successes, however, were achieved by local level indicators, perhaps because the distance between producer and user of the indicators is much smaller, making it easier to achieve a better ‘fit’ while also achieving legitimacy and relevance. For example, the Jacksonville Community Council Indicators are believed to have been instrumental in reducing infant mortality rates, recidivism and water pollution in the area. The Council of Europe SPIRAL project, meanwhile, achieved change to the delivery of food relief in France and the transformation of the approach to homelessness in Belgium.

And what lessons did the authors draw from all this?

“Indicators were successful when they had real relevance for policy makers. Crucially they need to measure something that policy makers believe they can influence. Subjective well-being indicators when used incorrectly can appear to fail this test, which explains efforts to build the evidence base for how policy can influence wellbeing. Another factor here is cost. In the current climate, indicators that provide clues for low cost policies, or indeed those that can help save money, are of particular interest.

Salience for a broader audience is also crucial and entails the elements of simplicity, understandability and good communication. Initiatives are effective when they allow one to produce a simple and attractive message that relates a meaningful concept. Using communications experts and avoiding taboo words were also identified as being important.

Indicators need credibility and legitimacy. Aside from the requirement of quality data, the appearance of neutrality was seen as the best route to achieve this with some interviewees comparing advocacy organisations’ data unfavourably with that of National Statistical Offices.

Developing the indicators with the audiences at whom they are targeted and/or encouraging participation (in the way that the Jacksonville Community Council Indicators initiative has done) is also seen as a key success factor. The importance of relationship-building also applies to policy makers with most initiatives that had achieved policy success citing direct face-to-face channels as vital.

Several barriers to the success of Beyond GDP indicators have also been identified. The economic crisis was viewed by many interviewees as a challenge for this agenda as it has moved the policy focus. Ideology and vested interests are also noted barriers with subjective well-being and composite indicators receiving strong resistance from those with libertarian or right of centre political views.”

Not sure I buy that last point – isn’t a crisis just as likely to create opportunities for adoption of new indicators, especially if the failings of existing indicators are seen as partly responsible for the crisis? I remember being struck by a presentation from the South Korean statistics office (I really do have all the fun), arguing that Korea was adopting a new quality of life index, based on over 100 objective and subjective indicators of wellbeing, because GDP was incapable of explaining why an ‘economic miracle’ like Korea had the highest suicide rate and the lowest birth rate in the OECD. Failure is a great source of innovation.

The Brainpool project continues and is worth following (especially if they ditch the naff titles). Oxfam is getting involved, working with nef in Brazil and India to explore likely reactions by politicians and voters to possible indicators. It will build on Oxfam Scotland’s Humankind Index, which was an earlier foray into this field. I hope the research includes applying a more serious power analysis to the findings in this report: what fractions of the state, private sector, political class or popular movements adopt new indicators, either individually, or in coalition with others? When and why? That could get very interesting.

GDP v Well-being – the Stiglitz Commission and other news

According to Otto von Bismarck, the father of modern Germany, ‘Laws are like sausages. It’s better not to see them being made.’ Having skimmed the report of the ‘The Commission on the Measurement of Economic Performance and Social Progress’, commissioned by President Sarkozy and released last week, I would say GDP (Gross Domestic Product, the standard measure of a country’s economic performance) is right up there alongside laws and sausages – it’s pretty unsavoury.

The Commission, chaired by Joe Stiglitz, first picks over the shortcomings of GDP, and argues that ‘those attempting to guide the economy and our societies are like pilots trying to steering a course without a reliable compass’. It then discusses how to a) improve GDP within its current definition, b) measure quality of life and c) measure sustainability. The report is not for the fainthearted – a 12 page executive summary, a 64 page ‘short narrative’, and a full 200 page set of ‘substantial arguments’. There are also some shorter, rather more accessible coverage in the Guardian, The Economist and Financial Times, among others, and op-eds by Stiglitz. Here are some excerpts from the exec sum, and some thoughts of my own at the end and in square brackets. It’s a long post, but a lot shorter than the original!

[Why we need better measures of services, the quality (rather than quantity) of goods and government activity]
‘The time has come to adapt our system of measurement of economic activity to better reflect the structural changes which have characterized the evolution of modern economies. In effect, the growing share of services and the production of increasingly complex products make the measurement of output and economic performance more difficult than in the past.

In some countries and some sectors, increasing “output” is more a matter of an increase in the quality of goods produced and consumed than in the quantity. Capturing quality change is a tremendous challenge, yet this is vital.

Governments play an important part in today’s economies. They provide services of a “collective” nature, such as security, and of a more “individual” nature, such as medical services and education. These services tend to be large in scale, and have increased considerably since World War II, but, in many cases, they remain badly measured. Traditionally, measures have been based on the inputs used to produce these services (such as the number of doctors) rather than on the actual outputs produced (such as the number of particular medical treatments).

Because outputs are taken to move in tandem with inputs, productivity change in the provision of these services is ignored [see graph for an example from Denmark]. It is thus important to come to grips with measuring government output.

[It's better to measure income/consumption and wealth, not just production]
There appears to be an increasing gap between the information contained in aggregate GDP data and what counts for common people’s well-being. Because no single measure can summarize something as complex as the well-being of the members of society, our system of measurement must encompass a range of different measures [through a] broad statistical system that captures as many of the relevant dimensions as possible:

When evaluating material well-being, look at income and consumption rather than production [as currently, with GDP]. The available national accounts data shows that in a number of OECD countries real household income has grown quite differently from real GDP per capita, and typically at a lower rate [see bar chart].

Income and consumption are crucial for assessing living standards, but in the end they can only be gauged in conjunction with information on wealth. A household that spends its wealth on consumption goods increases its current well-being but at the expense of its future well-being. Measures of wealth are central to measuring sustainability. What is carried over into the future necessarily has to be expressed as stocks – of physical, natural, human and social capital.

[Broaden income measures to non-market activities like leisure and raising families]
There have been major changes in how households and society function. For example, many of the services people received from other family members in the past are now purchased on the market. This shift translates into a rise in income as measured in the national accounts and may give a false impression of a change in living standards, while it merely reflects a shift from non-market to market provision of services…. [We need to] start with information on how people spend their time [including leisure, though putting a value on it is particularly difficult]

Well-being is multi-dimensional
The following key dimensions should be taken into account:
i. Material living standards (income, consumption and wealth);
ii. Health;
iii. Education;
iv. Personal activities including work
v. Political voice and governance;
vi. Social connections and relationships;
vii. Environment (present and future conditions);
viii. Insecurity, of an economic as well as a physical nature.

Objective and subjective dimensions of well-being are both important
The information relevant to valuing quality of life goes beyond people’s self-reports and perceptions to include measures of their “functionings” and freedoms. In effect, what really matters are the capabilities of people, that is, the extent of their opportunity set and of their freedom to choose among this set, the life they value.[in case you were wondering, yes, Amartya Sen was on the Commission]

Inequalities in human conditions are integral to any assessment of quality of life

Inequalities in quality of life should be assessed across people, socio-economic groups, gender and generations, with special attention to inequalities that have arisen more recently, such as those linked to immigration.

It is possible to collect meaningful and reliable data on subjective as well as objective well-being.

Subjective well-being encompasses different aspects (cognitive evaluations of one’s life, happiness, satisfaction, positive emotions such as joy and pride, and negative emotions such as pain and worry)

Sustainability means measuring stocks, not just flows
Sustainability assessment requires a well-identified dashboard of indicators. The distinctive feature of the components of this dashboard should be that they are interpretable as variations of some underlying “stocks (quantities and qualities of natural resources, and of human, social and physical capital.)

There are two versions to the stock approach to sustainability. One version just looks at variations in each stock separately, assessing whether the stock is increasing or decreasing, with a view particularly to doing whatever is necessary to keep each above some critical threshold. The second version converts all these assets into a monetary equivalent.

[The Commission suggests] a more modest approach, i.e. focusing the monetary aggregation on items for which reasonable valuation techniques exist, such as physical capital, human capital and certain natural resources. In so doing, it should be possible to assess the “economic” component of sustainability, that is, whether or not countries are over-consuming their economic wealth.

The environmental aspects of sustainability deserve a separate follow-up based on a well-chosen set of physical indicators. In particular there is a need for a clear indicator of our proximity to dangerous levels of environmental damage (such as associated with climate change or the depletion of fishing stocks.)’

So will all this make a difference? After all, people have been bashing GDP for decades, but it has proven remarkably impervious to attack. I think there are reasons for both optimism and pessimism on the report’s impact.

Optimism both because of its (intellectual and physical) weight (several Nobel prizewinners, a Presidential commission and high quality analysis), and because it is part of a wider groundswell of calls for new ways of measuring social and economic performance. On 8 September the European Commission issued a communication entitled ‘GDP and beyond: Measuring progress in a changing world’, which argued that ‘there are no insuperable technical obstacles to developing the quality and scope of our indicators even further so that policy decisions can progressively be based on a more integrated, balanced and timely view of social, economic and environmental facts.’ The Commission announced that it ‘intends to develop a comprehensive environmental index and improve quality-of-life indicators’ for inclusion in both the European System of Accounts, and member states’ own national accounts systems. So these ideas are becoming reality in the statistical systems of a major chunk of the global economy. This promising combination of political momentum and technical innovation is also reflected in the OECD’s ‘Measuring the Progress of Societies’ project, which has a big conference in South Korea next month. I am speaking there, so expect an update.

Pessimism because the Commission comes out against a single alternative to GDP (probably rightly), and instead proposes various ‘dashboards’ of indicators that allow people to construct different composite indices (like the Human Development Index and dozens of others) for particular purposes. That’s fine if all parties can agree on the particular composite that best reflects a particular issue, but otherwise, everyone grabs the (different) indicators that best ‘prove’ their case, and the debate rapidly gets polarized and stuck. One of the reason’s for GDP’s remarkable durability is that the complexity of the alternatives and improvements frightens the life out of policy makers. Statistics wonks are just not good at KISS (‘keep it simple, stupid’).

Part of me still wishes people would take heed of the sign that Einstein had hanging over his desk at Princeton: ‘Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted.’ But for the foreseeable future, policy makers will insist on metrics, and the rest of us should probably back Stiglitz, the OECD and the Commission in trying to ensure that they are as good a reflection of reality as possible.