“Index Mania” and Stories Untold
By admin November 6, 2018

Tags: , , , , ,

When doing research on international development, one can hardly open a newspaper or a web push without finding a reference to an international index. Being different from indexes solely used for investment and market outlook projection, these indexes are usually created by international organizations and cover key issues like corruption, human rights, education, freedom of speech. The scope of international ratings is still expanding due to the rising demand: “government officials refer to these indexes as measures of state performance, international organizations use them as comparative benchmarks, and journalists routinely cite them as authoritative in their stories”. It seems like we are blessed with a tool that everybody can understand and that is appropriate for a wide range of purposes.

In general, these indexes are based on original data collected by government and labor-intensive surveys with a large sample size. By building a framework with a complex of sub-indicators and giving them varied weights, creators have tried them best to eliminate lurking variables, reduce the confounding effect, and quantify the abstract ‘dependent variable’.

Although great efforts have been made, international indexes still draw harsh methodological and biases criticisms, especially when the definition of many terms they create is far from clear. For example, the definition of “innovation” adopted by the Global Innovation Index excessively emphasizes the element of “new(ness)” in innovations. Critics thus argue that innovation should be judged by its beneficial impact on society, rather than the novelty aspect. Besides, some indicators have been chosen simply because they were the data which was available.

When it comes to ‘freedom of the world’, each country’s scores aren’t a measure of anything concrete — they are, in essence, assigned by feel. Awarding 0 to 4 points for each of a country’s 10 political rights indicators and 15 civil liberties indicators is, by definition, an arbitrary exercise. This also happens in the measurement of corruption. It is found that overwhelmingly the same kinds of people are being asked for their perceptions. The opinions of an internationally focused elite, who is typically from a corporate background and perhaps a similar education, may be particularly informative in certain circumstances. As a result, the index cannot produce a broader truth about the public’s experience of corruption.

Global Peace Index was also widely challenged by many scholars in recent years. Failing to look at indicators of positive peace, such as the status of women and children, Qatar is in a “very high state of peace”, and therefore ranked higher than, for example, France, in 2014’s report. Others hold that the index gives poor weighting to each component. For example, the weighting for the rule of law is clearly too small. Another debate on peacefulness concerns violent demonstrations. Furthermore, rankings show changes in the index but don’t tell readers what’s driving them. Sometimes, the index doesn’t budge, but that doesn’t necessarily mean that there haven’t been significant changes that just happened to cancel each other out. In these scenarios, we got a number, but we are not really sure what it means.

These rankings become more problematic when we put the unintended impacts on policymaking into consideration. U.S. federal and state regulations bar some pension funds from governance and lower the buying low-rated investments, while Georgia and Rwanda have used their “most improved” awards on Doing Business Index as an “endorsement” to attract investment and to bolster domestic support for their governments. Realizing these possibilities and benefits, more policymakers are prone to design policies more to improve their rankings than to improve their real performance. Governments may be tempted to engage in “rating diplomacy” to improve the relative standing on the indexes instead of the situation on the underlying phenomena. Bahrain’s finance minister’s visit to the Heritage Foundation during the country’s 2011 crackdown on anti-government protests in Manama is a vivid example. The country wanted to as maintain its economic commitments despite the troubles.

What should be done in light of these problems? The raters should substitute nuance, caution and yet more diligence for shaky aggregates. Instead of trying hard to find common metrics, they should think about tailoring their evaluation according to the differences among countries and regions. For world leaders, staying calm and focusing on rooted problems can fundamentally guide them to better governance.


Read More:

The Global Peace Index 2014 – highlights and criticism

The Trouble with Combining, or Why I’m Not Touting the Global Peace Index

Is global competitive index a good standard to measure economic growth? A suggestion for improvement

The Tyranny of International Index Rankings

Niti Aayog is in denial about hunger in India – but the problem is worse than the statistics show

Why I have a problem with the Global Innovation Index

Global Innovation Index: General Deconstruction and Critique

There’s more to measuring corruption than Transparency International’s annual index, just released

Corrupting Perceptions: Why Transparency International’s Flagship Corruption Index Falls Short

“What’s wrong with the World Bank’s Human Capital Index?”, by David Edwards

Thanks for sharing !

Comments are disabled.