I have been asked how the report on the achievement of the SDGs is calculated, so I try to summarise it.
First of all, how would you characterise the SDGs' achievement in the last year, even without the SDGs, the main concern was the COVID19 pandemic. Of course, many of the SDG indices were affected. The only good thing is that the burden on the environment has been reduced and that we may be able to shift our economic activities to take account of the SDGs when we recover from the pandemic - the Great Reset set out in the GEF. However, as we have seen with the recent re-emergence of coronavirus infections worldwide, it isn't easy to see when this situation will improve. Despite this, there is a growing movement to achieve the SDGs, especially in Asia, where achievement seems to be faster than in other countries. However, it is still the developed countries in Europe that have achieved the highest levels of SDGs. So, how is the level of achievement calculated?
The overall score of a country's SDG index and its SDG scores can be interpreted as a percentage of optimal performance. In other words, 100 is when the SDG vision is achieved, and the SDG score is the current situation. The difference between the two is the distance, expressed as a percentage needing to overcome to achieve the vision.
Creating comparable scores and rankings, the report used the same basket of indicators across all countries. For example, I looked at the report on urban poverty, and it seems to include the number of homeless people. Then, the Min/Max standardisation is used because it is not possible to say whether a figure of 80% or 4,000,000 people is large or small without something else to compare.
In other words, there is a linear link between impact and what the number indicates. In practice, however, the impact and the number are not always linear: to take one extreme example, it is very different to say that the poverty rate will go from 80% to 100% is same as going from 20% to 40%. In a world where the poverty rate is 100%, it is a complete collapse of society, because there is no one to support that society. This is different in the society of 20% and 40% poverty because more than half of the population can support this society. The dashboard for this score says it's on a scale of 1 to 3, but in fact it also uses a number in the middle, say 0.5, so it's a 5-point scale.
Then, to make things trickier, there are 17 SDGs, so we have to weight each goal. Would a score of 2 for poverty be of equal value to a score of 2 for climate change? A poverty score in a developing country may be more meaningful than a climate change score. But when it is challenging to decide which goals are important even in one country, it is almost impossible to weight them across the 160+ countries that have adopted SDGs. I have seen the paper on the methodology and it seems that principal component analysis (PCA), expert opinion, and subjective weighting were also taken into account. But, they couldn't make up their minds (^^), so the report divided everything equally by 17. This equal division does not mean that "no weighting was given", but that all goals were weighted 1/17. This is definitely the wrong weighting, but there is no other way to do it. To be honest, I would use this method too. Just don't make the mistake of thinking that it is unweighted. About 10 years ago, when I was working as a JICA expert, a great Californian professor came to me, and he kept saying that his project was not weighted. I explained it to him, but he didn't understand this equal distribution thing. I hope you all understand this point!!
The paper explaining the analysis method said that the subjective weighting could contradict one of the SDGs' objectives, "No cherry-picking", so they did not use the method. That's true, but if it's not an academic study, I think practitioners can use subjective weighting because it has the non-scientific advantage that people who use it will want to adopt it more. In fact, that benefit is documented in the OECD's report on index analysis.
In summary, the SDG scores are produced in a rather vague way that I use to reconcile social science surveys and research problems in general. Some people may be disappointed, but that's to be expected in the social sciences, where there's not much reliable big data. However, if you look at the actual rankings, you can see a certain degree of credibility. So the SDG scores and rankings mentioned in this report can be taken as more than half the story. More importantly, we can understand how the SDGs are progressing in the same country or region by doing this annually. In this sense, I think this ranking is meaningful.
Comments