Tim Harford on The Ease of Doing Business Report

But before anything else, let’s take a moment to acknowledge the title of the article – if you haven’t seen the movie, please do. One of my favorite Judie Dench movies.

You may have heard of the problems associated with the Ease of Doing Business report. (The reason I have linked to the Wikipedia page rather than the original page is because it wasn’t opening for me. Your mileage may vary.)

Even a spreadsheet can become a victim of its own success. Just ask the World Bank’s Doing Business report. While many worthy publications from the World Bank are never downloaded, Doing Business has been a smash hit for years. No longer. Amid an ugly scandal about data manipulation that has left the head of the IMF, Kristalina Georgieva, fighting for her career, Doing Business has been cancelled.
The power struggle at the top of the fund involves: a three-way tussle for influence between the US, Europe and China; rivalry between Georgieva, former chief executive at the World Bank, and the current, Trump-nominated bank president David Malpass; and domestic US politics. (Democrats have long disliked the Doing Business report’s low-regulation tone.)
The accusation is that in 2017 the World Bank’s leadership, including Georgieva, pressured the Doing Business team to improve China’s ranking in order to keep the Chinese government happy. The case for the defence is that Georgieva’s team were merely double-checking a sensitive number, that China’s ranking barely moved (from 85th to 78th), and anyway China is now ranked far better (25th) than when Georgieva was at the bank. The fight is as fascinating as it is unedifying.

https://timharford.com/2021/11/notes-on-a-statistical-scandal/

---

By the way, if you want to learn how to write columns well, you could do a lot worse than reading these three paragraphs.

A short, interesting sentence to begin the column, followed by an easy to read first paragraph that explains what the problem is. The next two paragraphs provide context, give additional details, and bring the reader up to speed, so that Tim Harford can get to the points that he wants to make regarding the whole issue. And contrast that with what I have managed to do so far: four paragraphs, one lengthy excerpt, and two tangential points, one of which is meta. Ah well.

---

But all of that aside, take some time out to read Tim Harford’s column before reading what follows.

1. What was the report optimizing for?
   
   Originally, it seems to have been an attempt to help interested entities understand how easy (or not) it was to do business in a particular country. This helps entrepreneurs (domestic and international) understand some of the potential impediments to starting a business. The report lays out the processes involved in starting a business, and speaks about the length of time required to complete those process. That is surely a good thing, correct?
   
2. Is a report not the same as a ranking?
   
   What matters more to you as a student when it comes to examinations? Are examinations a way for you to reflect upon how much you’ve learnt and what remains to be learnt, or are examinations a way to understand where you are in the pecking order? The problem with the Ease of Doing Business report wasn’t the report itself, it was the rankings that were generated on the basis of the reports.
   As Tim Harford says in his column: “But Klein has one regret: the original decision to publish an overall ranking of which countries were the best and the worst in the world for doing business. Such aggregate rankings make little sense, but they are ubiquitous because they are clickbait. The Doing Business aggregate ranking was no exception. Without it, the report would never have received so much attention. But without the ranking, it is doubtful anyone would have cared enough to try to manipulate the data.”
   And of course the inevitable followed: the rankings became more important than the report itself.
   
3. A rare point of disagreement. Here is the quote from his column: “This newspaper recently celebrated the demise of the Doing Business indicators, complaining that countries were “expressly changing policies to score better”. That is a strange objection. Unless the indicators are valueless, when countries try to score better that is a feature, not a bug.”
   
   When Tim Harford says “this newspaper”, he is referring to the Financial Times, where he happens to be a columnist. I’m unable to access the original FT article from where this point was excerpted, but I happen to agree with excerpt above, and therefore disagree with Tim Harford. That being said, I certainly do wish that the original FT article had been worded better in the case of the sentence that we’re able to read.
   Think about that phrase up above: ““expressly changing policies to score better”.
   I think what they wanted to say was this: countries should ideally have been trying to figure out how to change policies so that in reality, on the ground, it became easier to do business. This should then have been reflected in the rankings. That would have been Utopian. Instead, policymakers and politicians in some cases tried to change the policies so that the ranking improved, without there being much change on the ground. That word, “expressly”, is doing a lot of lifting in that phrase – because all of what I have written is what I think they were trying to get at.
   Put another way, the indicators are not valueless, unless they’ve become the target. And that, really, is all that the FT was trying to say: the indicators did, in fact, become the target. Countries were more focused on the outcome (the ranking) rather than the process (has it actually become easier to do business?), and that is never a good idea.
   
4. Consider this quote: “The Doing Business aggregate ranking was no exception. Without it, the report would never have received so much attention. But without the ranking, it is doubtful anyone would have cared enough to try to manipulate the data.”
   
   It is a question we should all be asking ourselves repeatedly: what are you optimizing for?
   In this case, was the World Bank optimizing for drawing attention to the report? We live in a world in which signaling matters, Goodhart’s Law is real and status is the name of the game. So if the World Bank was optimizing for publicity, it should have acknowledged that all of what eventually happened was a very real risk.
   But if the World Bank was optimizing for preparing a good report that stood up to scrutiny, then it should have acknowledged that the opportunity cost of such a strategy is that hardly anybody would ever read it. But such, alas, is life.

---