About This Measurement Business

(C) GDP figures are “man-made” and therefore unreliable, Li said. When evaluating Liaoning’s economy, he focuses on three figures: 1) electricity consumption, which was up 10 percent in Liaoning last year; 2) volume of rail cargo, which is fairly accurate because fees are charged for each unit of weight; and 3) amount of loans disbursed, which also tends to be accurate given the interest fees charged. By looking at
these three figures, Li said he can measure with relative accuracy the speed of economic growth. All other figures, especially GDP statistics, are “for reference only,” he said smiling.


This is an excerpt from the Wikileaks archive, and people familiar with modern economic history will know it all too well. This is, of course, the famous Li Keqiang index. If you prefer, you can read the original Economist article about it, although for once, the trademark Economist pun in the headline falls short of their typically high quality.

GDP measurements have always been tricky, and reading about GDP – it’s evolution, the data collection, the computation and the hajjar problems that arise from there – should be mandatory for any student aspiring to learn economics. Here’s a post from six years ago about some sources, if you’re interested.

But back to that excerpt above. What Li Keqiang was saying was that GDP statistics in China would often give a misleading picture, and he preferred to reach his own conclusions on the basis of other economic data. His preferred metrics were the ones mentioned in the abstract above: electricity consumption, volume of rail cargo and loans disbursed. Think of it this way: he’s really asking three questions. Is stuff being produced? Is stuff being moved around? Is stuff being purchased?

But what about covid times? Do these measures stand up, or do we need new proxies for GDP?

The variant’s speed also means that China’s economic prospects are unusually hard to track. A lot can happen in the time between a data point’s release and its reference period. The most recent hard numbers on China’s economy refer to the two months of January and February. Those (surprisingly good) figures already look dated, even quaint. For much of that period, there was no war in Europe. And new covid-19 cases in mainland China averaged fewer than 200 per day, compared with the 13,267 infections reported on April 4th. Relying on these official economic figures is like using a rear-view mirror to steer through a chicane.
For a more timely take on China’s fast-deteriorating economy, some analysts are turning to less conventional indicators. For example, Baidu, a popular search engine and mapping tool, provides a daily mobility index, based on tracking the movement of smartphones. Over the seven days to April 3rd, this index was more than 48% below its level a year ago.


But as the article goes on to say, this metric will tell you about movement across cities. But metro traffic gives you an idea of intra-city mobility, as do courier company express deliveries (and we did some very similar exercises in India during the lockdowns, of course. Here’s one example for Pune district.)

But the point isn’t just to come up with what else might be useful as GDP proxies. A follow-up question becomes equally important: do the GDP statistics make sense? As the Economist articles says, good numbers for metrics such as investment in fixed assets are hard to square with declines in steel output. The article contains many other such examples, and what you should take away as a student is your ability to develop a “smell” test for a given economy. Don’t take the reported numbers at face value, but “see” if they seem to be in line with other statistics about that economy.

I really like this article as an introduction to this topic because it also hints at how statisticians need to be especially careful about comparing data over time. Weekly declines might happen because of festivals, bad weather or a thousand other things, which may of course be going on along with pandemic induced lockdowns. Teasing out the effects of just one aspect isn’t an easy thing to do.

And finally, think about how you can apply this lesson in other domains! Should an interviewer look only at marks, or try and figure out other correlates. Or, as Mr. Keqiang puts it, are marks “for reference only”? What about quarterly earnings reports? Press releases? Smell tests matter, and the earlier you start developing them, the better you get at detecting, and calling bullshit.

And finally, the concluding paragraph from the article we’ve discussed today:

To help avoid some of the traps lurking in these unconventional indicators, Mr Lu and his team watch “a bunch of numbers, instead of just one”. In a recent report he highlighted 20 indicators, ranging from asphalt production to movie-ticket sales. “If seven or eight out of ten indicators are worsening, then we can be confident that GDP growth is getting worse,” he says. Right now, he thinks, the direction is clear. “Something must be going very wrong.”



Leave a Reply