Some thoughts on forecasting

Shashank Patil, a BSc student writes in with this query:

“Could you suggest books(those criminally thick ones work well too!) or any other reference to understand the nuances of forecasting better? (particularly how to be skeptical about specific models, their shortcomings or what thought process should follow whenever I see a forecast model and its predictions, etc.)
I guess a lot of this should come with experience rather than through purposeful effort. But any guidance on this should be of great help.”

First things first: I wish I had had the wisdom to ask this question at that age. I and a friend of mine were just discovering the joys of playing around with Microsoft Excel and MATLAB, and were more focused on learning how to code and model than on asking “Well, hang on. Does this even make sense?” Kudos, Shashank, for being sceptical. It’ll serve you well while learning econometrics!

Now, that being said, I’ll get to books and resources a little further below, but first some thoughts about forecasting that might help.

There are, to my mind, three ways to forecast something.

The first is to build a model in which the outcome is a function of measurable inputs, excluding time. What that means in non-academic gobbledygook is this:

 

That’s a model, with measurable inputs. If x, then y, and if y, then z. And you can keep this going for as long as you want. You can guess, with some allowances for error, what’ll happen at the end. Raise interest rates, and people will borrow less. If people will borrow less, people will spend less. If people will spend less, demand will go down. And on and on and on.

Economic models are more complicated, because they deal with us, human beings. And much as we economists would like human beings to be rational, we don’t always live up to our expectations. But that apart, this is one way to forecast. Build a model, which is basically a scaled down version of reality, and hope that the model can “predict” what’ll happen next.

Or, and this is where we enter the badlands of econometrics, we can do time series modeling. Time series modeling is special in the sense that we try and predict what happens next on the basis of what has gone before.

 

 

Times series chart example from Russia
Click here for original chart and article

What will the value be in April 2000? A time series model will try and “guess” the value, based on past trends and values. The reason I tend to be a little (well, ok, more than a little) sceptical of this kind of analysis is because a) we ignore everything else that is going on in the world and b) absence of evidence is not evidence of absence.

That is to say, just because it has not happened in the past is no reason to believe that it will not happen in the future. But time series models, by definition, project out into the future by looking at the past!

And finally, betting markets! Crowdsource what the future will look like, by asking people to bet on their view of what the future will be like.

 

Here’s the introduction from a Wikipedia article (but do read the whole thing)

“Prediction markets (also known as betting markets, political betting markets, predictive markets, information markets, decision markets, idea futures, event derivatives, or virtual markets) are exchange-traded markets created for the purpose of trading the outcome of events. The market prices can indicate what the crowd thinks the probability of the event is. A prediction market contract trades between 0 and 100%. It is a binary option that will expire at the price of 0 or 100%. Prediction markets can be thought of as belonging to the more general concept of crowdsourcing which is specially designed to aggregate information on particular topics of interest. The main purposes of prediction markets are eliciting aggregating beliefs over an unknown future outcome. Traders with different beliefs trade on contracts whose payoffs are related to the unknown future outcome and the market prices of the contracts are considered as the aggregated belief.”

Also read Bryan Caplan on betting. And Robin Hanson. And Vitalik Buterin.

Now all that being said, here are the books I would recommend you read:

  1. Walter Enders: Applied Econometric Time Series. A little advanced, perhaps, but it remains, for me, the bible of time series forecasting.
  2. A Little Book of R for Time Series Forecasting. Short lovely read, with lots of examples you can try for yourself in R.
  3. Superforecasting (somewhat tangential, but a great read)
  4. Mastering Econometrics, an online video series, by Joshua Angrist (who also has a lovely book called Mostly Harmless Econometrics).

Thank you for the question, Shashank!

EC101: Links for 17th October, 2019

  1. “In order to combat global poverty, we must identify the most effective forms of action. This year’s Laureates have shown how the problem of global poverty can be tackled by breaking it down into a number of smaller – but more precise – questions at individual or group levels. They then answer each of these using a specially designed field experiment. Over just twenty years, this approach has
    completely reshaped research in the field known as development economics. This new research is now delivering a steady flow of concrete results, helping to alleviate the problems of global poverty.”
    ..
    ..
    A simple primer on the work that Duflo, Benerjee and Kremer have won the Nobel Prize for.
    ..
    ..
  2. “The first general comment is the idea of randomisation is hardly anything new for researchers who have studied or followed Indian development. The Planning Commission started something called Programme Evaluation Studies way back in 1954 which more or less studied the same thing. Agriculturists — both practitioners and researchers — have also used similar techniques of RCT to see what agricultural intervention worked.In my own research on banking history, I saw how Syndicate Bank started programmes on agricultural and rural development based on near similar ideas of randomisation. To be fair, the 2019 laureates have advanced these ideas using techniques from sampling, statistics, and econometrics to draw finer inferences.”
    ..
    ..
    Amol Agarwal over at Moneycontrol points out a more nuanced understanding of both this year’s Nobel Prize as well as the Nobel Prize for Economics in general. Well worth reading!
    ..
    ..
  3. The NYT profile on this year’s Nobel Prize.
    ..
    ..
  4. “The significance of what Angrist and Pischke termed the “credibility revolution in empirical economics” can be seen in the John Bates Clark Medal awards given to researchers who participated in that revolution. Between 1995 and 2015, of the fourteen Clark Medal winners, by my estimate at least seven (Card, Levitt, Duflo, Finkelstein, Chetty, Gentzkow, and Fryer) are known for their empirical work using research designs intended to avoid the problems that Leamer highlighted with the multiple-regression approach.”
    ..
    ..
    Mostly for those truly interested in economics, but Arnold Kling points out how more people should know about Ed Leamer.
    ..
    ..
  5. Heavily, heavily recommended: this is the longer version of the first link above, again by the Nobel Prize committee itself.