Forecasting The Future

All forecasting models are fun to learn about, and to tinker with in your software of choice. But it is equally true that all forecasting models are problematic.

First, they’re based on the assumption that the future will look like the past. Eventually, that will not be the case – this is a guarantee.

Second, even if they are based on the past, there is the problem of survivorship bias to consider in your sample of choice (my thanks to Aadisht for helping me realize this better).

And third, your predictions cannot – I repeat, cannot – account for all the underlying complexities. Forecasting is a ridiculously risky thing to do, and kudos to those who try, for this very reason.

I’d done a round-up of posts I had read in January 2020 (remember January 2020? Those were the days) that tried to predict what the world would look like when it came to India, technology and the world. I bring this up to re-emphasize the point I was trying to make in the previous paragraph: no matter how sophisticated your model, no matter how careful your sampling, and no matter however many dots you connect: reality will always have you beat.

That’s just how it is. Forecasting models work well until they don’t, and that one time they don’t can often be more costly than all the times they did.


And that brings me to this tweet:


What should you take away from this tweet (and the rest of the thread)?

My primary audience when I write here is, in a sense, myself back when I was an undergrad/post-grad student. So what advice would I want to give to myself after having read that Twitter thread?

  1. As Nitin Pai himself goes on to say in a subsequent tweet, this is a useful principle to have: Don’t try to predict the future.
  2. Respect skin in the game. Did he get it wrong? Sure he did. But hey, it takes courage to put your reasoning, your thoughts and your conclusions in the public domain. Feel free to disagree with the conclusions, but accord people who write in public the respect they deserve for having done so.
  3. Have the courage to admit you were wrong. We have two examples in front of us. One is the usual “I was misquoted/misunderstood” weasel talk. The other is an admission of error, straight up, and without qualifiers. Like the tweet above.
  4. Work at getting better. A publicly available record of your thoughts is invaluable, because it forces you to write after thinking carefully. It is also invaluable because you can outsource the “where can I get better” to the internet. And there are enough (trust me) people on the internet who will enthusiastically point out where you’re wrong. Use that advice constructively. By that I mean this, specifically: continue to write in the public domain, and that will mean making mistakes. Try not to make the same ones twice.

Like Nitin, I have written about what we’ve been going through, and how we might get out of it. All of it is available here on this blog. Some of it might turn out to be wrong – in fact, there’s a guarantee that if I write enough, some of it will be wrong. And given the pandemic that we’re going through, the stakes are impossibly high.

But it is the process of writing in public, and giving feedback on what other people write in public that drives our thinking forward.

So again, if you’re a student reading this: write. Write in the public domain. Make mistakes. Develop a thick enough skin to take on the criticism. Learn the (almost impossible to acquire) skill of figuring out when you’re wrong, and develop and hone the courage it takes to admit it.

And then, write again.


(Quick note: posting will be sporadic for some time.)