Tag: forecasting

Future 1, Economists 0

Trying to predict the future is a fool’s errand, but ours happens to be a field that will just not learn. Or that, at any rate, is the takeaway from an entertaining little read by The Economist:

Spare a thought for economists. Last Christmas they were an unusually pessimistic lot: the growth they expected in America over the next calendar year was the fourth-lowest in 55 years of fourth-quarter surveys. Many expected recession; The Economist added to the prognostications of doom and gloom. This year economists must swap figgy pudding for humble pie, because America has probably grown by an above-trend 3%—about the same as in boomy 2005. Adding to the impression of befuddlement, most analysts were caught out on December 13th by a doveish turn by the Federal Reserve, which sent them scrambling to rewrite their outlooks for the new year.

https://www.economist.com/leaders/2023/12/20/economists-had-a-dreadful-2023

And it’s not just macroeconomic forecasts. The article goes on to point out how we economists (maybe) botched up measurements of inequality, costs of restricting construction, measurement of social mobility over time, the impact of joblessness, and findings in behavioral economics.

Now this might seem like a rather snarky way to begin blogging in the new year (not to mention resuming work after a lengthy hiatus), but the reason I chose to do so was because of one key sentence in The Economist article:

“Another lesson is that disdain for economic theory in favour of the supposed realism of empirical studies may have gone too far.”

Maybe this particular blogger is getting a little too old and grumpy for his own good – and that is a possibility that can never be fully discounted! – but I happen to be in deep agreement with the idea that empirical studies have been taken too far. Good ol’ fashioned economic theorizing is more underrated than ever, and we would do well to get it back in the limelight.

Too true by half, if you ask me, and worth focusing upon. Also, as the article points out, let’s focus on the silver lining – yes, economics got a lot wrong last year, but on the other hand, we were wrong in all the right ways. Turns out we dodged a recession (well, so far, at any rate), we were wrong (probably) about the rise in inequality and we needn’t despair quite as much about the world as we thought we should.

That may not be the ringing endorsement our field needs, but it’s what we’ve got. Enough to work on, I’d say – let’s get on with it.

A Sunny Outlook

We’ve made remarkable progress since the days of Admiral FitzRoy. Predicting the weather is still, admittedly, a very difficult and very expensive thing, as this lovely little write-up makes clear, but it is also something we’re much better at these days. We have better instruments, better computing power, better mathematical and statistical tools to deploy, and the ability to synthesize all of these to come up with much better forecasts – but it’s not perfect, and it’s not, well, good enough.

Those last two words aren’t meant as a criticism or a slight – far from it. The meteorologists themselves feel that is is not good enough

Some years ago, I wrote a chapter in a book called Farming Futures. The book is about social entrepreneurship in India, and my chapter was about a firm called Skymet. Skymet is a private weather forecasting firm based partially out of Pune and partially out of Noida (along with other office in other locations). But researching for the chapter got me interested in both how the art and science of weather forecasting had developed over time, and where it is headed next.

Only trivia enthusiasts are likely to remember the name of the captain on whose ship Charles Darwin made his historic voyage that was to result in the publication of “On the Origin of Species”. Fewer still will remember that Admiral Robert FitzRoy committed suicide. The true tragedy, however, is that it is almost certainly his lifelong dedication to predicting the weather that caused him to take his own life.
We have, in the decades and centuries since, come a long way. Weather forecasting today is far more advanced than it was in Admiral FitzRoy’s day. Britain, for example, Admiral FitzRoy’s own nation, today has an annual budget of more than 80 million GBP to run its meteorological department. It has an accuracy of around 95% when it comes to forecasting temperatures, and an accuracy of around 75% when it comes to forecasting rain – anybody who is even remotely familiar with Britain’s notoriously fickle weather would know that this is no small achievement.

Farming Futures: Emerging Social Enterprises in India

Those numbers that I cited, and the tragic story of Admiral FitzRoy, come from a lovely book called The Weather Experiment.

But I first read about weather, and the difficulties associated with forecasting it in a book called Chaos, by James Gleick:

Lorenz enjoyed weather—by no means a prerequisite for a research meteorologist. He savored its changeability. He appreciated the patterns that come and go in the atmosphere, families of eddies and cyclones, always obeying mathematical rules, yet never repeating themselves. When he looked at clouds, he thought he saw a kind of structure in them. Once he had feared that studying the science of weather would be like prying a jack-in–the-box apart with a screwdriver. Now he wondered whether science would
be able to penetrate the magic at all. Weather had a flavor that could not be expressed by talking about averages. The daily high temperature in Cambridge, Massachusetts, averages 75 degrees in June. The number of rainy days in Riyadh, Saudi Arabia, averages ten a year. Those were statistics. The essence was the way patterns in the atmosphere changed over time…

Ch. 1, The Butterfly Effect, Chaos, by James Gleick

What is the Butterfly Effect, you ask? It gets its own Wikipedia article, have fun reading it.

All of which is a very long way to get around to the write-up we’re going to be talking about today, called After The Storm.

On 29 October 1999, a “Super Cyclone” called Paradip devastated parts of Odisha and the east coast of India. At wind speeds of almost 250 kms per hour, it ravaged through the land, clearing out everything in its path. Fields were left barren, trees uprooted like mere matchsticks, entire towns devastated. More than 10,000 people lost their lives.
Fast forward to two decades later. In 2020, bang in the middle of the Covid-19 pandemic, another cyclone—known as Amphan—speeds through the Bay of Bengal. It crashes into the land like Paradip did in 1999. Like before, many homes are destroyed and structures uprooted. But one thing is different: this time’s death toll is 98. That’s a 100 times lower than 1999’s casualties.
What made this difference possible? Simply put: better, timely and more accurate weather prediction.

https://fiftytwo.in/paradigm-shift/after-the-storm/

We’ve made remarkable progress since the days of Admiral FitzRoy. Predicting the weather is still, admittedly, a very difficult and very expensive thing, as this lovely little write-up makes clear, but it is also something we’re much better at these days. We have better instruments, better computing power, better mathematical and statistical tools to deploy, and the ability to synthesize all of these to come up with much better forecasts – but it’s not perfect, and it’s not, well, good enough.

Those last two words aren’t meant as a criticism or a slight – far from it. The meteorologists themselves feel that is is not good enough:

“It almost becomes like flipping a coin,” Professor Islam says. “The IMD is not to be blamed. They will be very good at predicting the weather three or four days in advance. Beyond that, it cannot be done because there is a fundamental mathematical limitation to these questions.”
“IMD can do another sensor, another satellite, they can maybe improve predictions from two days, to three days. But can they do ten days? There is no evidence. Right now there is no weather forecasting model on the globe. India to Europe to Australia, it doesn’t matter, it’s not there.”

https://fiftytwo.in/paradigm-shift/after-the-storm/

As Professor Islam says, he wants to move from up from being able to forecast the next four to five days, to being able to predict weather over the next ten days. Why? So that communities in the path of a storm have adequate time to move. What could be more important than that when it comes to meteorology.

So what’s the constraint? This is a lovely analogy:

“I give this example to my students,” the professor says, “Look, usually all of science and AI is based on this idea of driving with the rearview mirror. I don’t have an option, so I’m looking into my rearview mirror and driving. I will be fine as long as the road in the front exactly mirrors the rearview. If it doesn’t and I go into a turn? Disastrous accident.”

https://fiftytwo.in/paradigm-shift/after-the-storm/

It’s weird what the human brain will choose to remind you of, but this reminds me, of all things, of a gorilla. That too, a gorilla from a science fiction book:

Amy distinguished past, present, and future—she remembered previous events, and anticipated future promises—but the Project Amy staff had never succeeded in teaching her exact differentiations. She did not, for example, distinguish yesterday from the day before. Whether this reflected a failing in teaching methods or an innate feature of Amy’s conceptual world was an open question. (There was evidence for a conceptual difference.) Amy was particularly perplexed by spatial metaphors for time, such as “that’s behind us” or “that’s coming up.” Her trainers conceived of the past as behind them and the future ahead. But Amy’s behavior seemed to indicate that she conceived of the past as in front of her—because she could see it—and the future behind her— because it was still invisible.

Michael Crichton, Congo

That makes a lot of sense, doesn’t it? And that’s the fundamental problem with any forecasting tool: it necessarily has to be based on what happened in the past, because what else have we got to work with?

And if, as Professor Islam says, the road in the future isn’t exactly like the past, disaster lies ahead.

But Artificial Intelligence and Machine Learning need not be about predicting what forms the storms of the future might take. They can be of help in other ways too!

“It hit us that the damage that happened to the buildings in the poorer communities could have been anticipated very precisely at each building’s level,” Sharma explains. “We could have told in advance which roofs would fly away, and which walls would collapse, which not so. So that’s something we’ve tried to bring into the AI model, so that it can be a predictive model.”

“What we do is, essentially, this: we use satellite imagery or drone imagery and through that, we identify buildings. We identify the material and technology of the building through their roofs as a proxy, and then we simulate a sort of a risk assessment of that particular building, right? We also take the neighbouring context into account. Water bodies, how high or low the land is, what kind of trees are around it, what other buildings are around it.”

The team at SEEDS and many others like it are more concerned about the micro-impact that weather events will have. Sharma is interested in the specifics of how long a building made from a certain material will be able to withstand the force of a cyclone. This is an advanced level of interpretation we’re talking about. It’s creative, important and life-saving as well.

https://fiftytwo.in/paradigm-shift/after-the-storm/

In other words, we may not know the intensity of a particular storm, and exactly when and where it will hit. But given assumptions of the intensity of a storm, can we predict which buildings will be able to withstand a given storm and which ones won’t?

This is, as a friend of mine to whom I forwarded this little snippet said, is very cool.

I agree. Very cool indeed.

And sure, accuracy about weather forecasting may still be a ways away, and may perhaps lie forever beyond our abilities. But science, mathematics and statistics might still be able to help us in other ways, and that (to me) still counts as progress.

And that is why, all things considered, I’d say that when it comes to the future of weather forecasting, sunny days are ahead.

In case you haven’t already, please do subscribe to fiftytwo.in

Excellent, excellent stories, and the one I have covered today is also available in podcast form, narrated by Harsha Bhogle, no less. All their other stories are wroth reading too, and I hope you have as much fun going through them as I have.

Forecasting The Future

All forecasting models are fun to learn about, and to tinker with in your software of choice. But it is equally true that all forecasting models are problematic.

First, they’re based on the assumption that the future will look like the past. Eventually, that will not be the case – this is a guarantee.

Second, even if they are based on the past, there is the problem of survivorship bias to consider in your sample of choice (my thanks to Aadisht for helping me realize this better).

And third, your predictions cannot – I repeat, cannot – account for all the underlying complexities. Forecasting is a ridiculously risky thing to do, and kudos to those who try, for this very reason.

I’d done a round-up of posts I had read in January 2020 (remember January 2020? Those were the days) that tried to predict what the world would look like when it came to India, technology and the world. I bring this up to re-emphasize the point I was trying to make in the previous paragraph: no matter how sophisticated your model, no matter how careful your sampling, and no matter however many dots you connect: reality will always have you beat.

That’s just how it is. Forecasting models work well until they don’t, and that one time they don’t can often be more costly than all the times they did.

And that brings me to this tweet:

What should you take away from this tweet (and the rest of the thread)?

My primary audience when I write here is, in a sense, myself back when I was an undergrad/post-grad student. So what advice would I want to give to myself after having read that Twitter thread?

  1. As Nitin Pai himself goes on to say in a subsequent tweet, this is a useful principle to have: Don’t try to predict the future.
  2. Respect skin in the game. Did he get it wrong? Sure he did. But hey, it takes courage to put your reasoning, your thoughts and your conclusions in the public domain. Feel free to disagree with the conclusions, but accord people who write in public the respect they deserve for having done so.
  3. Have the courage to admit you were wrong. We have two examples in front of us. One is the usual “I was misquoted/misunderstood” weasel talk. The other is an admission of error, straight up, and without qualifiers. Like the tweet above.
  4. Work at getting better. A publicly available record of your thoughts is invaluable, because it forces you to write after thinking carefully. It is also invaluable because you can outsource the “where can I get better” to the internet. And there are enough (trust me) people on the internet who will enthusiastically point out where you’re wrong. Use that advice constructively. By that I mean this, specifically: continue to write in the public domain, and that will mean making mistakes. Try not to make the same ones twice.

Like Nitin, I have written about what we’ve been going through, and how we might get out of it. All of it is available here on this blog. Some of it might turn out to be wrong – in fact, there’s a guarantee that if I write enough, some of it will be wrong. And given the pandemic that we’re going through, the stakes are impossibly high.

But it is the process of writing in public, and giving feedback on what other people write in public that drives our thinking forward.

So again, if you’re a student reading this: write. Write in the public domain. Make mistakes. Develop a thick enough skin to take on the criticism. Learn the (almost impossible to acquire) skill of figuring out when you’re wrong, and develop and hone the courage it takes to admit it.

And then, write again.

(Quick note: posting will be sporadic for some time.)