Calling Bullshit: An Appreciation

This past Tuesday, I went on a long rant about exams in general, and exams especially in the year 2020. That rant was inspired by a Twitter thread put out by Prof. Carl Bergstrom.

Now, if you happen to share my views on examinations, I’m guessing you were already likely to be a fan of Prof. Bergstrom. Today, your fandom might just go up a couple of notches. Check out the first paragraph on my favorite discovery of 2020 so far – Calling Bullshit:

The world is awash in bullshit. Politicians are unconstrained by facts. Science is conducted by press release. Higher education rewards bullshit over analytic thought. Startup culture elevates bullshit to high art. Advertisers wink conspiratorially and invite us to join them in seeing through all the bullshit — and take advantage of our lowered guard to bombard us with bullshit of the second order. The majority of administrative activity, whether in private business or the public sphere, seems to be little more than a sophisticated exercise in the combinatorial reassembly of bullshit.

https://www.callingbullshit.org/

He and his collaborator on the project, Prof. Jevin West, are nothing if not thorough:

What do we mean, exactly, by bullshit and calling bullshit? As a first approximation:

Bullshit involves language, statistical figures, data graphics, and other forms of presentation intended to persuade by impressing and overwhelming a reader or listener, with a blatant disregard for truth and logical coherence.

Calling bullshit is a performative utterance, a speech act in which one publicly repudiates something objectionable. The scope of targets is broader than bullshit alone. You can call bullshit on bullshit, but you can also call bullshit on lies, treachery, trickery, or injustice.

In this course we will teach you how to spot the former and effectively perform the latter.

https://www.callingbullshit.org/

There’s a book, there’s videos of the course lectures (yes, you can earn credits for learning about bullshit), there’s a list of heuristics about detecting bullshit when it comes to interpreting visualizations, reading academic papers, and facial detection algorithms. There are case studies too!

And hey, if you insist on being politically correct (there’s merit in the argument that you shouldn’t, but hey, entirely your call) – well, they got you covered:

If you feel that the term bullsh!t is an impediment to your use of the website, we have developed a “sanitized” version of the site at callingbull.org. There we use the term “bull” instead of “bullsh!t” and avoid other profanity. Be aware, however, that some of the links go to papers that use the word bullsh*t or worse.

https://www.callingbullshit.org/FAQ.html

Some weeks ago, I promised somebody that I would come up with a lecture on demystifying statistics – and set myself the challenge of trying to come up with lecture notes without using a single equation.

As is the case with 95% of the things I really want to do, I promptly forgot all about it.

I haven’t seen all the videos yet on Calling Bullshit, but it does seem as if outsourcing this exercise – at least in part – to this fantastic website would be a really good idea.

Check out the syllabus here. A part of me is tempted to say that I would like to run this module as a summer school at GIPE, but you will remember what I said about things I really want to do.

But hey, there’s always hope, right?

Or should I be calling bullshit on myself?

Five Links About – Well, What Else?

It doesn’t matter whether you support Trump, Biden – or even Kanye. It doesn’t matter whether you read this at 10 in the morning on the 4th of November 2020, which is when I’ll be scheduling this post, or much later (and that could be hours, days, weeks or months later). I’ve tried to collate five sources that will give you the long view of whatever might happen on this day. With that in mind, here we go:

Ezra Klein speaks about the American divide, and posits that it isn’t about Republicans v Democrats (and read the whole excerpt, and then the whole book!):

Over the past decade, the dreams of democratic theorists everywhere actually came true. The internet made information abundant. The rise of online news gave Americans access to more information — vastly more information, orders of magnitude more information — than they had ever had before. And yet surveys showed we weren’t, on average, any more politically informed. Nor were we any more involved: Voter participation didn’t show a boost from the democratization of political information. Why?

…

But among those with cable and internet access, the difference in political knowledge between those with the highest and lowest interest in cable news was 27 percent. That dwarfed the difference in political knowledge between people with the highest and lowest levels of schooling. “In a high-choice environment, people’s content preferences become better predictors of political learning than even their level of education,” Prior wrote.

…

Misperceptions were particularly high when people were asked to describe the other party. Democrats believed 44 percent of Republicans earned more than $250,000 a year; it’s actually 2 percent. Republicans believed that 38 percent of Democrats were gay, lesbian, or bisexual; the correct answer is about 6 percent. Democrats believed that more than four in 10 Republicans are seniors; in truth, seniors make up about 20 percent of the GOP. Republicans believed that 46 percent of Democrats are black and 44 percent belong to a union; in reality, about 24 percent of Democrats are black and less than 11 percent belong to a union.

…

Here’s the kicker: As the charts below show, the more political media people consumed, the more mistaken they were, in general, about the other party. This is a damning result: The more political media you absorb, the more warped your perspective of the other side becomes.

https://www.vox.com/2020/1/28/21077888/why-were-polarized-media-book-ezra-news

… while Bruno Macaes hypothesizes that the split is between fiction and reality (and interpret that any way you will)

The main binary in American politics is not between left and right, but between fiction and reality. One experiences particular fictions, but at some point they must be revealed as no more than fictions. They must be switched off, in anticipation of new stories.

https://brunomacaes.substack.com/p/biden-the-kill-switch

This article is impossible to excerpt from, but deserves to be read in full, multiple times. Ross Douthat on what the right, the centre and the left learned from four years of Trump.

A worthwhile read on – no matter the outcome, whenever you read this – the Nate Silver/Taleb debate:

Because FiveThirtyEight only predicts probabilities, they do not ever take an absolute stand on an outcome: No ‘skin in the game’ as Taleb would say. This is not, however, something their readers follow suit on. In the public eye, they (FiveThirtyEight) are judged on how many events with forecasted probabilities above and below 50% happened or didn’t respectively (in a binary setting). Or, they (the readers) just pick the highest reported probability as the intended forecast. For example, they were showered with accolades when after, ‘calling 49 of 50 states in the 2008 presidential race correctly’ Nate Silver was placed on Times 100 most influential people list. He should not have accepted the honor if he didn’t call a winner in any of the states!

https://towardsdatascience.com/why-you-should-care-about-the-nate-silver-vs-nassim-taleb-twitter-war-a581dce1f5fc

And hey, take the long view!

In 44 chronological episodes, the “Presidential” podcast takes listeners on an epic historical journey through the personality and legacy of each of the American presidents. Created and hosted by Washington Post reporter Lillian Cunningham, “Presidential” features interviews with the country’s greatest experts on the presidency, including Pulitzer Prize-winning biographers Doris Kearns Goodwin, David McCullough, Jon Meacham and Bob Woodward. Start listening at the very beginning, with the life of George Washington, or jump ahead to any president whose story you want to better understand.

https://www.washingtonpost.com/graphics/business/podcasts/presidential/

On Conducting Examinations, Especially in 2020

This is less a blogpost, and more of a rant. Consider yourself warned! 🙂

This past Saturday, I cam across a most excellent Twitter thread:

https://twitter.com/CT_Bergstrom/status/1322369355930165248

The author, Carl Bergstrom (Wikipedia article here, University profile here) makes a detailed, reasoned argument against online proctoring of examinations, especially in 2020. In my blogpost, I’m going to riff on this Twitter thread, and some related points, and build an argument against the way we conduct examinations in Indian Universities.

To begin, watch this Sugata Mitra TED talk about education:

Just the first five minutes or so is enough for the argument I will be making today, but you really should watch the full thing. There was, as it turns out, a case for rote memorization at one point of time. But today, as Mitra says in the video, it is the computers that are the clerks. They do (and to be fair to them, they do it much better than we ever could) the job of remembering everything, so that we don’t have to.

To every single professor reading this blogpost: when was the last time you yourself spent a day working without looking up something on the internet? When was the last time you researched something, wrote something, without using your computer, or the internet on some device?

Then why do we insist on examining our students for their ability to do so, when we ourselves don’t do it? And for those of our students who are not going to get into academia, they wouldn’t last in their organizations for even an hour if they tried to work without using computers and the internet.

They would (should!) be fired for being Luddites!

And yet, to land up in that firm, they must spend the last week of their lives as a college student cooped up for three hours in a classroom without a computer, without the internet, and use pen and paper to write out the important features of xyz in abc lines.

I can’t possibly be the only one that spots the incongruity, surely?

Here, from the Twitter thread I spoke about earlier, is a picture of Bloom’s taxonomy:

Image

Read the Wikipedia article about Bloom’s taxonomy, or look carefully at the picture above. At best – and I think I am being charitable here – our question papers in higher education in India reach the evaluate stage, but certainly not create. And even that is a stretch.

Moreover, even if we somehow agree that we do indeed reach the evaluate stage, we are effectively asking students to evaluate based on their memory alone. Why?

One, won’t students write a better evaluation of whatever theory we are asking them to write about if we give them the ability to research while writing? Second – and I know this is repetitive, but still – are they ever going to write an evaluation without having access to to the internet?

In plain simple English: We train students for 25 years to get awesome at memorizing stuff, and then expect them to do well in a world which doesn’t value this skill at all.

(To be clear, some things you should remember, of course. Think of it as a spectrum – and I am not suggesting that we move to the end of the spectrum where no memorization is required. I am suggesting, however, that we are at the end of the spectrum where only memorization is required. Close enough, at any rate).

Coming specifically to this year, the year of online examinations, here’s a tweet that was quoted in Bergstrom’s thread:

There really isn’t much to say, is there? All universities the world over have sent out variations of this nightmare this year, and in some cases, repeatedly. It’s the whole null hypothesis argument all over again – we assume all students to be guilty until proven otherwise. That is, we assume everybody will cheat, and therefore force everybody to comply with ridiculously onerous rules – so as to prevent the few who might actually cheat.

And cheating, of course, being looking up stuff on the internet. The argument itself is pointless, as I have explained above, and we go to eye-popping lengths to enforce the logical outcome of this pointless argument.

Prof. Bergstrom makes the same points himself in the Twitter thread, of course:

https://twitter.com/CT_Bergstrom/status/1322372971730644992
https://twitter.com/CT_Bergstrom/status/1322375318108213248
https://twitter.com/CT_Bergstrom/status/1322376239164743680

This year, especially, is a good opportunity to turn what is otherwise a disaster of a situation into meaningful reform of the way we conduct examinations.

Students, parents – indeed society at large – will spot the incongruity of learning online, but being examined offline. If we, in higher education in India do not spot this incongruity and work towards changing it – well then, we will have failed.

And finally, the last tweet in the thread is something we would all do well to remember:

https://twitter.com/CT_Bergstrom/status/1322378737464172547

Reflections on “India Moving” by Chinmay Tumbe

Every now and then, you read about courses you would really, really want to attend yourself (and keep an eye out this Thursday for another such course). Here is one such course, which I found out about as a consequence of reading the book India Moving: A History of Migration by Chinmay Tumbe:

Hitchhiker’s Guide to Business and Economies across Five Centuries

As you’d expect (indeed, demand), the two major academic requirements for the course each have a weightage of The Answer To The Ultimate Question of Life, Universe and Everything. (Where can I sign up?)

Reading the book was an enjoyable experience for two reasons. One, there is a wealth of knowledge to be gleaned from the book – Chinmay Tumbe has clearly spent years and years engaged in researching this topic, and it shows. But second, the book never becomes turgid, because sprinkled with a judiciously light hand throughout the book are little snippets of delightful humor, and you almost end up turning the page to find out when the next one will appear.

Migration is a tricky topic to study, by the way, especially in terms of causality. Shown here is a little diagram that has helped me think about the issue as systematically as possible:

Taken from Todaro and Smith’s Textbook on Development Economics

Note that while this diagram is about rural to urban migration, I often find it helpful to use the same framework to think about migration out of India. The inputs into the decision-making process remain more or less the same. The professor inside of me wants to spend some time in explaining this chart more, but I’ll resist, for two reasons. One, the length of this blogpost. Two, I honestly think it is fairly self-explanatory!

Now, the reason I bring up this chart is because while it works in the sense that you understand how to think about migration, it really needs a complementary good. That good being this book!

For example, it is one thing to talk about rural-urban migration, but quite another to learn about the origin of the sobriquet ‘maca paos‘. Or the chapter on migration from Ratnagiri (the title of which has a lovely pun too), which speaks about the psychology of children (boys and girls), and how it may have been shaped by the trends of migration.

The book, in other words, talks about theory, but also works in examples that fill the book with more than just a dreary recital of facts, figures and theories.

The book is built around one central idea, that of the Great Indian Migration Wave, which is the author’s term for a phenomenon that has lasted for well over a century, and encompasses over 200 million people. As Chinmay Tumbe puts it, this Great Indian Migration Wave was ‘responsible for the formation of cities such as Kolkata and Mumbai, and ended up producing freedom fighters, political leaders, regional cultures and culinary delights’ – and much else besides!

Having explained the central idea in the second chapter, the rest of the book is about the consequences of the Great Indian Migration Wave, including chapters on the migration of capital (my term, not his), the folks who chose to not return, the impact of partition and finally the impact of migration on development.

Each chapter is full of little, and often surprising, nuggets of information. For example, we learn that:

If Kerala were to be a country , it would rank among the five most remittance – dependent nations in the world as its remittance to GDP ratio stood at a staggering 30 per cent .

Location 926 (All locations refer to the Kindle version)

… and you might say that well, this isn’t all that surprising, given what we know of migration patterns from Kerala to the ‘Gelf‘. But then how about this?

Annual migration to the Gulf from Uttar Pradesh and Bihar now exceeds that from Kerala

Location 1001

Thirteen years after I got my passport, I finally understood what ECNR was all about!

In the 1840s , legal provisions introduced a ‘ protector of immigrants ’ at the port of disembarkation and a ‘ protector of emigrants ’ at the port of embarkation . The latter now survives as a department in India’s ministry of external affairs . As per an Act of 1844 , the protector of emigrants would ensure that ‘ no emigrant shall embark without a certificate from the Agent , countersigned by the Protector ’ leading to a curious phrase that is now imprinted in Indian passports as ‘ Emigration Check Not Required ’ .

Location 1067

1840’s! Why do we still have this on our passports?

The next chapter looks at migration based on a mixture of geography and community. We learn about the Parsis, the Marwaris, the Chettiars among others, and one revelatory point for me was how much of migration there was between India and Myanmar before the IInd World War, and how badly it turned out.

Another highlight for me was a potential explanation for why communities from Punjab, Rajasthan, Gujarat and Sindh were as exceptional as they are:

Claude Markovits , a historian of Indian business , provides one possible explanation for the exceptional selectivity of merchant migrations from the region comprising Punjab , Rajasthan , Gujarat and Sindh . His ecological argument credits the ‘ dry zone ’ for honing skills in risk management , which provide a comparative advantage in trade and finance , over regions with more reliable access to water .

Location 1695

Which, as the book goes on to say, may or may not be true, but it is certainly a point worth considering. This chapter also helped me understand why business is a ‘dirty’ word in India – and the answer may well be very, very ancient indeed!

The Arthashastra , India’s most famous treatise on statecraft compiled over 1800 years ago , mistrusted traders arguing that that they were ‘ all thieves , in effect , if not in name ’ and that ‘ they shall be prevented from oppressing the people ’. The caste system itself accorded the trading caste the third position , ritually lower than the priests and the warriors . Further , taboos on overseas travel were pervasive . In such a milieu , the trading castes were not likely to be in a position to expand their network or gain acceptance among other castes .

Location 1700

(Although I still remain confused about why taboos on overseas travel were pervasive. And I think this is a very important question, too. What is it about India (and, I think, China) that made our societies reluctant to travel far and wide?)

Elsewhere in the book, Chinmay Tumbe also points out how low acceptance of Indian migrants was as much a problem in the receiving country as well:

According to an official report in 1910 , Indians were ‘ the most undesirable of all Asiatics and the peoples of the Pacific states were unanimous in their desire for exclusion ’

Location 2165

How long does it take you to lose your Indian identity if you migrate abroad? There are two quotes in this regard that stood out to me:

Cricket itself became a litmus test of loyalty . The Tebbit test , named after Lord Tebbit , proposed that Indians could be considered to have successfully assimilated into British society only if they rooted for England during cricket matches against India . With a few exceptions like Madras – born Nasser Hussain , former captain of the England cricket team , most people of Indian origin would flunk this test even today , even though they have integrated on various other counts such as language and civic and political participation .

Location 2090

Second, Punjab is famously known in India for its agricultural might but also infamously known for its ‘ missing women ’ phenomenon . Sex ratios at birth are skewed starkly in favour of males due to a preference for sons and abortion of female foetuses . As it turns out , studies have shown that this cultural trait persists in Canada , even if it is under different material conditions . Old habits often die hard in the diaspora.

Location 2216

What I remain curious about is whether it is only ‘us’, or whether this applied to migrants from other nationalities as well. That is, are Indians unique in taking a long time to assimilate, particularly from a cultural viewpoint, or is this true for other nationalities as well. That is, of course, beyond the scope of this book, but in this regard, there was this delightful little nugget:

If there is one striking cultural practice that unites Indians across all these identities and separates India from the places they usually migrate to , it is the use of water in toilets as opposed to tissue paper abroad . Indians ’ love for water is closely matched with their love for steam generated by the pressure cooker , a wonderful gift of humankind celebrating both rice and noise , amplified many times over in silent honk – less neighbourhoods abroad . It is no wonder then that a mug and a pressure cooker are important objects packed in the luggage of a first – time visitor or settler in new lands .

Location 2415

What unites Indians abroad may well be a continuation of our innate Indian-ness, but another thing that unites us is also the reasons we choose to go abroad. For example, is it not puzzling that Bihar and Kerala have similar rates of outmigration? Chinmay Tumbe’s answer to this little puzzle is linked back to our abysmally low rates of urbanization (although of course, there is the thorny little problem of trying to figure out our actual urbanization rate)

One reason I really loved the book was because while it speaks about migration, it does so by dipping into a whole host of related factors, such as caste:

According to one nasty upper – caste parody , ‘ untouchable ’ sweepers did not care about the violent chaos surrounding them during Partition , as nobody was going to ‘ touch ’ them in any case .

Location 2620

…. and gender:

There is , however , an alternative brain drain that is rarely talked about . This refers to the migration of high – skilled spouses , almost always women , who accompany the high – skilled migrant workers , but do not end up working for remuneration due to restrictions placed by families or visa regulations . Within India , this leads to a colossal under – utilization of talent as millions of female graduates forgo active professional careers upon their move to a new state or city , to look after children or the family . Outside India , visa restrictions on work for dependents can kill aspirations and dull the brains . In the USA , over a 1,00,000 Indian dependents live on the less – known H4 visa , known as the depression visa . Studies have shown how this brings about a loss of self – confidence and discomfort because of financial dependence on the spouse , even for remittances , and overall , retards professional careers due to the erosion of skills . In such cases , the American sitcom Desperate Housewives offers only partial relief to the brain pain.

Location 3204

…besides, of course, poverty, rural-urban divides, community and ethnicity, which we have touched upon earlier.

My ‘standard’ papers to recommend to students to read about migration are Theories of International Migration: A Review and Appraisal, by Massey et al and The Costs and Returns of Human Migration, by Sjaastad. (And I can’t resist adding that you really should read We Wanted Workers, by Borjas and Open Borders: The Science and Ethics of Immigration, by Caplan. Two diametrically opposite views about immigration, but that is exactly why you should read both.) Here’s an older EFE post about immigration, with a set of five links (the fifth of which is this very book).

In short, an excellent book, and heavily recommended!

Two further notes:

  1. There is an excellent set of links about migration on Chinmay Tumbe’s homepage. For those of you who want to go down a seemingly endless rabbit-hole, well, dive in!
  2. He has got a new book coming out in early December, called The Age of Pandemics. I have (of course!) pre-ordered it, and hope to write a review as soon as I get a chance to read it.

Video for 1st Nov, 2020: The Important before the Urgent

Tweets for the week ending 31st October, 2020

Ok…

Links for Friday, 30th October, 2020

On the Foxconn, um, factory in Wisconsin:

Such announcements are far from unusual for Gou, and often, nothing comes of them. In Vietnam in 2007, in Brazil in 2011, in Pennsylvania in 2013, and in Indonesia in 2014, Foxconn announced enormous factories that either fell far short of promises or never appeared. Just this year, the industries minister of Maharashtra, India, which aggressively pursued one of Gou’s multibillion-dollar projects in 2015, finally confirmed the factory isn’t coming, saying the state had learned a lesson about believing businesses promising big investments.
In China, where Foxconn employs the vast majority of its million workers, these sorts of announcements are called “state visit projects,” according to Willy Shih, a Harvard business school professor and former display industry consultant. Officials get a ribbon-cutting photo op, the company gets political goodwill, and everyone understands that the details of the contract are just an opening bid by a company that will ultimately do whatever makes economic sense.

https://www.theverge.com/21507966/foxconn-empty-factories-wisconsin-jobs-loophole-trump

I wish I could explain statistics as clearly as this:

Let’s say we have 100 people who have volunteered for the trials. We’ve divided them into two groups of 50 each. One will be administered the experimental drug, the other a placebo — i.e. something that looks identical, but has no medicinal value at all. There are rules for administering a placebo correctly, and I’ll come to those. For now, let’s assume they have been followed.
The trial runs its course. The placebo group reports that one person has recovered, whereas the group that got the actual drug reports that five have recovered. What, if anything, can we conclude? Is this just chance? Is there a real difference between the groups? Is this enough to conclude anything about the efficacy of the drug?

https://www.livemint.com/opinion/columns/opinion-significance-of-double-blind-drug-trials-11602211204718.html

Old men, friendships, and chimpanzees.

As they got older, the chimps developed more mutual friendships and fewer one-sided friendships. They also exhibited a more positive approach to their whole community, continuing grooming of other chimps, including those that weren’t close friends, at the same rate, but with a drop in aggression. Other primates don’t necessarily follow this pattern as they grow older, according to the authors. Some monkeys tend to withdraw from social relationships and their aggression levels stay high.

https://www.nytimes.com/2020/10/22/science/aging-chimps-friendship.html

Krish Ashok writes a passionate (and erudite, but that’s a given, no?) defence of… maida.

Maida is technically more all-purpose than all-purpose flour because, with a little bit of food science, you can turn this unfairly maligned flour into flaky Malabar parottas, crisp luchis, fluffy naans and kulchas, airy bhaturas, pillowy soft loaves of bread, crunchy-yet-chewy pizzas and delectable cakes without having to buy multiple kinds of flours to do it all.

https://lifestyle.livemint.com/food/discover/masala-lab-why-maida-is-not-the-flour-world-villain-111603383026971.html

And while on food, this excellent, entertaining article on custard:

Corn flour comes from pounding the kernel into a white powder that forms a non-Newtonian fluid––a liquid that doesn’t change viscosity under stress––when mixed with water. Its greatest virtue is that it contributes to thickness and volume without tasting like anything.
Its use as a food product was patented in Britain in 1854 by a man named John Polson Jr., who began manufacturing it in a factory in Paisley, Scotland owned by his father, John Polson, and his partners William Polson and John Brown. Some of their first advertisements declared that the product “was preferred on account of its plainness.”

https://fiftytwo.in/story/powder/

India, Bangladesh, GDP. Sigh.

When I explain GDP to folks unfamiliar with the concept, I often use the analogy of marks.

“Do you”, I intone in the most professorial voice I can muster, “remember how many marks you scored in your math exam when you were in the 4th grade?”

The point behind asking that question is to help the class realize that there were many other things going on in their life in the 4th grade. The measurement of how well you did on the specific questions you were asked in that test on that day do very little to show you how much math you actually learnt that year. Leave alone, of course, the question of how little the math test had to do with all of what you learnt while you were in the 4th grade.

A similar point was made about GDP recently, in the Business Standard:

Take GDP first. In India, we don’t measure the output of 65 per cent of the economy and make only well-informed guesses about the remaining 35 per cent.

https://www.business-standard.com/article/opinion/the-10-year-upa-nda-scorecard-120102400048_1.html

That’s exactly right, of course. You shouldn’t obsess over GDP numbers, much like you shouldn’t obsess over grades. But we do obsess over both!

And the analogy between marks and GDP works really well especially now, because when it comes to GDP, we now have a Sharmaji ka beta in the neighbourhood.

Hello, Bangladesh.

About two years ago, India’s Home Minister Amit Shah spoke of “infiltrators” who were hollowing out the country “like termites”. A Minister from Bangladesh retorted that Shah’s statement was “inappropriate”, “unwanted”, and “not based on information”. The IMF’s recent per capita GDP projections for South Asian countries show that the alleged ‘termite factory’ is shining — Bangladesh, which has been doing better than both India and Pakistan on social and human development indicators for several years now, is also beginning to march ahead on the economic front.

https://indianexpress.com/article/explained/an-expert-explains-how-bangladesh-has-reduced-gap-and-is-now-projected-to-go-past-india-6906206/

In much the same way that you shouldn’t compare marks obtained by students, you really shouldn’t compare GDP per capita between nations.

But (and you knew there was a but coming along, didn’t you), as I also say in my classes – what else you got, eh? It’s all well and good to say we shouldn’t, but it’s not like we have readymade alternatives. And if you take the GDP factory away from us economists, how do we fill our days?

TCA Srinavasa-Raghavan, in the same column cited above, has three answers:

Only three things: Food inflation, because it has a direct bearing on welfare; foreign exchange reserves, because they serve as a powerful signalling device to foreign investors and sellers of goods; and the revenue deficit. These are the only things the Centre has total control over. In determining all other indicators, the states play a big role.

https://www.business-standard.com/article/opinion/the-10-year-upa-nda-scorecard-120102400048_1.html

Read the whole article (which, I’m sorry, may well be behind a paywall). I don’t necessarily agree with all of it, about which more below, but the point that GDP is overrated as a useful barometer for the state of the economy is a point I agree with wholeheartedly.

TCA’s suggestions about what is to be used instead (food inflation, the revenue deficit and forex reserves) are worth considering, but there is a long list of alternatives that have been suggested. Here is just one example:

Provincial officials have long been suspected of overstating growth. Adding their figures together suggests that China’s economy was $364 billion bigger in 2009 than the total in the national accounts. Mr Li preferred to track Liaoning’s economy by looking at other indicators: the cargo volume on the province’s railways, electricity consumption and loans disbursed by banks.

https://www.economist.com/asia/2010/12/09/keqiang-ker-ching

Other folks may come up with other things to use as a proxy for measuring the state of the economy, but really, it is the old story of the six blind men and the elephant all over again. Whatever you use will give you only a limited picture. That’s just the nature of the beast.

Worse! Whatever you agree to measure instead of GDP immediately becomes susceptible to Goodhart’s Law:

In a paper published in 1997, Anthropologist Marilyn Strathern generalized Goodhart’s law beyond statistics and control to evaluation more broadly. The phrase commonly referred to as Goodhart’s law comes from Strathern’s paper, not from any of Goodhart’s writings:

When a measure becomes a target, it ceases to be a good measure.

https://en.wikipedia.org/wiki/Goodhart%27s_law

(Emphasis added)

So sure, you could ask that food inflation, revenue deficits and forex reserves be the target. But it’ll just be cobras or rat tails all over again.

So GDP, whether you like it or not, whether its measurement is favorable or not, is not going to go away anytime soon, whether in India or elsewhere.

Consider the concluding paragraph from a column in the Livemint yesterday by R Jagannathan:

This does not make GDP calculations worthless, but the real focus should be on sectors. More than macroeconomics, sectoral understanding and microeconomics ought to be central to policy-making. Future GDP will best be estimated as a sum of its parts, and not as a whole extrapolated from numbers in the more visible parts of the economy.

https://www.livemint.com/opinion/online-views/the-fallacy-of-equating-growth-with-the-pursuit-of-higher-gdp-11603811210462.html

Yes, well, sure. Absolutely.

Now if only we could figure out the how.

Reflections on The Entrepreneurial State, by Mariana Mazzucato

The full title of the book is “The Entrepreneurial State: Debunking Public vs. Private Sector Myths“, and the author is Mariana Mazzucato, Professor in the Economics of Innovation and Public Value at University College London (UCL), and Founder/Director of UCL’s Institute for Innovation and Public Purpose.

The key point made in the book is that entrepreneurship is not – and should not – the responsibility of the private sector. Indeed, it cannot be the responsibility of the private sector.

Early on in the book, she makes the strongest case there is to be made for her thesis, by arguing that the United States of America has known this, and practiced this, for years on end. The rest of the world, she says, would do well to emulate the USA:

If the rest of the world wants to emulate the US model they should do as the United States actually did, not as it says it did: more State not less.

LOCATION: 372 (Note that the location refers throughout to the Kindle version)

There are a lot of excellent reviews out there already. See this one in the New York Times, for example. It is a mostly favorable review. Or, if you want a slightly more critical one, see this one in The Guardian. Indeed, there are many others out there.

I want to focus on three key points in this essay: horizons, incentives and spillovers. Let’s tackle each in turn.

Horizons

Moonshots is a word that has become increasingly popular over the last two decades, and it refers to projects or even ideas that have a relatively low chance of succeeding. The payoff, if these ideas succeed, is so large that that it may compensate for the relatively low probability of this actually happening. That, of course, is exactly what expectations are all about.

But for a firm, particularly one that may not have the luxury of time and money on its side, placing bets on projects that may not work out – and indeed most of them will not – is a rather risky thing to do. Money is an obvious constraint, but a less obvious one is time.

Firms just do not have the luxury of waiting while a project turns out to be successful… eventually. These kind of moonshots, then, are perhaps best handled, for this specific reason, by the state.

In fact, the point is even more nuanced, because a firm is much more likely to (if at all) invest in a moonshot project based on a specifically desired outcome. The word project itself is an indication of this fact – this is not “blue sky research” that we are talking about.

But blue sky research is important!

A core difference between the US and Europe is the degree to which public R&D spending is for ‘general advancement’ rather than mission-oriented. Market failure theories of R&D are more useful to understand general ‘advancement of knowledge’–type R&D than that which is ‘mission oriented’ (Mazzucato 2015). Mission-oriented R&D investment targets a government agency programme or goal that may be found, for example, in defence, space, agriculture, health, energy or industrial-technology programmes (Mazzucato and Penna 2015).

LOCATION: 1549

Governments need to focus, for the sake of their own economies, their domestic firms and their long term growth, on focusing on moonshot projects, precisely because firms are reluctant to do so. The state needs, in other words, to take risks that private firms will not.

Saying this is easy, but how to go about doing this?

That is, if governments need to tackle long-term low-probability-of-success and uncertain-outcome initiatives that are important, but unlikely to be taken up by the private sector, the question that then arises is: how?

Mazzucato offers two points in this regard that I found interesting:

Block (2008, 188) identifies the four key characteristics of the DARPA model:

1. A series of relatively small offices, often staffed with leading scientists and engineers, are given considerable budget autonomy to support promising ideas. These offices are proactive rather than reactive and work to set an agenda for researchers in the field. The goal is to create a scientific community with a presence in universities, the public sector and corporations that focuses on specific technological challenges that have to be overcome.

2. Funding is provided to a mix of university-based researchers, start-up firms, established firms and industry consortia.

3. There is no dividing line between ‘basic research’ and ‘applied research’, since the two are deeply intertwined. Moreover, the DARPA personnel are encouraged to cut off funding to groups that are not making progress and reallocate resources to other groups that have more promise.

4. Since the goal is to produce usable technological advances, the agency’s mandate extends to helping firms get products to the stage of commercial viability. The agency can provide firms with assistance that goes well beyond research funding. Part of the agency’s task is to use its oversight role to link ideas, resources and people in constructive ways across the different research and development sites.

LOCATION: 1808

In effect, she is suggesting that government alone cannot do this, it needs to be a “scientific community” that is decentralized, has autonomy, sets the agenda, and applies Darwinian principles (see point 3). Hmm, sounds familiar. Different context, but a similar lesson!

And elsewhere in the book, her example of how Japan did this in the 1970’s is instructive:

The general point can be illustrated by contrasting the experience of Japan in the 1970s and 1980s with that of the Soviet Union (Freeman 1995). The rise of Japan is explained as new knowledge flowing through a more horizontal economic structure consisting of the Ministry of International Trade and Industry (MITI), academia and business R&D. In the 1970s Japan was spending 2.5 percent of its GDP on R&D while the Soviet Union was spending more than 4 per cent. Yet Japan eventually grew much faster than the Soviet Union because R&D funding was spread across a wider variety of economic sectors, not just those focused on the military and space as was the case in the Soviet Union. In Japan, there was a strong integration between R&D, production and technology import activities at the enterprise level, whereas in the Soviet Union there was separation.

LOCATION: 1142

And…

Equally important were the lessons learned by Japanese people that went abroad to study Western technologies for their companies, and relationships between those companies and US firms. These companies benefited from the lessons of the US (hidden) ‘Developmental State’, and then transferred that knowledge to Japanese companies which developed internal routines that could produce Western technologies and eventually surpass them.

LOCATION: 1156

So, bottom-line: the state has to get in this business, but it can’t “go” it alone. There needs to be a community of academicians, researchers, firms, scholars – and as the example of Japan shows, this community needs fostering, and horizontal collaboration.

Or, if you prefer to put it simply, this is going to be hard.

Incentives

Academia suffers from the same problem that government bureaucracy does in India: the incentives are all wrong. Both are about risk minimization.

A professor in a college has no incentive to try and do something new, something risky, something innovative. Why, if you think about it, should she? Your best case scenario is that it works, but you get no upside for it: remember, wages aren’t a function of what you do, they are a function of how long you have been in the system. Your worst case scenario is that what you tried to do blows up in your face. So why take the risk?

And it is the same, of course, with a government bureaucrat. And that makes the conclusion of the previous section even more problematic, for where, exactly, are you going to unearth government bureaucrats willing and able to make this happen?

I’m all for the state being more entrepreneurial. I buy into the idea. But I worry, especially in a country like India, about the feasibility of it, for hey, incentives matter!

In a blogpost I had written earlier this year about the budget, I had touched upon this point:

Here is Ninan’s solution:

“Is there a solution? Yes, railway engineers of old like the metro builder E Sreedharan, builders of government companies like D V Kapur and V Krishnamurthy, and agricultural scientists like M S Swaminathan have shown how they made a difference when given a free hand. Vineet Nayyar as head of Gas Authority of India was able to build a massive gas pipeline within cost and deadline in the 1980s. The officers who are in charge of Swachh Bharat and Ayushman Bharat, and the one who has cleaned up Indore, are others who, while they may not match China’s speed, can deliver. Perhaps all we have to do is to spot more like them and give them a free hand.”

But as any experienced HR professional will tell you, spotting them is very difficult, even in the corporate world. And as any corporate CEO will tell you, giving these talented folks a free hand is even more difficult. And as any student of government bureaucracy will tell you, achieving the intersection set of these two things in a governmental setup is all but impossible.

And so what we need to study and copy from China is not so much anything else, but lessons in achieving, and sustaining, excellence in government bureaucracy. Or, if you prefer, how to improve state capacity.

In short, quality of government, not size of government, is what matters for freedom and prosperity.

https://econforeverybodyblog.wordpress.com/2020/02/17/how-to-think-about-the-budget/

That point resonates even more in this context: fostering an ecosystem led by the government is dead in the water without either the proper incentives, or at least bureaucrats who are able to work through poorly designed incentives. It is a hard problem, state led entrepreneurship, and made harder by the problem of incentives.

Spillovers

Or externalities, if you prefer. It doesn’t matter how hard the problem is, the payoffs are worth it!

Ruttan (2006) argues that large-scale and long-term government investment has been the engine behind almost every GPT (general purpose technology) in the last century. He analysed the development of six different technology complexes (the US ‘mass production’ system, aviation technologies, space technologies, information technology, Internet technologies and nuclear power) and concluded that government investments have been important in bringing these new technologies into being.

LOCATION: 1570

(Note: emphasis added)

If those GPT’s are the outcome of general, as opposed to specific, R&D, sign me up. They are magnificent positive externalities. Indeed, elsewhere in the book, Mazzucato points to how almost everything produced by Apple today simply could not have been produced without an entrepreneurial state:

LOCATION: 2326

The final point that I’ll make relates to how Mazzucato proposes “capturing” some of these externalities:

Where an applied technological breakthrough is directly financed by the government , the government should in return be able to extract a royalty from its application . Returns from the royalties , earned across sectors and technologies , should be paid into a national ‘ innovation fund ’ which the government can use to fund future innovations . Granting a return to the State should not prohibit the dissemination of new technology throughout the economy , or disincentivize innovators from taking on their share of the risk . Instead it makes the policy of spending taxpayers ’ money to catalyse radical innovations more sustainable , by enabling part of the financial gains from so doing to be recycled directly back into the programme over time .

Location 3735

Mazzucato does present alternative schemes to the one shown above, but this is the one that strikes me as being the one with the most promise, if administered well, with appropriate risk-mitigation built in. But again, saying that is much easier than actually getting it done.

But all the being said, one simple fact is inescapable: India needs to be thinking about how to get something like this off the ground, and ASAP.

For that reason alone, more of us should be reading this book.

Notes on “Does Palantir See Too Much?”

Just this past weekend, I finally got around to finally finishing The Lord of The Rings.

I know, I know. Spare me.

Worse, I finished the movies, not the book, and I do not have the faintest idea about when I’ll get around to reading it, but that is a story for another day. (Heh.)

Today’s notes, coincidentally, are about Palantir, but the new age one, not the panopticon from Tolkein’s universe. The new Palantir is a firm started by Peter Thiel, and is the subject of a rather long (and very nice) profile of the firm and it’s Chief Executive Officer, Alex Karp.

First, what is Palantir Technologies? Here’s Wikipedia – note that I have combined sentences across different paragraphs in this excerpt:

Palantir Technologies is a public American software company that specializes in big data analytics. Headquartered in Denver, Colorado, it was founded by Peter Thiel, Nathan Gettings, Joe Lonsdale, Stephen Cohen, and Alex Karp.

The company is known for three projects in particular: Palantir Gotham, Palantir Metropolis and Palantir Foundry. Palantir Gotham is used by counter-terrorism analysts at offices in the United States Intelligence Community (USIC) and United States Department of Defense…

…Palantir Metropolis is used by hedge funds, banks, and financial services firms…

…Palantir Foundry is used by corporate clients such as Morgan Stanley, Merck KGaA, Airbus, and Fiat Chrysler Automobiles NV

https://en.wikipedia.org/wiki/Palantir_Technologies

… and here’s the NYT amgazine story:

Its two primary software programs, Gotham and Foundry, gather and process vast quantities of data in order to identify connections, patterns and trends that might elude human analysts. The stated goal of all this “data integration” is to help organizations make better decisions, and many of Palantir’s customers consider its technology to be transformative.

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

But the story gets more interesting in the very next line in the article…

Karp claims a loftier ambition, however. “We built our company to support the West,” he says. To that end, Palantir says it does not do business in countries that it considers adversarial to the U.S. and its allies, namely China and Russia. In the company’s early days, Palantir employees, invoking Tolkien, described their mission as “saving the shire.”

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

There’s two questions at play here, really. First, what does Palantir Technologies do (that’s the first excerpt from the NYT story)? And second, why does it do what it does (and that’s the excerpt right above)?

Now, the reason I find this so interesting is that the instinctive argument that you might want to make against Palantir Technologies is “but privacy!”. And the second excerpt above is, in a sense, Palantir’s response.

Although Palantir claims it does not store or sell client data and has incorporated into its software what it insists are robust privacy controls, those who worry about the sanctity of personal information see Palantir as a particularly malignant avatar of the Big Data revolution. Karp himself doesn’t deny the risk. “Every technology is dangerous,” he says, “including ours.”

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

Technology is technology – what you do with it is what matters is a rather old argument, but that’s the argument that is being used here. There’s more though – if we don’t, somebody else will. Better the known devil, etc.

Once the data has been integrated, it can be presented in the form of tables, graphs, timelines, heat maps, artificial-intelligence models, histograms, spider diagrams and geospatial analysis. It is a digital panopticon, and having sat through several Palantir demos, I can report that the interface is impressive — the search results are strikingly elegant and easy to understand.

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

Elsewhere in the article, the author speaks about how the work isn’t glamorous, and is really just glorified plumbing. Well, maybe – but as anybody who has lived in a house will tell you, it is plenty important. Good plumbing is plumbing you don’t notice, but reap the benefits of – and that seems to be Palantir’s USP.

While Thiel provided most of the early money, the start-up secured an estimated $2 million from In-Q-Tel, a venture-capital firm that finances the development of technologies that can help the C.I.A.
Karp says the real value of the In-Q-Tel investment was that it gave Palantir access to the C.I.A. analysts who were its intended clients.

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

Did In-Q-Tel pay to help start Palantir, or did it hire consultants for 2 million dollars? Did Palantir agree to work for only 2 million dollars to get access to the CIA?

Bottom-line: the world is a non-zero sum game.

According to Thiel, their conversations generally took place late at night in the law-school dorm. “It sounds too self-aggrandizing, but I think we were both genuinely interested in ideas,” he says. “He was more the socialist, I was more the capitalist. He was always talking about Marxist theories of alienated labor and how this was true of all the people around us.”

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

This excerpt is from a section which is about Karp figuring out his education and career, and we learn about his Jewish, rebellious background as well. I found this clip interesting because from Peter Thiel’s viewpoint, succeeding in selling the idea behind Palantir to Karp is one of the biggest validations there could possibly be. If he bought into the story, well, there must be something to it. Second, what better way to maintain checks and balances than to have somebody like Karp running the show?

In fact, Thiel hiring Karp for this job becomes more and more interesting the more you learn about Karp. Thiel has a quote in the article about needing someone who was smart and scrappy, but left unsaid, perhaps, is someone who was very unlike Thiel. And not just unlike Thiel, also unlike the typical CEO. A person who worries about the alienation of labor, likes solitary pursuits, and dreams of being an intellectual in Europe isn’t the person you would have in mind as the typical CEO of a firm like Palantir. But that, it would seem, was the whole point. Well, that, and being a bachelor by choice wouldn’t hurt, given the traveling nature of the job.

(Although there is a section in the article in which Karp insists that he being who he is hasn’t helped him or Palantir.)

Karp and Thiel say they had two overarching ambitions for Palantir early on. The first was to make software that could help keep the country safe from terrorism. The second was to prove that there was a technological solution to the challenge of balancing public safety and civil liberties — a “Hegelian” aspiration, as Karp puts it.

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

Karp and Thiel make for a Hegelian pair themselves!

When I asked Thiel about the risk of abuse with Palantir, he answered by referring to the company’s literary roots. “The Palantir device in the Tolkien books was a very ambiguous device in some ways,” he said. “There were a lot of people who looked into it and saw more than they should see, and things went badly wrong when they did.” But that didn’t mean the Palantir itself was flawed

…

He continued: “The plot action was driven by the Palantir being used for good, not for evil. This reflected Tolkien’s cosmology that something that was made by the good elves would ultimately be used for good.”

…

A moment later, he added: “That’s roughly how I see it, that it is ultimately good and still very dangerous. In some ways, I think that was reflected in the choice of the name.”

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

I found this fascinating, and I also found it useful to think about this from the Wikipedia article about the original Palantir:

A major theme of palantír usage is that while the stones show real objects or events, they are an unreliable guide to action, and it is often unclear whether events are past or future: what is not shown may be more important than what is selectively presented. Further, users with sufficient power can choose what to show and what to conceal

https://en.wikipedia.org/wiki/Palant%C3%ADr

The technology is what it is – and as Karp himself points out, it is susceptible to misuse. More importantly, the technology in combination with the person(s) who are using it is, at least potentially, an even more dangerous tool.

Karp made clear that he was opposed to Trump’s immigration policies: “There are lots of reasons I don’t support the president; this is actually also one of them.” He told me that he was “personally very OK with changing the demographics of our country” but that a secure border was something that progressives should embrace. “I’ve been a progressive my whole life,” he said. “My family’s progressive, and we were never in favor of open borders.” He said borders “ensure that wages increase. It’s a progressive position.” When the left refuses to seriously address border security and immigration, he said, the right inevitably wins. To the extent that Palantir was helping to preserve public order, it was “empirically keeping the West more center-left.”

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

To understand a big data firm started by one the world’s most successful VC’s, one should end up reading about a German philosopher from the 18th century – for what could possibly more Hegelian than that excerpt?

And finally, the last sentence in the article:

“Palantir,” he said, “is the convergence of software and difficult positions.”

https://www.nytimes.com/interactive/2020/10/21/magazine/palantir-alex-karp.html

That, it would seem, is a reasonably good way to understand Pater Thiel’s investment philosophy!

Read the whole NYT article yourself (at least twice, if you ask me).