In Praise of Randomness

In fact, this blogpost will not be enough. Randomness needs an entire book, maybe more.

This post is a reflection on Tim Harford’s excellent column titled “Sometimes a random solution is best“.

Here is the opening paragraph:

Just over a decade ago, Egypt’s Coptic Christians chose their new pope. The names of three favoured candidates were placed in a glass bowl, then a blindfolded boy selected from the trio at random. Religious people can appeal to the idea that the outcome wasn’t truly random; God himself decided on Tawadros II. Yet it is a seemingly unsettling way to deal with a serious decision.

https://timharford.com/2023/10/sometimes-a-random-solution-is-best/

As always, read the whole thing. Lots of advantages to random solutions, and here are the ones that Tim lists out:

  1. It promotes efficiency. Take grants, for example. If the amount of the grant isn’t that large, evaluating the grant can often cost more than the grant that is awarded! So long as everybody who is on a list has met some criteria, allocate grants randomly among these applications.
  2. Implicit in this, of course, is the point that the money that has been saved when it comes to detailed evaluations can be put to other uses. Maybe award twelve grants instead of ten? We have, in other words, increased the (potential) supply of grants.
  3. It incentivizes applications from diverse backgrounds. Folks from ethnic minorities (to use Tim’s own example) are more likely to apply if the process is randomized (can you guess why?). In other words, we have increased the (potential) demand for grants.
  4. Think about it: the market for grants has been made more efficient by making the process more random. Huh.
  5. Sometimes, Tim tells us, a randomized solution is better than listening to the experts, and he points us to the arrhythmia case study from the 1980’s.

Particularly in the case of entrance examinations in India, there is another benefit to applying randomized solutions. Rather than give the seats in a college to the highest rankers in an entrance examination, why not give admission randomly, so long as a minimum cutoff has been met?

Examinations in India, particularly the entrance variety, usually optimize for speed while solving a lot of fairly basic questions. That is, the highest scores tend to belong to those students who are unusually quick at solving a lot of not-very-difficult problems very quickly. The degree of difficulty for these questions varies a bit in the case of different universities, sure, but I would think what I have described here ought not to be too controversial.

But how is optimizing for the ability to quickly solve how far Rahul has walked if he turns right then left then right then left and so on a good measure of your ability to read and understand The Wealth of Nations? It might not hurt, sure, but hey – it might not help either! In fact, I would be much more comfortable with a hypothesis that said it is likely to not help. It has been a while since I read The Wealth of Nations, alas, but not once did I have to rely on my ability to do algebra very quickly in my head. I had to rely, instead, on my ability to persevere.

Would you agree that our entrance exams don’t optimize for perseverance?

What else do they not optimize for?

So, so long as a certain minimum benchmark is met, why not randomize admissions, and see if the cohort that comes in is better than the previous cohorts? And if so, along which dimensions?

It is unlikely that such a system will be adopted anytime soon, because of a variety of reasons. But all I’m saying is that there are, indeed, benefits to being random.

No?

Ec101: Links for 19th December, 2019

  1. “Based on the provided support, it is apparent then that it’s advantageous to be as random as possible for generation of ideas, but sticking with a particular response is predictive of creative originality. So next time your friends say that you are “sooo random,” hold your head up high and keep at it. But don’t forget to spot those brilliant ideas among the dis-order, and focus. Such is the recipe for creativity.”
    ..
    ..
    On the benefits of being random.
    ..
    ..
  2. “Convex functions play an important role in many areas of mathematics. They are especially important in the study of optimization problems where they are distinguished by a number of convenient properties. For instance, a strictly convex function on an open set has no more than one minimum. Even in infinite-dimensional spaces, under suitable additional hypotheses, convex functions continue to satisfy such properties and as a result, they are the most well-understood functionals in the calculus of variations. In probability theory, a convex function applied to the expected value of a random variable is always bounded above by the expected value of the convex function of the random variable.”
    ..
    ..
    That is from the Wikipedia article on convexity, and the next sentence after the excerpt leads us to…
    ..
    ..
  3. Jensen’s inequality!
    ..
    ..
  4. “The point is subtle and widely misunderstood. Here’s a simple example. Suppose that the average return is 10%. If $100 is invested for two periods the average payoff is $100(1.1)^2=$121. But on average that is not what happens. More typically, you get say 0% in the first period and 20% in the second period, i.e. $100(1.0)*(1.2)=$120. Notice that the average return is exactly the same, 10%, but the total payoff is smaller in the second and more realistic case”
    ..
    ..
    And Alex Tabbarok explain why Jensen’s Inequality matters
    ..
    ..
  5. As does Nassim Nicholas Taleb.