Not long ago, I have written much about how statistics affect our lives, so how appropriate it is, on the occasion of the 25th anniversary of the Outstanding Students Awards, for me to reflect on how statistical science views the nature of success. Of bedrock conclusions, I’m afraid the science has offered few: curious readers can review recent books on the science of “happiness”; research into why businesses succeed has produced many PhDs but not one durable insight; equally barren is the field of studying why stock prices go up or down. But not to despair, for there is plenty to be learned from the way statisticians approach this subject. Let me elaborate.
Understanding what causes what is one of the fundamental problems in statistical science; it is also one of the preoccupations of the human mind, as psychologists have discovered. Statisticians, especially those working in the social sciences, have developed an unshakeable faith that everything has multiple causes. This world is as far away as you can imagine from the textbook world of one-factor-at-a-time, all-other-things-being-equal, ceteris paribus. Would a child earn higher test scores if she were enrolled at a private boarding school instead of an under-funded, public school, assuming all else equal? But this transplanted student has poorer parents, lives in worse neighborhoods, and has attended lesser-ranked schools than the typical private-school pupil. In the statistician’s world, few other things can be held constant when one lever is being shifted.
Explaining success is no different. Success has multiple causes. It is not turned on one lever. There are a million paths to success. This is not an easy lesson to learn. One only has to open a few textbooks at random to find ceteris-paribus assumptions, or browse a newspaper headline or two to see asserted the one company earnings report that induced a market crash. Only textbook authors get away with such a naive view of life; in real life, we don’t.
Another mainstay in the statistician’s worldview is what I call the “fudge factor.” A universal component of the discipline’s mathematics (normally named the “residual” or “error term”), the fudge is a grab-all for everything that is not knowable; its inclusion, our nod to Mother Luck. Luck plays a role in everything in life. No matter the talent, no matter the hard work, no matter the pedigree or street smarts, no matter the advantage of location, nor of the supporting cast, one doesn’t succeed without some bits of luck. In the same way, a statistical model is incomplete without the inclusion of the fudge factor.
I have sometimes been asked about U.S. college admissions. One certainty is that the collection of superbly-talented, hard-working, over-achieving high-school seniors can fill the roster of any topranked college many times over. The decision might come down to this sort of circumstances: Did the applicant’s special talents happen to match those of the reader? In what mood was the reader when reviewing his application? Did the reader just reject – or accept – the previous 10 applicants? For the majority of these deserving students, it is clear that luck largely selects who gets the fat acceptance package.
People say you make your own luck. I find that hard to believe. It is, of course, important to identify and cultivate one’s talent, and to put in honest, hard efforts. But no one controls luck. If there is a way to create luck, it wouldn’t be luck. If statisticians know what causes luck, we won’t need to fudge. That said, one can get into positions to receive the gift of luck, which could come in a million ways.