The Philosopher's Cornered
Issue 6
Dealing with Randomness
by Dan Wolaver
Weather
is
hard to understand because it's complicated—there are so
many factors that influence it. So to get our heads around it, we
simplify the model by just taking the seasons and the earth's rotation
into account. Somewhere around February 1st the daytime temperature
bottoms out at about 25°, and it increases steadily until it reaches
about 85° around August 1st. Then it starts it's steady decline
back toward 25°. The earth's rotation makes the night
temperatures about 20° below the daytime temperatures. This
simplified model is shown in the right-hand graph here.
The problem with the simple model is that we are surprised when the high
for January 6th is 68°, and again when the low for March 6th is –4°
(see the left-hand graph here).
We're tempted to say something is wrong because the behavior doesn't
fit our simple model. But that's just the way randomness
works. A lot of factors come together at once and produce the
"perfect storm," and it just happens occasionally; there's
nothing wrong. The weather reports do their best to take many
factors into account, but even they have problems with accuracy more
than ten days out. So how do we deal with randomness that can't be
predicted accurately? We look at centuries of history and build
our house high enough to survive the 100-year flood level (the
"perfect storms"). If a 500-year flood level happens in
our lifetime, we rely on insurance.Another complex system is the economy. The Dow-Jones Industrial Average acts erratically, but over the long term it increases because "money gets money" (see the red curve in the graph here). In the early 1900s the smoothed stock market (red curve) took roughly 32 years to double in value. In the early 2000s it doubles in about eight years, so long-term investment pays off better now. It's risky to try to take advantage of the random ups and downs (the black curve) because they're unpredictable. There are too many factors in the model, but professional investment advisors have better models than most of us. So how do we deal with the randomness? Invest for the long term and get professional advice. Some resort to astrology or "lucky numbers"—desperate acts in the face of the unknown! We can build better models if we get to understand randomness. Although it is, by definition, unpredictable, it has some properties that we can identify. Average and standard deviation are identifiable properties. If we flip a coin, we expect to get half heads on average. Suppose we flip the coin 100 times and get 60 heads instead of 50. Should we be surprised? Most of the time (68% of the time) that we flip a coin 100 times, the number of heads will be in the range 50 ±5, so 60 heads would be unusual. Here the number ±5 is called the standard deviation. It's calculated by taking the square root of the number of flips and dividing by 2. So the square root of 100 is 10, and half of that is 5. Suppose a pollster asks 400 voters who they're going to vote for—A or B—and finds that 50% are going to vote for A and 50% for B; the spread between the candidates is 0%. But the pollster warns that the margin of error for the spread is ±5%. Where did this 5% come from? Well, he "flipped the coin" 400 times and it came up "A" 200 times. What is the standard deviation? The square root of 400 is 20, and half of that is 10. And 10 is 2.5% of the 400 responses. The spread's margin of error is twice this, or ±5%, since what A gets is at B's expense. The actual outcome could be as much as 52.5% for A and 47.5% for B, or the other way around. Sometimes we get an understanding of randomness just by playing around and getting familiar with it. Suppose we shake up 400 white marbles and 400 black marbles in a bag and pour them into a tray. It will probably look something like the left-hand pattern here. Our first reaction is that something has caused white and black clumps to form—the marbles aren't mixed well enough. But random distribution is clumpy—not even. We would probably be more satisfied with the distribution on the right-hand side, which is more even but obviously not random.. In the midst of randomness we sometimes long for a simplicity that is predictable and safe. Rather than trying to insulate ourselves from the complex world with its surprises, we should embrace the adventure and learn to deal with randomness as an old friend. And that's my philosophy. |