# The Pathological Distribution

### Shaped like a bell, but does it behave like a bell?

I’ve spent a lot of time telling you about the Central Limit Theorem. There’s a good reason for that: its consequences can be felt everywhere in research. The Central Limit Theorem really puts the normal distribution in the spotlight:

The normal distribution goes by some other names, since some people can’t get past the use of the word “normal” in everyday language. Sometimes it’s called a Gaussian distribution, other times it’s called the bell distribution. I usually prefer to call it a normal distribution since there are actually other famous distributions that take on bell shapes.

In this issue, we will talk about another important aspect of the normal distribution that would be cool to highlight. There’s several distributions with a bell-shape, but slight differences in their shape can have extremely weird consequences down the line.

Today, you will learn about a pesky cousin of the normal distribution.

## Fat Tails, Fat Bells

Behold, a distribution that defies all the intuitions of Biostatistics MS students everywhere:

This pathological, little distribution visualized above is called the **Cauchy distribution.** The Cauchy distribution certainly looks like a skinner, shorter version of the normal distribution. Something that should be appreciated about both probability distributions is that the *areas underneath the curve* should be equal to 1. This is a fundamental property of all probability distributions, since the probability of all events should intuitively be 1 too.

With a normal distribution, it’s famously known that 99.7% of all the probability in a normal distribution falls within 3 standard deviations of the mean. Using statistical jargon, all the possible values of a normal distribution are** highly concentrated around its mean**. This is especially surprising since a normal distribution can theoretically take values across *the entire real line. *

Another way of phrasing this is that even though a normal distribution can take on *infinite amoun*t values, we will only observe some within a small “clump”. Since these values are associated with a probability, they have a special interpretation:

Values around the average of a normal distribution are considered “commonplace”, values at the tails are considered “very rare”

The picture above certainly shows that the Cauchy distribution is relatively skinnier, but where has the all of that “density” gone in the Cauchy distribution Just as carrot cake goes straight to my legs, the probability density of a Cauchy distribution has gone to its **tails** aka more extreme values.

That is to say, **we observe extreme observations much, much more frequently with Cauchy distributions, compared to a normal distribution. **

## A problem with expectations

Okay, but so what?

Well, for one thing, there is no such thing as an “average” value in a Cauchy distribution. Unlike a normal distribution, a Cauchy distribution has no **population average**. This is extremely weird. This would be like asking “What’s the average intelligence of a group of people whose IQs comes from a Cauchy distribution?" and being told, “That question makes no sense.”

To illustrate this point, I’ve created a gif of what the sample average is for an increasing number of standard Cauchy samples (aka the middle is at 0). For reference first, I also have another GIF of what the sample average is like for more and more standard normals (who have mean 0):

Now see what happens when we try the same for a Cauchy sample:

Both of the distributions I’m sampling from to create the GIFs above are centered at zero, but notice how differently they end up. Very quickly, the sample average comes very close to the population average of 0, and it gets closer and closer with more observations. However, since a Cauchy distribution has heavier tails, more extreme values are more common and they greatly influence the resulting average. This results in sudden spikes that drive the average away from the middle 0.

It’s harder to see that the y-axis scales of the graph are very different. We can see the effect is more when they’re on the same plot:

To wrap up this section, I want to note that the formal name for an “average” is the **mean** or the **expectation. **Stat jokes, I’ll be here all day, everybody.

## The pathological distribution

The Cauchy distribution is typically mentioned to people as a passing curiosity. It’s a rare case that violates one of the requirements needed to use the Central Limit Theorem, so that’s why it’s famously quoted as “pathological.” If we try to use the sample average to estimate the center of a Cauchy distribution, this is what happens when you start aggregating all of the averages:

The distribution is horribly skewed in both directions. Some of the sample averages reach as high as -70 or 24, and this is all coming from a standard Cauchy, which we’ve seen above. It’s definitely not the normal distribution that’s guaranteed by CLT, but does it still have a distribution?

Yes, it does. The sample average of a bunch of standard Cauchy observations *also *follows a standard Cauchy distribution.

The green shape is still our histogram, but the red line is the probability density function of a standard Cauchy. The sample average of standard Cauchys… is still Cauchy. In less jargony words, if you try to calculate an average from a population where extreme values are not-so-rare, then your resulting averages will also be extreme at a not-so-rare rate.

## Why should I care about this?

Realistically, most people will not have to encounter the Cauchy distribution in their lives. As a Ph.D student in biostatistics, **I** rarely encounter this distribution.

The key takeaway you should leave with this article is the idea of **heavy tails**. With well-behaved distributions like the normal distribution, very extreme events are *so rare and improbable *that you can **assume** that they will never happen in our lifetime. They *might* happen, but maybe once every trillion years, to give you an idea of how implausibly rare it is.

Problems happen when you assume that these very extreme events can be ignored. Nassim Nicholas Taleb explored this idea in his book The Black Swan. Not to be confused with the movie, the idea of a “black swan” refers to an extremely rare event. For a long time, people believed that swans were only white. This was the prevailing belief until, at long last, someone witnessed a black swan.

In the context of the 2008 housing crisis, lenders assumed that defaults on loans were much rarer than they actually were. That is to say, these defaults had *heavier tails* than were truly appreciated. When these defaults happened at a faster rate than expected… we got something like the 2008 housing crisis. People assumed a normal, when they should have been expecting a Cauchy.

May you also be prepared for the first Black Swan in your life.

## Create your profile

## Only paid subscribers can comment on this post

Sign in## Check your email

For your security, we need to re-authenticate you.

Click the link we sent to , or click here to sign in.