What a difference a year makes

Here we are in September 2009, awaiting a whole raft of articles to be published, TV documentaries to be aired, and legislation to be recommended, all of it focused on and in response to what happened across the financial markets 12 months ago. I therefore feel excused from having to comment upon it myself.

What I would like to highlight is the impact that the last 12 months has had on the some of the statistics that we rely on when it comes to measuring risk, and show how even some very simple models have changed hugely through incorporating the new data. I also want to highlight how an incomplete presentation of these statistics can have huge implications on our understanding.


One of the more (in)famous quotes of last year is from David Viniar, Goldman’s chief financial officer, who said, "We were seeing things that were 25-standard deviation moves, several days in a row." I do not intend to add further to the large amount of predominantly critical commentary focused on these particular words, but I did think that they provide a basic framework to work from. Now while David was no doubt referring to the short-horizon movements of a particular asset that Goldman held , for simplification I will consider monthly movements of a general index, the S&P 500. The principle is exactly the same, but by doing this I sidestep both the issues of identifying the asset, as well as the well documented issues of using daily data.

The two datasets used in comparison are the 60 monthly returns up to August 2008 (i.e., the five years prior to last year's crash) and the 60 monthly returns up to August 2009. I have selected 60 months, as this is the horizon used in most long term risk models, and if we look at these returns against a normal background we get the following chart:

While there is some obvious kurtosis, skew is minimal and the normal assumption does not seem extreme. Assuming the normal distribution, the descriptive statistics for these distributions are

The reduction in the average return reflects not only the recent downturn but also excludes the postive market run through 2004. The big change though is in the standard deviation of those returns, the value almost doubling.

Paraphrasing David Viniar, we see that the realised return of September 2008, a month that saw the S&P500 fall 9.08%, was a 3.5 standard deviation event as of August 2008, but only a 2 standard deviation event were it to happen in September 2009. These numbers are much more digestible than the 25 deviations he was talking about, but do we really appreciate the difference between 3.5 and 2 deviations?

If we accept the normal distribution for our model, then the September 2008 return was a once in 345 year event from the point of view of someone last year, while using all the data that we have today shows that it would be expected to occure every 3.5 years, a difference of a hundredfold. In actual fact the S&P500 Index has delivered return of less that -9.0% on three occasions over the last 12 months, suggesting that this multiple is even too low!

In summary, statistics are calculated using the data available and report descriptive values. Depending on how those values are framed when they are reported can have a huge impact on risk understanding. Shouldering the burden of improving the understading of risk, we must all take care and resist the urge to throw out even simple statistics without any accompanying education.

To receive future posts by e-mail, subscribe to this blog.