Earthquakes and Credit Rating Agencies

Credit rating agencies have been under intense heat for a long time.  Recently, Congress has joined the action.  For those unaware, the issue surrounds (mostly) AAA ratings.  Triple A bonds, investors believed, had "very little" risk.  The problem, however, results because no one knew what that meant.  But, it's worse than that: Even if investors knew what that meant, we won't live long enough to know if the agencies were right.

The business of underwriting catastrophic risk, like earthquakes and hurricanes, faces the same problem.  How can you tell if someone is good or bad at pricing risk?  The events they price happen (thankfully) truly infrequently.  So, when they lose money, are they wrong or unlucky? 

Suppose you know for certain that an event has a one in 500 year chance of occurring.  The event happens next year.  That's unfortunate luck.  You could be great a pricing such risk.  In fact, you could have been paid superbly to take the risk.  However, you lost.

Now, complicate matters ever so slightly, but reasonably.  You think the event happens once in 500 years.  But, maybe it's one in 400, maybe it's one in 600.  You can't be sure.  (Ask someone who didn't get a D in his first undergrad statistics class to tell you how many years it will take to know whether it's one in 400, 500 or 600. ) Now, if the event happens next year, you don't know if you were unlucky and right or unlucky and wrong, no matter how much premium you collected to take the risk.

What does this have to do with AAA rated bonds?  All we can say for sure in 2010 is that an incredibly rare economic catastrophe happened in 2008 and 2009.  We can't really say how rare with any degree of precision.  Worse, the rating agencies never even declared default probabilities for different bonds.

Where does that leave investors?  It's actually pretty difficult to say the rating agencies did anything wrong based on outcomes.  Bonds defaulted.  An incredibly rare event happened, but we don't know how rare.  And, bonds defaulted.

Do I defend the rating agencies?  No, I dare not.  Fault the rating agencies for their processes.  Do not fault them for defaulted bonds and observable outcomes.