Understanding and Managing Model Risk: A Practical Guide for Quants, Traders and Validators

Part I

Theory and Practice of Model Risk Management

1

Understanding Model Risk

1.1 WHAT IS MODEL RISK?

In the last years, during and after the credit crunch, we have often read in the financial press that errors on ‘models’ and lack of management of ‘model risk’ were among the main causes of the crisis. A fair amount of attacks regarded mathematical or quantitative models, like the notorious Gaussian copula, that were accused to be wrong and give wrong prices for complex derivative, in particular credit and mortgage-related derivatives. These criticisms to valuation models have been shared also by bank executives and people that are not unexperienced on the reality of financial markets. In spite of this it is not very clear when a model must be considered wrong, and as a consequence it is not clear what model risk is.

We can probably all agree that model risk is the possibility that a financial institution suffers losses due to mistakes in the development and application of valuation models, but we need to understand which mistakes we are talking about.

In the past, model validation and risk management focused mainly on detecting and avoiding errors in the mathematical passages, the computational techniques and the software implementation that we have to perform to move from model assumptions to the quantification of prices. These sources of errors are an important part of model risk, and it is natural that model risk management devotes a large amount of effort to avoid them. We will devote a share of the second part of this book to related issues. However, they regard that part of model risk that partially overlaps with a narrow definition of operational risk: the risk associated to lack of due diligence in tasks for which it is not very difficult to define what should be the right execution. Is this what model validation is all about? In natural science, the attempt to eliminate this kind of error is not even part of model validation. It is called model verification, since it corresponds to verifying that model assumptions are turned correctly into numbers. The name model validation is instead reserved to the activity of assessing if the assumptions of the model are valid. Model assumptions, not computational errors, were the focus of the most common criticisms against quantitative models in the crisis, such as ‘default correlations were too low’.

The errors that we can make in the assumptions underlying our models are the other crucial part of model risk, probably underestimated in the past practice of model risk management. They are the most relevant errors in terms of impact on the reputation of a financial institution that works with models. A clear example is what happened with rating agencies when the subprime crisis burst. When they were under the harshest attacks, rating agencies tried to shield themselves from the worst criticisms by claiming that the now evident underestimation of the risk of credit derivatives was not due to wrong models, but to mistakes made in the software implementation of the models. Many market operators, that knew the models used by rating agencies, did not believe this justification, and it had no other effect than increasing the perception that wrong models were the real problem. What is interesting to notice is that admitting wrong software appeared to them less devastating for their reputation than admitting wrong models.

Unfortunately, errors in mathematics, software or computational methods are easy to define and relatively easy to detect, although this requires experience and skills, as we will see in the second part of the book. Errors in model assumptions, instead, are very difficult to detect. It is even difficult to define them. How can we, as the result of some analysis, conclude that a model, intended as a set of assumptions, has to be considered wrong? We need to understand when a valuation model must be called wrong in order to answer to our first crucial question, what is model risk?

In this section we look for the answer. The first sources we use to clarify this issue are the words of a few legendary quants that in the past have tried to say when models are right or wrong in order to give a definition of model risk. You will see that not even among quants there is consensus about what model risk is. But then, when we apply these approaches to past crises to understand how they could have protected us from the worst model losses, we will see that the different approaches can lead to similar practical prescriptions.

1.1.1 The Value Approach

As early as 1996, before both the LTCM collapse and the credit crunch, the two events that put most critical pressure on the risk involved in using mathematical pricing models, one of living legends of quantitative finance, Emanuel Derman, wrote a paper titled Model Risk. This is a natural starting point to define our subject, also because it can be seen as the foundation of one of the two main schools of thought about model risk. The views of the author on the subject are further specified by a later paper written in 2001 that addresses model validation prescriptions, under the title The Principles and Practice of Verifying Derivatives Prices.

Derman notices first that the previous years had seen the emergence of an ‘astonishingly theoretical approach to valuation of risky products. The reliance on models to handle risk’, he points out, ‘carries its own risk’. Derman does not give a definition of model risk, but he indicates some crucial questions that a model validator should have in mind:

1. Is the payoff accurately described?

2. Is the software reliable?

3. Has the model been appropriately calibrated to the prices of the simpler, liquid constituents that comprise the derivative?

4. ‘Does the model provides a realistic (or at least plausible) description of the factors that affect the derivative’s value?’

Can we deduce a definition of model risk from these points? The first two points are not trivial. When speaking of approximations and numerics in Chapter 6 we will talk of errors to avoid in implementation, and we even devote the entire Chapter 10 to the errors that can be made in the description of a payoff. However, these points do not add to our understanding of Derman’s ideas about the nature of the errors we can make in model assumptions.

The third point instead underlines a feature that models must have: the capability to price consistently with the market the simpler instruments related to a derivative, namely to perform the so-called calibration. This is an important issue, on which we will focus later on. But not even this point clarifies what model risk is. All banks, now, calibrate their models to liquid market prices. For any asset class or financial product there are many models which are different from each other and yet can all be calibrated very well to the market. Once we have satisfied this calibration constraint, are we sure that model risk has been eliminated, or instead the core of model risk is crucially linked to the fact that we have different models allowing for good calibration, so that calibration does not solve our model uncertainty?

A better clarification is given in the fourth point. From this we can deduce a definition of model risk. Once we are sure that we have correctly implemented payoff and software, and our model appears calibrated to the liquid underlying products, we have a residual risk that seems to be the core of model risk:

Model risk is the risk that the model is not a realistic/plausible representation of the factors affecting the derivative’s value

This is confirmed when Derman says that for less liquid or more exotic derivatives one must verify the ‘reasonableness of the model itself’. There is more. Derman (1996) gives an account of the things that can go wrong in model development, and he starts from some examples where lack of realism is surely the crucial problem:

‘You may have not taken into account all the factors that affect valuation … You may have incorrectly assumed certain stochastic variables can be approximated as deterministic …. You may have assumed incorrect dynamics … You may have made incorrect assumptions about relationships’. E. Derman, Model Risk.

So, is Derman saying that we should try to find out what the true model is? No, in fact he never uses those somewhat exoteric concepts like the true model or right model. He states, and it is hard to disagree, that a model is always an ‘attempted simplification of a reality’, and as such there can be no true or perfectly realistic model. But realism and reasonableness, coupled with simplicity, must remain crucial goals of a modeller, and their lack creates model risk.

Is Derman saying that we must look for realism and reasonableness in all aspects of the model? Not either. We must care for those aspect that have a relevant impact, limiting the analysis to ‘the factors that affect the derivative’s value’.

This approach to model risk is probably the one shared by most practitioners of finance and beyond, and does not appear too far away from the views expressed more recently by Derman. For example, in the ‘Financial Modeler's Manifesto’, written with Paul Wilmott, another legend of quant finance, we read among the principles that a modeler should follow ‘I will never sacrifice reality for elegance without explaining why I have done so. Nor I will give the people who use my model false comfort about its accuracy’. We refer to this, and to Derman's recent book `Models Behaving Badly – Why Confusing Illusion with Reality Can Lead to Disaster, on Wall Street and in Life', whose title is already very telling, for more about Derman's views.

It is clear to everyone that knows finance and does not confuse it with mathematics and not even with physics, that there is not such a thing as the ‘true value’ of a derivative that the model should be able to compute. However realism and capability to describe the actual behaviour of the relevant risk-factors are crucial principles to judge a model, and more realistic models should be preferred. Somewhat, we can say that the right model and the right value do not exist in practice, but wrong models and wrong values do exist, they can be detected and we should commit ourselves to find models giving values as ‘little wrong’ as possible, and then manage the residual unavoidable risk. This is the reason why we talk of ‘Value approach’.

There are cases where we can all agree that the price given by some models does not correspond to the value of a derivative. Most of these cases are trivial. If we are selling an out-of-the money option on a liquid volatile underlying, the model we use must incorporate some potential future movement of the underlying. We cannot use a deterministic model, assuming no volatility. Otherwise we would be selling the option for nothing, based on an assumption that can be disproved just waiting a bit and seeing the price of the underlying move in the market.

We will see other examples which are less trivial and yet we can easily spot that some assumptions are not realistic. To give an example regarding the infamous credit models, you will see in Chapter 2 the case of default predicted exactly by spreads going to infinity according to standard structural models or in Chapter 3, speaking of Gaussian copula, again a default predicted exactly, and some years in advance, by the default of another company. These assumptions are unrealistic and yet they are hidden in two very common models. When they do not impact in a relevant way the value of a derivative, we can consider them harmless simplifications. When, like in the examples we will analyze, we can show that they impact strongly the value of a derivative, we should raise a warning. At times it is more difficult to say when a relevant feature of a model is realistic or not; in this case we will have to use our judgement, collect as much information as possible and try to make the best possible choice.

You may at first think that everyone must agree with such a reasonable and no-nonsense approach, and with the definition of model risk it implies. It is not like that. A view on Model Risk that starts from completely different foundations is analyzed in the next section.

1.1.2 The Price Approach

If Derman has been one of the fathers of quantitative modelling between the end of the eighties and the nineties, Riccardo Rebonato marked the development of interest rate models – the field where the most dramatic quantitative developments have been done – between the end of the nineties and the subsequent decade. He has been a master in bridging the gap between complex mathematics and market practice. After the turn of the century Rebonato wrote a paper titled Theory and Practice of Model Risk Management that presents a view on the subject strongly different, at first sight, from the classic view explained above.

Rebonato (2003) takes the point of view of a financial institution, which is worried not only of the material losses associated to model risk, but even more of the effect that evidence of model risk mismanagement can have on the reputation of a financial institution and its perceived ability to control its business. Under this point of view, this classic definition of model risk and model validation are misplaced. In fact derivatives need to be marked-to-market, as we will see in Section 1.3, and this means that the balance-sheet value of a derivative must come as much as possible from market prices.

If this is the situation, what should the main concern of a model validation procedure be? Should we worry so much that ‘the model provides a realistic (or at least plausible) description of the factors that affect the derivative’s value’? Well … at least this is not the first concern we must have, since, to use the words of Rebonato, ‘Requiring that a product should be marked to market using a more sophisticated model (ie, a model which makes more realistic assumptions) can be equally misguided if … the market has not embraced the superior approach.’

These considerations lead Rebonato to an alternative definition of model risk, that has become so popular that we can consider it the motto of a different approach to model risk, the Price approach:

‘Model risk is the risk of occurrence of a significant difference between the mark-to-model value of a complex and/or illiquid instrument, and the price at which the same instrument is revealed to have traded in the market’. Rebonato R., Theory and Practice of Model Risk Management

Rebonato (2003) justifies this view pointing out that the real losses that hit an institution’s balance sheet usually do not appear ‘because of a discrepancy between the model value and the true value of an instrument’, but through the mark-to-market process, because of a discrepancy between the model value and the market price.

Fair enough. It is hard to disagree with such statements. As long as the market agrees with our model valuation, we do not have large losses due to models. When we evaluate with a model which is the same one used to reach market prices, we do not have model losses arising from mark-to-market thus we have no accounting losses. More interestingly, we can also avoid material losses, because, if the market agrees with our valuation model, we can always sell an asset or extinguish a liability at the price at which we have booked. This is true even if the market model is, to use the words of Rebonato, ‘unreasonable, counterintuitive, perhaps even arbitrageable’.¹

This has another implication. When the market price can be observed quite frequently, there is little time during which the model price and market price of a derivative can diverge, so that big model risk is unlikely to be generated. If a bank notices a mispricing, this will be controlled by provisions such as stop-losses and will not generate losses so big to worry an institution, although they can worry a single trader. The problem arises with very complex or illiquid products, for which market prices are not observed frequently. Then the model price of a derivative and its market price can diverge a lot, and when eventually the market price gets observed a large and sudden loss needs to be written in the balance-sheet, with effects on a bank which are also reputational.

The different definition of model risk given by Rebonato (2003) requires, at least at first sight, a different approach to model validation. Large losses with reputational damage emerge when a sudden gap opens between market price and model booking. This can happen for three reason:

1. The reason can be that we were using a model different from the market consensus, and when we are forced to compare ourselves with the market – because of a transaction or because the market consensus has become visible – this difference turns into a loss. From this comes the first prescription of the Price approach, given strongly in Rebonato (2003), to gather as much information as possible on the approach currently used by the majority of the market players. This can be done through different channels. We follow Rebonato (2003) and we add some more of our own, which have become more important after Rebonato’s paper was written.

A. Some channels are based on the idea that if we can observe prices from counterparties, then we can perform reverse-engineering of these prices, namely we can understand which models were used to generate them. Examples of how this can be performed are in Chapter 2, in Section 4.1 and throughout the book. How can we collect counterparty prices when the market is not liquid?

getting as much information as possible about the deals which are struck in the market or other closeout prices such as those for unwindings and novations.

analyzing the collateral regulations with counterparties. Collateral is the amount of guarantees (usually cash) exchanged between banks in order to protect the reciprocal exposures from counterparty risk. The amount of collateral must be kept equal to the expected discounted exposure, that corresponds approximately to the price of all deals existing between two counterparties. We can observe this frequent repricing from our counterparties, in some cases also specifically for a single deal, to get information on the models they use.

monitoring broker quotes (that usually do not have the same relevance as prices of closed deals) and consensus pricing systems such as Mark-it Totem. This is a service that collects quotes from market operators on a range of different over-the-counter derivatives, eliminates the quotes that appear not in line with the majority, and then computes an average of the accepted quotations. The market operators whose quotes were accepted get informed about the average. There are derivatives for which this service provides a very relevant indication of market consensus. Today, this is considered an important source of market information.

B. A few channels suggested by Rebonato (2003) regard gathering market intelligence by

attending conferences and other technical events where practitioners present their methodologies for evaluating derivatives.

asking the salesforce for any information they have about counterparty valuations. Additionally, salespeople can inform us if the prices computed with our models appear particularly competitive in the market (are we underestimating risk?) or are regularly beaten by competitors’ prices (are we being too conservative?).

Rebonato (2003) says finally that ‘contacts with members of the trader community at other institutions are invaluable’. We can rephrase it, less formally, as follows: keep in touch with your college mates that work in other banks and make them speak out about the model they use at the third pint of beer at the pub.

2. If, thanks to any of the above channels, we are confident that we are using the same model prevailing in the market and this model is not changing, the only cause for large gaps between our booking and market prices can be the model/operational errors like software bugs or errors in describing the payoff. Therefore these errors must be avoided.

3. The two points above do not appear to help us in the past examples of big market losses. In 1987 there appeared to be a market consensus on the use of something similar to the Black and Scholes formula to price equity derivatives. After the market crash in October 1987 the pricing approach changed dramatically, with a clear appearance of the smile. The market consensus had moved from a lognormal model to some approximation of a model with fat-tails, may it be a stochastic volatility model or a model admitting jumps, and this was a big source of losses. Those that had sold out-of-the-money puts for nothing had to book a loss not only because of the fall of the underlying, but also because the volatility used by market player to evaluate them became much higher than the one used for at-the-money options. Even following the above points 1) and 2) of the Price approach, we would have been completely exposed to such losses. Similar market shifts in the pricing approach to interest rate derivatives characterized the aftermath of the LTCM crisis in 1998. And we have recently experienced the most dramatic event of this type with the subprime crisis and the fall of the Gaussian copula based pricing framework for CDOs. This gives the third way in which we can have a large gap between the way we were pricing and the market price: even if we are using the market consensus model, the market consensus can suddenly change. This issue is taken into account by Rebonato (2003) that, after presenting knowledge of the market approach as the first task of a model risk manager, adds that ‘the next important task of the risk manager is to surmise how today’s accepted pricing methodology might change in the future.’

There is actually one fourth case that I have to add, but that we can resume in the third one. It is the case when our market intelligence research reveals that there is no model consensus in the market, a case that we analyze in Chapter 2. Also in this case the diligent risk manager will try ‘to surmise’ which future consensus will emerge. Some other indications on how to behave in this case are given in Chapter 2.

Now the crucial question that a model risk manager surely will ask is: how the hell can we surmise or anticipate how the market model is going to change? Can we devise some patterns in the dramatic changes in model consensus that have led to big model losses? It is unavoidable to start our analysis of this point from the recent, still hurting credit crunch crisis. In the following account I do not minimally try to be exhaustive in describing the reasons and the mechanism of the crisis; with the amount of books and papers written on this that would be indubitably redundant. I will try instead to focus only on the modelling aspect of what happened in 2007, and in doing this I will try to single out what I find are the most relevant elements.

1.1.3 A Quant Story of the Crisis

Let us recall what was the situation before the subprime crisis burst. An efficient market intelligence would have revealed that there existed a consensus, agreed upon at least among the most active market participants, over the pricing of those credit derivatives where the crisis burst first.

Rating agencies and banks used the Gaussian copula, that we resume here and analyze in detail in Chapter 3, for computing the prices of bespoke CDO’s. For the few that, during the crisis, were not able to learn what CDOs are, see the next section. We call ‘bespoke’ those CDO’s which are written on a portfolio of credit risks whose correlations are not liquidly traded. The predominant mass of CDO’s, including mortgage-backed CDO’s, were bespoke. While the Gaussian copula was used by the majority of players, there were differences in the computation of correlations. Rating agencies computed correlations historically while banks had a mixed approach. On one hand they kept an approach consistent with rating agencies since they needed ratings to market their products, on the other hand they often performed mark-to-market of CDO’s by a Gaussian copula with a correlation smile given by some mapping approach that will be explained in Section 3.5.

The modelling frameworks used made it almost always possible to extract from a portfolio of defaultable mortgages a good size of senior and mezzanine CDO tranches (explained below) whose risk was evaluated to be quite low, allowing in particular to give high ratings to these tranches. Senior and mezzanine tranches had been at the heart of the expansion of portfolio credit derivatives before the crisis, and then they were the first market where the crisis burst. The optimistic judgement on the riskiness of these products was crucial to fuel the growth of their market. In fact, the underlying mortgages generated a high level of returns, which kept the spread paid by these products much higher than a risk-free return (even 200bp over Libor for AAA securities) in spite of the low risk certificated by high ratings. This correspondence of high returns and low certified risk made the products very attractive.

In the following Section 1.2.1 we explain better how the demand and supply markets for CDOs were composed, which provides an even better understanding as to why rating was a crucial element for the investment choices of funds and also banks. There we tackle also another issue that you may have already heard of: did rating agencies and banks really believe the above picture? The issue is tricky. It may be that the modelling framework we are going to present was so much liked in the market because, by minimizing the risk of CDO’s, it matched well the distorted perception of risk of some operators with an artificially short-term investment horizon, like those we will see in Section 1.2.1. More likely, there were surely bona-fide players that truly believed the optimistic picture (I have met some of them), there were some others that were bending models to their purposes, and a large mass of operators that did not have elements to make an informed judgement and followed someone else’s advice.

Here this is of limited relevance to us, because what counts was that there was a consensus on the modelling assumptions for valuation. This model consensus was followed by the active operators and as such it protected those using it from model losses, as noticed by the Price approach to model risk, no matter if the model was right or wrong, believed by all players or not. The losses arose when the model consensus fell, and the causes of this fall we are going to study, to understand how the market consensus on a model can suddenly change.

The pre-crisis market model

CDO’s are derivatives where one party buys protection from the losses that can arise from a portfolio of investments, for example mortgages, while the other party sells protection on these portfolio losses. What makes them special is that here the loss is tranched. What does it mean? If we buy protection on the tranche with attachment point A (for example 3% of the total notional) and detachment B (for example 6%) we only receive the part of the loss that exceeds A and does not exceed B.

For a portfolio with 100 mortgages in it, all for the same notional and with zero recovery in case of a default, the situation of a buyer of the above 3%–6% tranche is as follows (notice that the buyer of a tranche is the protection seller). Until the first three defaults in the portfolio, he suffers no losses. At the fourth loss, namely when total losses have just exceeded its 3% attachment point, he loses of the nominal of its tranche. He will lose another third at the fifth loss, and the last third at the sixth loss, when its 6% detachment point is touched. From the seventh default on, he will lose nothing more. He has already lost everything. For him the best situation is when there are either 1 or 2 or 3 defaults, because he loses nothing, and the worst situation is any in which there are 6 or more defaults, because in this case, irrespective of the precise number of defaults, he has lost everything.

Such a tranche was often called ‘mezzanine’, since it has intermediate position in the capital structure. A tranche , that suffers the first losses, is called an equity tranche for any , while a tranche positioned at the opposite end, , of the capital structure is called a senior tranche. Also tranches that were intermediate but with sufficiently high attachment and detachment were usually called senior.

The expected loss for an investor depends on the correlation assumed among the default events. Let us consider an investor that has sold protection for a nominal of 100, first on the most equity tranche possible, the , with maturity of 5 years. We suppose that all the mortgages have a probability to default within 5 years, and they have a 1, or , default correlation . In the market standard, that will be fully understood (including its tricky and misleading aspects) in Chapter 3, a default correlation means, in this case, that all mortgages will default together. What is the distribution of the loss in 5 years?

so that the expected loss

If instead we say there is zero default correlation, then the one-hundred default events for the one-hundred mortgages are independent. Now the probability of having zero defaults is , so that

leading to

Take instead a protection sale on the most senior tranche, . Under correlation , the distribution of the loss is

so that

If instead we say there is zero default correlation, now the probability of having 100 defaults is , so

We can notice first that an equity tranche is more risky than a senior tranche. They are the same for correlation, but for all lower levels of correlation the senior tranche is less risky. Then we notice that for equity tranches risk decreases with correlation, while for senior tranches risk is increasing with correlation from almost no risk at 0 correlation up to maximum risk at unit correlation.

Now we give a rough description (improved in Chapter 3) of the market model for these derivatives, trying in particular to explain how this modelling framework allowed regularly to extract from a bunch of mortgages a number of tranches with low risk.

The market model was made up, following the approach of a Gaussian copula, by default probabilities and correlations. The historical approach, favoured by rating agencies, based the correlations on observing past data and extrapolating some conclusions from it. The mapping approach, often used by banks and funds, was based on a modification of today correlations from some reference markets which are very different from and much less risky than the bespoke CDOs to which it was then applied. We will show in 3.5 that this approach, which was supported by mathematical considerations with very little financial justifications, was biased towards underestimating the correlations of subprime CDOs and in general of all CDOs more risky than the reference markets. This bias was not immediate to detect, probably because of the lack of transparency and intuitiveness of the methodology. We have included the unveiling of this model error in Chapter 3 devoted to stress testing of models.

In this section we focus instead on the historical estimation approach, because it was this approach, used by rating agencies, that led to those favourable ratings which were the crucial driver of the market growth. And it was the break-down of this historical approach that then ignited the crisis. The users of this approach took as an input the historical default rates of mortgages, divided into the national rate and the regional rates, which were often rather different from the national one. From these data they could compute the correlation among the default events of the different borrowers. The historical evidence was that subprime borrowers, that are known for being unreliable, defaulted most often for their personal financial problems, with a low dependence on the regional trend of the economy and an even lower one on the national trend. The historical evidence on the default of subprime mortgagers, formally organized as in Remark 1, was the foundation of the tendency to give low correlation to subprime mortgagers, reducing the riskiness of senior tranches in subprime CDO.

In the years preceding the crisis, someone suspected that this model may not be anymore reasonable for the current times. In fact during the first decade of this century the number of subprime mortgages had been increasing, while the losses on them had been low, and this was due to a fact not taken into account by historical data. During the entire decade house prices had been increasing, and the evolution of the financial system had made it easy to perform equity withdrawals, which means the mortgager getting cash from an increase in the price of his house, without selling it. The simplest way for a mortgager to get this is to refinance his debt. If I bought a house for $100.000, using a $100.000 mortgage guaranteed by my house, but after one year my house is worth $150.000, I can go to another bank and get a $150.000 mortgage guaranteed by my house. I can use $100.000 to extinguish the previous mortgage and spend the remaining $50.000, including paying regularly the interest on my mortgage. Clearly at the same time I have also increased my total indebtment, increasing in the long or medium run my risk of default.

Why were banks or mortgage companies happy about this? Again, because of the increasing house prices: mortgage lenders that, with a default, became proprietors of a house with a price higher than the value of the mortgage, and easy to sell, can have a gain and not a loss from the default. This led to an expansion of mortgages, that in turn sustained the increase of house prices on which the mortgage expansion itself was based.

It is clear that the picture was changed by the new situation: now the fact of having losses on mortgages depended crucially on the trend of house prices, since as long as the trend is increasing losses are less likely. This should alter also our reasoning on correlation, since the dependence on a common trend creates stronger correlation. If the real reason that made the market function is the one we described above, a generalized decrease in house prices should first of all create problems to refinance the debt for all mortgagers, increasing the probability that they default together, and secondly, after a default, it increases the probability that these defaults generate losses due to lower house prices. Rating agencies knew this somewhat, but this did not change dramatically their correlation assumptions: the large number of AAA ratings remained. This is justified as follows by Brunnermeier (2009):

‘Many professional investors’ statistical models provided overly optimistic forecasts about structured mortgage products for a couple of reasons: 1) they were based on historically low mortgage default and delinquency rates that arose in a credit environment with tighter credit standards, and 2) past data suggested that housing downturns are primarily regional phenomena—the U.S. had never experienced a nation-wide housing slowdown. The seemingly low cross-regional correlation of house prices generated a perceived diversification benefit that especially boosted the evaluations of AAA-rated tranches.’

The rating agencies followed again historical observations, and they noticed that, at least in the recent past considered, ‘the U.S. had never experienced a nation-wide housing slowdown’. This is the crucial observation, together with the other ‘housing downturns are primarily regional’. House prices had gone down in single states, but then, when looking at the national numbers, the house prices had never decreased during the historical period used for evaluating CDO’s. Thanks to this evidence, the correlation was increased only for mortgagers belonging to the same state, but not for mortgagers living in different states. Since the CDO’s designed by banks tried to pool together names coming as much as possible from different states, the rating agency models gave low correlation to the names in the pool, making senior tranches deserve a high rating.

Thus for the first approach that rating agencies had used in the past correlation of subprime was low since subprime are based mainly on idiosyncratic risk. For the more up-to-date model, that took into account the link between subprime losses and house prices, the crucial implicit assumption justifying low correlation was in assuming that the national house trend can only be increasing, what Oyama (2010) calls the system of loans with real estate collateral based on the myth of ever-increasing prices.²

What happened to this myth in 2007? If you want a more detailed technical account of the modelling framework used by agencies, banks and funds to compute correlations, you can read the following remark. Otherwise, you can get directly to the answer in the next section.

Remark 1. Technical Remark on Factor Models. Rating agencies were using factor models, where the default time of a mortgager happens before the mortgage maturity T in case a standardized Gaussian random variable

is lower than a threshold H,

where is the cumulative probability function of a standardized Gaussian distribution, so that once has been estimated we can say that default happens before maturity when

This model lacks in any real dynamics, in the sense that with such a model one can find only silly answers to questions such as: given that the mortgager has survived until a date T1 in the future, what is the likelihood that he will survive until ? But we will leave this aspect to Chapter 3, when we analyze the liquidity problems that followed the subprime crisis and the difficulties to deal with them using this model. For the time being, we focus on the fact that the variable X is the one through which these models can capture the dependency, and therefore the correlation, between the default times of different mortgagers. They assume that for the mortgager ‘i’ of the state ‘a’ we have a factor X shaped as follows

where γUS is the factor which is common to all mortgagers in the , γa is a term common to only the mortgagers in state a and independent of the US factor, and Yi is an idiosyncratic factor that takes into account the probability that mortgager i defaults independently of the trend of the national or regional economy. The loadings , and are the weights of the three different terms. If we believe that the dependency on the national factor γUS is the crucial one, we are going to have

if instead we believe that the mortgagers usually default for their personal reasons, we are going to set

It is logic to take these three factors independently. In fact if there are links between the different states in the US, this will be captured by a higher weight , while if there is a link between mortgagers in the same state a, this will be captured via a higher weight . Notice that if we take the three factors YUS, Ya and Yi to be all standardized Gaussians , and we set

we have kept the property that Xi is , in fact

and

The interesting thing for us is that this factor model also determines the correlation between the default risk of different mortgagers. In fact for two mortgagers i and j belonging to the same state a we have

while if the two mortgagers belong to two different states a and b we have

The historical evidence was that subprime borrowers had a low dependence on the regional trend of the economy and an even lower one on the national trend. Thus and were low, leading to low default correlation. Then the importance of the trend of house prices became more important: the effect of the national economy on the probability of default of a mortgager was through the possibility that national house prices went down; the effect of the regional economy was through the possibility that house prices in a given state went down. Then there was the residual factor Yi associated to the classic default risk of a subprime: he loses his job and after his default it is difficult to sell his house, independently of the trend of the housing market. Inspired by the above historical evidence, analysts took to be very low, since the national housing trend had always been increasing and could not be a reason for defaults. The dominant factor was the state factor , since state housing trends can turn from increasing to decreasing and a decreasing trend can lead to default. Thus they had a very low correlation for names belonging to different states, and a higher one for names belonging to the same state, getting low correlations for CDOs diversified across states, as most CDOs were. We are back to the crucial question: what happened to the myth of ever-increasing national house prices in 2007?

The strike of reality

We follow Brunnermeier (2009), an early but very accurate account of the preparation and burst of the crisis, along with some other sources, to describe those seemingly minor events in 2007 that had such a strong impact on the future of finance.

An increase in mortgage subprime defaults was registered as early as February 2007, but it seemed a temporary slow-down with no consequences. However, something else happened in March. From the Washington Post on 25 April 2007, we read that sales of homes in March fell 8.4 percent from February, the largest one-month drop since January 1989, when the country was in a recession. Operators tried to play down the relevance of such figures. David Lereah, chief economist for the Realtors group, attributed the downturn partly to bad weather in parts of the country in February that carried over to transactions closed in March.

But there was something more relevant. The median sales price fell to $217,000 in March, from $217,600 in March 2006. It is a very small decrease. But in light of the above analysis it is easy to see just how disquieting it must have appeared to operators in the CDO market. The situation became even worse later, and did not only concern ‘houses’, but real estate in general. Figure 1.1 illustrates the dramatic reversal of the price trend in the crucial sector of commercial property, that also happened around the summer of 2007, with some early signs in the preceding months.

Figure 1.1 The credit crunch is the first example of model consensus collapse that we analyze

Many operators in those days seemed to change their minds about the prospects of the market. UBS shut down their hedge fund focused on subprime investments. Moodys put a number of tranches on downgrade review: while not yet a downgrade, it usually anticipates one. Others tried to carry on as normal. Bear Sterns in mid June injected liquidity to save one of their funds investing in subprime mortgages, that was experiencing liquidity troubles. It is interesting to note that Bear Sterns had no contractual obligation to do this, but acted to save its reputation.

From 25 to 26 July 2007 the CDX 5-year maturity index, a good measure of the average credit risk of US senior corporations, jumped by 19% from 57bp to 68bp. Nor was the reaction limited to the US. The i-Traxx 5-year maturity spreads, an indicator of the confidence of market operators on the credit perspectives of the European economy, jumped from 36bp to 44bp, a 22% increase that was by far the largest one day-jump in its history to date. For the financials sub-index, the situation was even more dramatic: the jumps was from 23bp to 33bp, a jump of 43%. From Monday, 22 July to Friday, 27 July, just one working week, the spread of financials almost tripled, from 23bp to 59bp.

It seems market operators had put two and two together. If house prices go down, mortgage equity withdrawals become difficult and defaults in the subprime markets are doomed to rise. This will result in banks and mutual funds becoming proprietors of the houses of the defaulted mortgages. In a context of falling house sales and falling house prices, this will turn into material losses.

If banks can suffer losses from the default of mortgages, all the mortgage-based derivatives that were sold as virtually risk-free AAA assets will soon become junk. There are so many of them around that the whole financial system will suddenly be in big trouble, banks will have to reduce their lending and this will turn into an increased risk of default for all economic players worldwide. The decrease in national house prices shattered the foundations of a splendid, if fragile, edifice: the economic system built in the first decade of the 21st century.

The first wave of this tide hit the CDS and CDO market. On 31 July American Home Mortgage Investment Corporation announced it was unable to meet its obligations, and it defaulted officially on 6 August. Everything that followed has been recounted in hundreds of books, and we will not reprise it here. We will mention the topic again in Section 3.4, where we explain that after the subprime crisis burst and the initial clustered credit losses, these losses generated liquidity stress and a panic that exacerbated the liquidity stress. There we will show why the Gaussian copula market model is particularly unfit too for the management of the risk of clustered losses, an element that certainly did not help anticipate the real risks associated with CDO investments, nor did it help ease the panic and the liquidity crunch once the crisis had burst. But that’s another story and one told in Chapter 3.³

1.1.4 A Synthetic View on Model Risk

Let us go back to our initial question. What can trigger the market suddenly to abandon a consensus pricing methodology, as happened with the subprime crisis? The analysis of the crisis shows what happened in that case: an event related to the fundamentals was observed. There was a decrease in house prices at national level. This reminded market operators that the model used was crucially based on a hypothesis extremely biased towards an aggressive scenario, that of ever-increasing house prices, that could be macroscopically disproved by evidence. The solidity of the market could be destroyed by a generalized decrease in house prices, a scenario that had previously been considered impossible. Now this scenario was becoming a reality in the housing market. We can say that the crisis burst when an element of unrealism of the model was exposed to be more relevant than previously thought.

Clearly, we are speaking with the benefit of recent hindsight, but the death of a model seen in this crisis is a typical pattern of crisis in regard both to quantitative and qualitative models. The losses in the derivatives world in 1987 were driven in part by the appearance of the skew (a decreasing pattern of implied volatilities when plotted against the option strike), that corresponds to abandoning a Gaussian distribution of returns and replacing it with a distribution where there is more likelihood of large downwards movements of the underlying stock prices. This was clearly driven by the fact that such a large downward movement had just happened in reality, in the stock market crash of Black Monday, as we can see in Figure 1.2. The model change was effected without any sophistication, but simply by moving implied volatilities to patterns inconsistent with the previous Gaussian model, and it was done very fast.

Figure 1.2 Another example of model shift is the 1987 Stock Market crash

Figure 1.3 An example of a model shift triggered also by a piece of research

Even the dot com bubble of the ’90s was sustained by a sort of model, mainly qualitative but with some quantitative implications on simple financial indicators, that predicted a change of paradigm in the economy that should sustain all internet companies in obtaining performances never seen before. When the model was disproved by the reality that internet companies had started to default, the bubble burst.

Another example is the hypothesis of deterministic recovery, usually set at , that was used in pricing credit derivatives before the crisis. When the credit crunch and, in particular Lehman’s default, showed that recoveries with one single digit were quite likely in a time of crisis, there was a move by many players towards stochastic recovery models.

These conclusions are confirmed by an example given by Rebonato (2003) where the consensus was not changed by a crisis, but by a new piece of research. The paper ‘How to throw away a billion dollar’ by Longstaff, Santa-Clara and Schwartz, around the year 2000, pointed out that if the market term structure was driven by more than one factor, then using a one-factor model exposed banks to losses and prevented them from exploiting opportunities for profit (the issue is explained in more detail in Section 2.8.2 and in Chapter 9). The outcry caused by this piece of research was the final blow that made a majority of banks move to models with a higher number of factors. Since the number of factors driving a realistic representation of the term structure behaviour should certainly be higher than one, this market shift can also be associated with the fact that an element of unrealism of the model was exposed to be more relevant than previously thought.

The importance of this factor in the sudden changes of modelling consensus in the previous crises has an interesting consequence. The patterns of model changes show that also for the Price approach the points mentioned by Derman (1996) become important ‘You may have not taken into account all the factors that affect valuation … You may have incorrectly assumed certain stochastic variables can be approximated as deterministic … You may have assumed incorrect dynamics … You may have made incorrect assumptions about relationships’. This means that in the Price approach we also need to understand if a model is sufficiently realistic, or at least if it is sufficiently reasonable and robust to changes in market reality not to expose users to sudden losses as soon as a not particularly unlikely scenario turns out to be true.

This reduces the practical difference between the Price approach and Value approach. The fundamental requirement of the Value approach, that models should give ‘a realistic (or at least plausible) description of the factors that affect the derivative’s value’, is also important in the Price approach. Additionally, if an institution is particularly worried about the reputational side of model losses, losses which are revealed to be based on an unrealistic model are particularly difficult to justify. This is true even when the unrealistic model has been used by the majority of players, as shown by the example on rating agencies set out at the beginning of this chapter.

The Price approach makes the fundamental contribution of pointing out an element that appeared to have been overlooked in classic approaches: the importance of understanding the modelling consensus in the market, which we cannot afford to overlook since derivatives are regularly marked to market. However, notice that this could have been done relatively easily for the CDO market before the crisis, by finding out about Gaussian copula, historical correlations and mapping, and yet if the approach had stopped there, assuming that as long as we are consistent with the market consensus we have no model risk, it would still have led to big losses. Avoiding this requires the second step of the approach, surmising how the model consensus can change, and this makes realism relevant also to the Price approach, as the main element whose lack can trigger a sudden change in the model, and as a blueprint that allows us to make an informed guess about which new model may in due course replace the old one.

On the other hand, classic approaches to model validation, focused only on using realistic rather than consensus models, would make the life of a bank