Scribd Logo
Upload

Welcome to Scribd!

  • 41 views

    The Art of PD Curve Calibration: Dirk Tasche December 15, 2012

    Uploaded by

    karasa1
    This document discusses techniques for calibrating probability of default (PD) curves. It presents a framework for exploring various calibration techniques and evaluating their suitability. The document tests calibration techniques on publicly available agency rating and default data that is typical for these techniques. It finds that the popular technique of "scaling the PD curve" is theoretically questionable and does not perform well on test datasets, while two other techniques called quasi moment matching and scaling the likelihood ratio are both theoretically sound and perform much better on the test datasets.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd

    The Art of PD Curve Calibration: Dirk Tasche December 15, 2012

    Uploaded by

    karasa1
    41 views36 pages
    This document discusses techniques for calibrating probability of default (PD) curves. It presents a framework for exploring various calibration techniques and evaluating their suitability. The document tests calibration techniques on publicly available agency rating and default data that is typical for these techniques. It finds that the popular technique of "scaling the PD curve" is theoretically questionable and does not perform well on test datasets, while two other techniques called quasi moment matching and scaling the likelihood ratio are both theoretically sound and perform much better on the test datasets.

    Original Description:

    Credit risk doc

    Original Title

    ArtPDCrvCalib

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses techniques for calibrating probability of default (PD) curves. It presents a framework for exploring various calibration techniques and evaluating their suitability. The document tests calibration techniques on publicly available agency rating and default data that is typical for these techniques. It finds that the popular technique of "scaling the PD curve" is theoretically questionable and does not perform well on test datasets, while two other techniques called quasi moment matching and scaling the likelihood ratio are both theoretically sound and perform much better on the test datasets.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    41 views36 pages

    The Art of PD Curve Calibration: Dirk Tasche December 15, 2012

    Uploaded by

    karasa1
    This document discusses techniques for calibrating probability of default (PD) curves. It presents a framework for exploring various calibration techniques and evaluating their suitability. The document tests calibration techniques on publicly available agency rating and default data that is typical for these techniques. It finds that the popular technique of "scaling the PD curve" is theoretically questionable and does not perform well on test datasets, while two other techniques called quasi moment matching and scaling the likelihood ratio are both theoretically sound and perform much better on the test datasets.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 36

    The art of PD curve calibration

    Dirk Tasche

    December 15, 2012


    PD curve calibration refers to the task of transforming a set of conditional proba-
    bilities of default (PDs) to another average PD level that is determined by a change
    of the underlying conditional PD. This paper presents a framework that allows to
    explore a variety of calibration techniques and the conditions under which they are
    t for purpose. We test the techniques discussed by applying them to a publicly avail-
    able dataset of agency rating and default statistics that can be considered typical
    for the scope of application of the techniques. We show that the popular technique
    of scaling the PD curve is theoretically questionable and does not perform well on
    the test datasets. We identify two calibration techniques that are both theoretically
    sound and perform much better on the test datasets.
    Keywords: Probability of default, calibration, likelihood ratio, Bayes formula, rat-
    ing prole.
    1. Introduction
    The best way to understand the subject of this paper is to have a glance at table 1 that illustrates
    its main results. Table 1 compares the default rates (column 2) that were observed in 2010 and
    2011 for non-nancial corporates with dierent S&P rating grades (S&P, 2011, 2012) to default
    rate forecasts that were made with four dierent techniques discussed in this paper (columns 3,
    4, 5 and 6). The process of forecasting default rates for the grades of a rating system is called
    calibration, and the resulting forecast default rates are called PD (probability of default) curve.
    Although some forecast techniques (columns 4 and 6) show a better performance in table 1 than
    the others, there is no technique that is better than the others in both forecast years. All forecast
    techniques fail in predicting the rating grades with 0% default rate but as explained below
    this is intended. Table 12 in section 4 (for a dierent universe of S&P rated names) demonstrates
    another deliberate failure of the forecast techniques: While the observed default rates are non-
    monotonic all the forecast default rates are increasing with deteriorating credit quality. This
    again is an intended eect.

    E-mail: [email protected]
    The author currently works at the UK Financial Services Authority. The opinions expressed in this paper are
    those of the author and do not necessarily reect views of the Financial Services Authority.
    1
    a
    r
    X
    i
    v
    :
    1
    2
    1
    2
    .
    3
    7
    1
    6
    v
    1


    [
    q
    -
    f
    i
    n
    .
    R
    M
    ]


    1
    5

    D
    e
    c

    2
    0
    1
    2
    Table 1: 2010 and 2011 grade-level forecast default rates for S&P non-nancial corporates rat-
    ings. P-values are for the
    2
    -tests of the implied default proles. All values in %.
    Default rate Invariant default Invariant AR Scaled PDs Scaled likelihood
    prole (4.10) (4.11) (4.12) ratio (4.13)
    2010: Unconditional default rate 1.423
    AAA 0 0.0005 0.0001 0.0002 0.0001
    AA+ 0 0.001 0.0002 0.0005 0.0003
    AA 0 0.0022 0.0005 0.0011 0.0007
    AA- 0 0.0042 0.0011 0.0021 0.0014
    A+ 0 0.0069 0.002 0.0037 0.0025
    A 0 0.0131 0.0045 0.0075 0.005
    A- 0 0.0262 0.0107 0.0163 0.0108
    BBB+ 0 0.0476 0.0223 0.0317 0.0211
    BBB 0 0.0836 0.046 0.0598 0.0397
    BBB- 0 0.1391 0.0881 0.1062 0.0707
    BB+ 0 0.2011 0.1355 0.1614 0.1076
    BB 0 0.2758 0.196 0.2314 0.1546
    BB- 0 0.4249 0.3175 0.3797 0.255
    B+ 0 0.7616 0.6005 0.7402 0.5036
    B 0.7634 1.6652 1.4378 1.7877 1.2639
    B- 2.0747 4.4114 4.1636 5.1613 4.1758
    CCC-C 21.9895 14.1473 15.7635 13.2909 16.4903
    P-value Exact 12.02 23.46 9.5 29.77
    2011: Unconditional default rate 0.945
    AAA 0 0.0002 0.0001 0.0002 0.0001
    AA+ 0 0.0004 0.0002 0.0004 0.0003
    AA 0 0.0009 0.0004 0.0009 0.0006
    AA- 0 0.0018 0.0009 0.0017 0.0012
    A+ 0 0.0031 0.0016 0.003 0.0021
    A 0 0.0063 0.0035 0.0061 0.0043
    A- 0 0.0134 0.008 0.0132 0.0093
    BBB+ 0 0.0258 0.016 0.0257 0.0182
    BBB 0 0.0477 0.0308 0.0483 0.0343
    BBB- 0 0.0833 0.0572 0.0859 0.0612
    BB+ 0 0.1245 0.0885 0.1305 0.0931
    BB 0 0.1758 0.1252 0.1871 0.1338
    BB- 0 0.2817 0.1979 0.307 0.2208
    B+ 0.2237 0.5327 0.3824 0.5985 0.4361
    B 0.7782 1.2522 1.0389 1.4455 1.0957
    B- 4.2636 3.6293 3.701 4.1734 3.6343
    CCC-C 16.8067 13.1196 15.0199 10.7469 14.5952
    P-value Exact 69.43 91.6 38.71 86.98
    2
    The subject of this paper basically is to explain
    how the forecasts in tables 1 and 12 were made, and
    how the performance of the forecast approaches can be measured and compared.
    The scope of the concepts and techniques described in this paper is not limited to data from
    rating agencies but covers any rating system for which data from an estimation period is avail-
    able. It should, however, be noted that the focus in this paper is on grade-level default rate
    forecasts while the problem of forecasting the unconditional (or portfolio-wide) default rate is
    not considered. Forecasting the unconditional default rate is an econometric problem that is
    beyond the scope of this paper (see Engelmann and Porath, 2012, for an example of how to
    approach this problem).
    This paper appears to be almost unique in that it solely deals with the calibration or recalibration
    of PD curves. Calibration of PD curves is a topic that is often mentioned in the literature but
    mostly only as one aspect of the more general subject of rating model development. For instance,
    Falkenstein et al. (2000) deployed the approach that is called invariant PDs in table 1 without
    any comment about why they considered it appropriate. There are, however, some authors
    who devoted complete articles or book sections to PD curve calibration. Van der Burgt (2008)
    suggested a predecessor of the technique that is called quasi moment matching (QMM) in this
    paper. Bohn and Stein (2009, chapter 4) discussed the conceptual and practical dierences
    between the scaled PDs and invariant likelihood techniques. More recently, Konrad (2011)
    investigated in some detail the interplay between the calibration and the discriminatory power
    of rating models.
    In this paper, we revisit the concept of two calibration steps as used by Bohn and Stein (2009).
    According to Bohn and Stein (2009) the two steps are a consequence of the fact that usually
    the rst calibration of a rating model is conducted on a training sample in which the proportion
    of good and bad might not be representative of the live portfolio. The second calibration step,
    therefore, is needed to adjust the calibration to the right proportion of good and bad.
    We argue more generally that the two steps actually relate to dierent time periods (the esti-
    mation and the forecast periods) which both can be described by the same type of one-period
    model. This view encompasses both the situation where a new rating model is calibrated and the
    situation where an existing rating model undergoes a possibly periodic recalibration. The es-
    timation period is used to estimate the model components that are assumed to be invariant (i.e.
    unchanged) or in a specic way transformed between the estimation and the forecast periods.
    Calibration techniques for the forecast period are essentially determined by the assumptions of
    invariance between the periods.
    The roadmap for this paper is as follows:
    In the following section 2, we present a dataset that is typical as a starting point for
    the sort of calibration we are going to consider in the remainder of the paper. We also
    discuss the constraints to which the calibration approach must be subject and explore their
    appropriateness with statistical tests.
    Section 3 provides the theoretical framework that is needed for dealing with the technical
    aspects of PD curve calibration. In particular, we derive a result (proposition 3.2) on the
    3
    characterisation of the one-period model by unconditional rating prole and likelihood
    ratio. This result proves useful for the identication of proper and accurate calibration
    techniques.
    In section 4 we describe a variety of techniques for PD curve calibration and examine
    their performance with a real data example (the S&P data presented in section 2). The
    suitability of the techniques described depends strongly upon what data (e.g. the uncon-
    ditional rating prole) can be observed at the time when the forecast exercise takes place.
    We therefore discuss the dierent possibilities and assumptions in some detail. It turns out
    that the popular scaling the PD curve approach is both theoretically questionable and
    not very well performing on the example dataset. We identify two techniques (quasi mo-
    ment matching and scaling the likelihood ratio) that are theoretically sound and perform
    much better when deployed for the numerical example.
    Section 5 concludes the paper.
    2. Data and context
    All numerical examples in this paper are based on the S&P rating and default statistics for
    all corporates and non-nancial corporates only as presented in table 2. Only with their 2009
    default data report S&P (2010) began to make information on modied-grade level issuer num-
    bers readily available. Without issuer numbers, however, there is not sucient information to
    calculate rating proles and conduct goodness-of-t tests for rating proles. This explains why
    we only look at default statistics from 2009 onwards. For the purposes of this paper, data from
    Moodys is less suitable because Moodys do not provide issuer numbers at alphanumeric grade
    level and apply a special treatment to withdrawn ratings that creates undesirable uncertainty
    with regard to samples sizes (Moodys, 2012).
    We chose to look at S&Ps non-nancial corporates default statistics (lower panel of table 2) as
    an example of a nice dataset where the observed default rates increase with deteriorating credit
    quality. S&Ps all corporates default statistics (upper panel of table 2) represent an example of a
    more interesting, somewhat problematic dataset because it includes some instances of inversions
    of observed default rates. Inversion of default rates means that the default rate observed for a
    better rating grade is higher than the default rate of the adjacent worse rating grade.
    2.1. Default rate inversions
    Table 3 shows that for non-nancial corporates the grade-level default rates recorded by S&P for
    2009, 2010 and 2011 increase with deteriorating credit quality as one would expect. In contrast,
    there are a number of inversions in the all corporates columns of the table, i.e. there are some
    counter-intuitive examples of adjacent rating grades where the less risky grade has a higher
    default rate than the adjacent riskier grade. Notable for this phenomenon is, in particular, the
    pair of BBB- and BB+ in 2009 with 1.09% defaults in BBB- and 0% defaults in BB+.
    Should we conclude from the existence of such inversions that there is a problem with the rank-
    4
    Table 2: S&Ps corporate ratings and defaults in 2009, 2010 and 2011. Sources: S&P (2010,
    tables 51 to 53), S&P (2011, tables 50 to 52), S&P (2012, tables 50 to 52).
    2009 2010 2011
    Rating grade # rated # defaults # rated # defaults # rated # defaults
    All corporates
    AAA 81 0 72 0 51 0
    AA+ 37 0 25 0 36 0
    AA 188 0 143 0 120 0
    AA- 245 0 209 0 207 0
    A+ 340 1 353 0 357 0
    A 510 2 474 0 470 0
    A- 546 0 528 0 560 0
    BBB+ 498 2 457 0 473 0
    BBB 541 1 583 0 549 0
    BBB- 459 5 430 0 508 1
    BB+ 266 0 254 2 260 0
    BB 295 3 276 1 319 0
    BB- 441 4 379 2 403 0
    B+ 438 24 393 0 509 2
    B 482 48 436 3 586 7
    B- 303 52 290 6 301 12
    CCC-C 190 92 220 49 138 22
    All 5860 234 5522 63 5847 44
    Non-nancial corporates
    AAA 15 0 12 0 13 0
    AA+ 13 0 8 0 6 0
    AA 57 0 49 0 41 0
    AA- 62 0 50 0 46 0
    A+ 91 0 84 0 85 0
    A 215 0 197 0 205 0
    A- 283 0 275 0 274 0
    BBB+ 301 0 277 0 301 0
    BBB 370 0 392 0 374 0
    BBB- 310 0 287 0 357 0
    BB+ 205 0 184 0 193 0
    BB 240 2 222 0 249 0
    BB- 355 4 296 0 327 0
    B+ 389 20 341 0 447 1
    B 436 43 393 3 514 4
    B- 259 50 241 5 258 11
    CCC-C 167 86 191 42 119 20
    All 3768 205 3499 50 3809 36
    5
    Table 3: S&P grade-level default rates for corporates in 2009, 2010 and 2011. Sources: S&P
    (2010, tables 51 to 53), S&P (2011, tables 50 to 52), S&P (2012, tables 50 to 52). All
    values in %.
    All corporates Non-nancial corporates
    Rating grade 2009 2010 2011 2009 2010 2011
    AAA 0.00 0.00 0.00 0.00 0.00 0.00
    AA+ 0.00 0.00 0.00 0.00 0.00 0.00
    AA 0.00 0.00 0.00 0.00 0.00 0.00
    AA- 0.00 0.00 0.00 0.00 0.00 0.00
    A+ 0.29 0.00 0.00 0.00 0.00 0.00
    A 0.39 0.00 0.00 0.00 0.00 0.00
    A- 0.00 0.00 0.00 0.00 0.00 0.00
    BBB+ 0.40 0.00 0.00 0.00 0.00 0.00
    BBB 0.18 0.00 0.00 0.00 0.00 0.00
    BBB- 1.09 0.00 0.20 0.00 0.00 0.00
    BB+ 0.00 0.79 0.00 0.00 0.00 0.00
    BB 1.02 0.36 0.00 0.83 0.00 0.00
    BB- 0.91 0.53 0.00 1.13 0.00 0.00
    B+ 5.48 0.00 0.39 5.14 0.00 0.22
    B 9.96 0.69 1.19 9.86 0.76 0.78
    B- 17.16 2.07 3.99 19.31 2.07 4.26
    CCC-C 48.42 22.27 15.94 51.50 21.99 16.81
    All 3.99 1.14 0.75 5.44 1.43 0.95
    ordering capacity of the rating methodology? The long-run average grade-level default rates
    reported by S&P (2012, Table 23) suggest that the observation of inversions as in Table 3 might
    be an exception. Fishers exact test (Fisher, 1922; Casella and Berger, 2002, Example 8.3.30)
    allows to verify this explanation.
    The test can be described as follows:
    Let Y
    i
    , i = 1, 2 be independent and binomially distributed random variables with sizes n
    i
    ,
    i = 1, 2 and success probabilities p
    i
    , i = 1, 2.
    The Null hypothesis is p
    1
    p
    2
    , against the alternative p
    1
    > p
    2
    .
    Under the Null hypothesis the distribution of Y
    1
    conditional on Y
    1
    +Y
    2
    is hypergeometric:
    P[Y
    1
    = y
    1
    | Y
    1
    +Y
    2
    = y] =
    _
    n
    1
    y
    1
    __
    n
    2
    yy
    1
    _
    _
    n
    1
    +n
    2
    y
    _
    . (2.1a)
    Assume that y
    1
    successes were observed for Y
    1
    and y successes for Y
    1
    and Y
    2
    together.
    6
    Table 4: p-values for Fishers exact test of 2009 all corporates PD of better grade was less than
    or equal to PD of worse grade against PD of better grade was greater than PD of
    worse grade. Data are the rating and default numbers from columns 2 and 3 in table 2.
    Null hypothesis Alternative p-value
    PD(AAA) PD(AA+) PD(AAA) > PD(AA+) 1
    PD(AA+) PD(AA) PD(AA+) > PD(AA) 1
    PD(AA) PD(AA-) PD(AA) > PD(AA-) 1
    PD(AA-) PD(A+) PD(AA-) > PD(A+) 1
    PD(A+) PD(A) PD(A+) > PD(A) 0.785
    PD(A) PD(A-) PD(A) > PD(A-) 0.233
    PD(A-) PD(BBB+) PD(A-) > PD(BBB+) 1
    PD(BBB+) PD(BBB) PD(BBB+) > PD(BBB) 0.469
    PD(BBB) PD(BBB-) PD(BBB) > PD(BBB-) 0.991
    PD(BBB-) PD(BB+) PD(BBB-) > PD(BB+) 0.101
    PD(BB+) PD(BB) PD(BB+) > PD(BB) 1
    PD(BB) PD(BB-) PD(BB) > PD(BB-) 0.582
    PD(BB-) PD(B+) PD(BB-) > PD(B+) 1
    PD(B+) PD(B) PD(B+) > PD(B) 0.996
    PD(B) PD(B-) PD(B) > PD(B-) 0.999
    PD(B-) PD(CCC-C) PD(B-) > PD(CCC-C) 1
    Reject then the Null hypothesis at level if p-value, where
    p-value = P[Y
    1
    y
    1
    | Y
    1
    +Y
    2
    = y]
    =
    min(n
    1
    ,y)

    k=y
    1
    _
    n
    1
    k
    __
    n
    2
    yk
    _
    _
    n
    1
    +n
    2
    y
    _
    .
    (2.1b)
    In order to apply Fishers test to the S&P data we revisit table 2. The results of the 16 tests
    applied are presented in table 4. For instance, to obtain the p-value of 10.1% for the test of PD
    of BBB- is less than or equal to PD of BB+ against the alternative PD of BBB- is greater than
    PD of BB+ we apply formula (2.1b) with n
    1
    = 459, n
    2
    = 266, y
    1
    = 5 and y = 5. As a matter
    of fact, 10.1% is the least p-value reported in table 4. Hence, we cannot conclude that there
    are statistically signicant inversions in the grade-level all corporates default rates although the
    inversion involving BBB- and BB+ is nearly signicant at 10% type-I error level. But even if
    this inversion had been signicant we should not have been surprised because when 16 tests are
    conducted there is a high probability that at least one of them will erroneously indicate rejection
    of the Null hypothesis.
    As a conclusion from the discussion of default rate conversions we will assume for the remainder
    of the paper that the PD curve associated with the corporate S&P ratings can be modelled as
    a monotonic curve.
    7
    Table 5: S&P rating proles for corporates at the beginning of 2009, 2010 and 2011. Sources:
    S&P (2010, tables 51 to 53), S&P (2011, tables 50 to 52), S&P (2012, tables 50 to 52)
    and own calculations. All values in %.
    All corporates Non-nancial corporates
    Rating grade 2009 2010 2011 2009 2010 2011
    AAA 1.38 1.30 0.87 0.40 0.34 0.34
    AA+ 0.63 0.45 0.62 0.35 0.23 0.16
    AA 3.21 2.59 2.05 1.51 1.40 1.08
    AA- 4.18 3.78 3.54 1.65 1.43 1.21
    A+ 5.80 6.39 6.11 2.42 2.40 2.23
    A 8.70 8.58 8.04 5.71 5.63 5.38
    A- 9.32 9.56 9.58 7.51 7.86 7.19
    BBB+ 8.50 8.28 8.09 7.99 7.92 7.90
    BBB 9.23 10.56 9.39 9.82 11.20 9.82
    BBB- 7.83 7.79 8.69 8.23 8.20 9.37
    BB+ 4.54 4.60 4.45 5.44 5.26 5.07
    BB 5.03 5.00 5.46 6.37 6.34 6.54
    BB- 7.53 6.86 6.89 9.42 8.46 8.58
    B+ 7.47 7.12 8.71 10.32 9.75 11.74
    B 8.23 7.90 10.02 11.57 11.23 13.49
    B- 5.17 5.25 5.15 6.87 6.89 6.77
    CCC-C 3.24 3.98 2.36 4.43 5.46 3.12
    All 100.00 100.00 100.00 100.00 100.00 100.00
    2.2. Pearsons chi-squared test for count data
    A question of similar importance for the estimation of PD curves is the question of whether or
    not the unconditional (or all-portfolio) rating prole (i.e. the distribution of the rating grades)
    of a portfolio can be assumed to be unchanged over time. Table 5 shows the unconditional rating
    proles of all corporates and the non-nancial corporates only for the three years for which there
    are publicly available S&P data. It is hard to assess from the percentages whether the proles
    are broadly invariant during the three years and whether or not the proles of all corporates
    and the non-nancial corporates only are identical up to random dierences. We use Pearsons
    -squared test for count data (Pearson, 1900) to answer these questions and also to assess the
    accuracy of the forecast techniques discussed in the remainder of the paper.
    Description of the -squared test. Assume that on a set {1, . . . , k} of k > 1 classes we have
    made a total of n > 0 independent observations which each falls into one of the classes. Hence
    8
    Table 6: -squared tests for identity of rating proles from table 5. All p-values have been
    calculated by -squared approximation.
    Class probabilities Observed frequencies p-value
    Corporate prole 2009 Corporates 2010 0.000616
    Corporate prole 2009 Corporates 2011 < 10
    10
    Corporate prole 2010 Corporates 2011 < 10
    10
    Non-nancial prole 2009 Non-nancials 2010 0.09314
    Non-nancial prole 2009 Non-nancials 2011 6.3 10
    7
    Non-nancial prole 2010 Non-nancials 2011 < 10
    10
    Corporate prole 2009 Non-nancials 2009 < 10
    10
    Corporate prole 2010 Non-nancials 2010 < 10
    10
    Corporate prole 2011 Non-nancials 2011 < 10
    10
    there are n
    i
    observations with class i and it holds that
    n =
    k

    i=1
    n
    i
    . (2.2a)
    We want to test the hypothesis that the observations (n
    1
    , . . . , n
    k
    ) are the realisation of a multi-
    nomial distribution with class probabilities q
    1
    > 0, . . ., q
    k1
    > 0, q
    k
    = 1

    k1
    i=1
    q
    i
    > 0. Pearson
    suggested the test statistic
    T =
    k

    i=1
    (n
    i
    nq
    i
    )
    2
    nq
    i
    . (2.2b)
    It is well known that for large n the statistic T is approximately -squared distributed with
    k 1 degrees of freedom (see, e.g., van der Vaart, 1998, Theorem 17.2). As to the question of
    when the approximation may be applied, it is common practice to use it when all expected class
    frequencies under the hypothesis are greater than or equal to ve, i.e.
    nq
    i
    5 for all i = 1, . . . , k. (2.2c)
    If this condition is violated we can still approximate the distribution of T by Monte-Carlo sim-
    ulation. For the purpose of this paper, we ran the Monte-Carlo simulation with 5,000 iterations
    when we could not apply the -squared approximation.
    Table 6 shows that
    the all corporates rating proles change signicantly over the years,
    in all three years the all corporates and the non-nancial corporates rating proles are
    signicantly dierent,
    also the non-nancial corporates rating proles change a lot over the years. Only the 2009
    and 2010 proles can be seen as statistically indistinguishable at 5% type-I error level.
    9
    2.3. Consequences for the calibration of PD curves
    From the initial data analyses in sections 2.1 and 2.2 we can draw two conclusions:
    Forcing monotonicity of estimated PD curves can make sense if it is justied by statistical
    tests or long-run average evidence.
    In general, we cannot assume that the rating prole of a portfolio does not change over
    time, even if random uctuation is ignored. However, this assumption can be veried or
    proven wrong with statistical tests. As we will see in the section 4, depending on the
    outcome of the tests there are dierent options for the forecast of PD curves.
    Although never a default of an AAA-rated corporate was observed within one year after having
    been rated AAA (Moodys, 2012; S&P, 2012) we should nonetheless try and infer a positive
    one-year PD for AAA. For the possibility of default technically is not excluded even for AAA
    rated (or any other investment grade) entities. In particular, even if we may assume that the
    rating agencies will not allow AAA rated companies to default without having down-graded
    them in time it might happen that an AAA companys management for whatever reason decides
    to have the company defaulting.
    This is why in the following we restrict ourselves to only deploy PD curve estimation techniques
    that guarantee to deliver positive PDs for all rating grades.
    On the basis of the data presented in this section, it is also worthwhile to clarify precisely the
    concept of a two-step (or two-periods) approach to the calibration of a rating model as mentioned
    by Bohn and Stein (2009): The rst period is the estimation period, the second period is the
    calibration and forecast period. The two periods are determined by their start and end dates
    and the observation and estimation horizon:
    h is the horizon for the PD estimation, i.e. a borrowers PD at date T gives the probability
    that the borrower will default within T and T +h.
    The start date T
    0
    of the estimation period is a date in the past.
    T
    1
    T
    0
    + h is the date when the calibration or recalibration of the rating model takes
    place. The calibration is for the current portfolio of borrowers whose ratings at T
    1
    should
    be known but whose future default status at T
    1
    +h is still unknown.
    The end date T
    2
    = T
    1
    + h of the forecast period is in the future. Then the default status
    of the borrowers in the current portfolio will be known.
    With regard to the two-periods concept for calibration, for the remainder of the paper we make
    the following crucial assumptions:
    For the sample as of date T
    0
    everything is known:
    The unconditional rating prole at T
    0
    ,
    the conditional rating proles (i.e. conditional on default and conditional on survival
    respectively) at T
    0
    ,
    the unconditional (base) PD for the time interval between T
    0
    and T
    0
    +h,
    the conditional PDs (i.e. conditional on the rating grades at T
    0
    for the time interval
    10
    between T
    0
    and T
    0
    +h.
    At date T
    1
    could be known:
    The unconditional rating prole.
    A forecast of the unconditional (base) default rate. In general this will be dierent
    from the unconditional PD for the estimation period between T
    0
    and T
    0
    +h.
    We will use the rating and default data for 2009 from table 2 as an example for the estimation
    period (i.e. h = 1 year, T
    0
    = January 1, 2009, T
    0
    + h = December 31, 2009). We will consider
    both 2010 and 2011 as examples of one-year forecast periods based on the estimation of a model
    for 2009 (i.e. T
    1
    = January 1, 2010 or T
    1
    = January 1, 2011). The gap of one year between the
    estimation period 2009 and the forecast period 2011 reects the gap that is likely to occur in
    practice when the full data from the estimation period usually become available only a couple
    of months after the end of the period.
    3. One-period mechanics
    This section describes a one-period statistical model of a borrowers beginning of the period
    rating grade and end of the period state of solvency. This model is applicable to both the
    estimation and the forecast periods as discussed in section 2.3. In particular, we will consider
    the following model characteristics and their relationships:
    Unconditional rating distribution (prole).
    Conditional (on default and survival) rating distributions (proles).
    Unconditional PD.
    PD curve (PDs conditional on rating grades).
    Accuracy ratio (discriminatory power).
    Likelihood ratio.
    We rely on the standard model used for topics like pattern recognition, medical diagnoses,
    or signal detection (see, e.g., van Trees, 1968). Hand (1997) presents a variety of applications
    (including credit scoring) for this type of model.
    Speaking in technical terms, in this paper we study the joint distribution and some estimation
    aspects of a pair (X, S) of random variables. The variable X is interpreted as the rating grade
    1
    assigned to a solvent borrower at the beginning of the observation period. Hence X typically
    takes on values on a discrete scale in a nite set which we describe without loss of generality as
    {1, 2, . . . , k}. This implies that the marginal distribution of X is characterised by the probabilities
    Pr[X = x], x = 1, . . . , k, which we call the unconditional rating prole.
    1
    In practice, often a rating model with a small nite number of grades is derived from a score function with
    values on a continuous scale. This is usually done by mapping score intervals on rating grades. See Tasche
    (2008, section 3) for a discussion of how such mappings can be dened. Discrete rating models are preferred by
    practitioners because manual adjustment of results (overrides) is feasible. Moreover, results by discrete rating
    models tend to be more stable over time.
    11
    Convention: Low values of X indicate low creditworthiness (bad), high values of X indicate
    high creditworthiness (good).
    The variable S is the borrowers state of solvency at the end of the observation period, typically
    one year after the rating grade was observed. S takes on values in {0, 1}. The meaning of S = 0
    is borrower has remained solvent (solvency or survival), S = 1 means borrower has become
    insolvent (default). In particular, S is always observed with a time lag to the observation of X.
    Hence, when S is observed X is already known but when X is observed today S is still unknown.
    We write D for the event {S = 1} and N for the event {S = 0}. Hence
    D N = {S = 1} {S = 0} = , D N = whole space. (3.1)
    The marginal distribution of the state variable S is characterised by the unconditional probability
    of default p which is dened as
    p = Pr[D] = Pr[S = 1] [0, 1]. (3.2)
    p is sometimes also called base probability of default. In the following we assume 0 < p < 1 as
    the cases p = 0 and p = 1 are not of practical relevance.
    3.1. Model specication
    Recall that the two marginal distributions of X and S respectively do not uniquely determine
    the joint distribution of X and S. For easy reference we state in the following proposition the
    three equivalent standard ways to characterise the joint distribution.
    Proposition 3.1 The joint distribution of the pair (X, S) of the rating variable X and the state
    of the borrower variable S is fully specied in either of the following three ways:
    (i) By the joint probabilities
    Pr[X = x, S = 0] = Pr[{X = x} N], x = 1, . . . , k, and
    Pr[X = x, S = 1] = Pr[{X = x} D], x = 1, . . . , k.
    (3.3a)
    (ii) By the unconditional PD p = Pr[D] = 1 Pr[N] and the distributions of X conditional on
    D and N respectively:
    Pr[X = x| D] =
    Pr[{X = x} D]
    p
    , x = 1, . . . , k, and
    Pr[X = x| N] =
    Pr[{X = x} N]
    1 p
    , x = 1, . . . , k.
    (3.3b)
    x Pr[X = x| D] and x Pr[X = x| N] are called the conditional rating proles
    (conditional on default and survival respectively).
    (iii) By the unconditional rating prole x Pr[X = x] and the conditional PDs
    Pr[D| X = x] =
    Pr[{X = x} D]
    Pr[X = x]
    , x = 1, . . . , k. (3.3c)
    x Pr[D| X = x] is called the PD curve associated with the grades x = 1, . . . , k.
    12
    For further reference we note how the specication of the joint distribution of (X, S) given in
    proposition 3.1 (ii) implies the representation provided in proposition 3.1 (iii):
    By the law of total probability, the unconditional rating prole Pr[X = x], x = 1, . . . , k
    can be calculated as
    Pr[X = x] = p Pr[X = x| D] + (1 p) Pr[X = x| N]. (3.4a)
    Bayes formula implies the following representation of the PD curve Pr[D| X = x]:
    Pr[D| X = x] =
    p Pr[X = x| D]
    p Pr[X = x| D] + (1 p) Pr[X = x| N]
    . (3.4b)
    Also for further reference, we observe how the specication of the joint distribution of (X, S)
    given in proposition 3.1 (iii) implies the representation provided in proposition 3.1 (ii):
    Again by the law of total probability, the unconditional PD p can be calculated as
    p =
    k

    x=1
    Pr[D| X = x] Pr[X = x]. (3.5a)
    With regard to the conditional rating proles, it follows directly from the denition of
    conditional probability that
    Pr[X = x| D] = Pr[D| X = x] Pr[X = x] / p, and (3.5b)
    Pr[X = x| N] =
    _
    1 Pr[D| X = x]
    _
    Pr[X = x] / (1 p). (3.5c)
    The equivalence between equations (3.4a) and (3.4b) on the one hand and equations (3.5a) and
    (3.5b), (3.5c) on the other hand allows the calculation of one set of characteristics once the
    other set of characteristics is known. But the equivalence also represents a consistency condition
    that must be kept in mind if one of the characteristics is changed. In particular, if for a given
    unconditional rating prole there are independent estimates of the unconditional PD and the
    PD curve equation, (3.5a) becomes a crucial consistency condition.
    In this paper, we use quasi moment matching (QMM) as described in appendix B to transform
    the grade-level empirical default rates into smoothed PD curves. As mentioned in section 2.3,
    such smoothing of the empirical PD curve is needed in order to
    force monotonicity of the PD curve and
    force the PDs to be positive.
    Matching in this context means tting a two-parameter curve to the empirically observed un-
    conditional default rate and discriminatory power. The discriminatory power is measured as
    accuracy ratio whose general formula is given in (B.4a). Using the conditional rating proles
    dened by (3.3b) the accuracy ratio can also be described by
    AR =
    k

    x=2
    Pr[X = x| N] Pr[X x 1 | D]
    k1

    x=1
    Pr[X = x| N] Pr[X x + 1 | D]. (3.6)
    13
    3.2. Likelihood ratio
    We will see in section 4 that the specication of the one-period model by unconditional rating
    prole and PD curve (see proposition 3.1 (iii)) is unsatisfactory if we want to combine a forecast
    period prole with an estimation period PD curve. For according to equations (3.4a) and (3.4b)
    both components depend upon the unconditional PD which might be dierent in the estimation
    and forecast periods. The likelihood ratio is a concept closely related to the PD but avoids the
    issue of dependence on the unconditional PD.
    In the context of credit ratings, it can be reasonably assumed that all components of the condi-
    tional rating proles Pr[X = x| D] and Pr[X = x| N], x = 1, . . . , k are positive. For otherwise,
    there would be rating grades with sure predictions of default and survival which is unlikely to
    happen with real-world rating models. We can therefore dene the likelihood ratio associated
    with the rating model:
    (x) =
    Pr[X = x| N]
    Pr[X = x| D]
    , x = 1, . . . , k. (3.7)
    The likelihood ratio (x) species how much more (or less) likely it is for a survivorss rating
    grade to come out as x than for a defaulters rating grade. Observe that (3.4b) can be rewritten
    as
    Pr[D| X = x] =
    p
    p + (1 p) (x)
    , x = 1, . . . , k. (3.8a)
    This is equivalent to an alternative representation of the likelihood ratio:
    (x) =
    1 Pr[D| X = x]
    Pr[D| X = x]
    p
    1 p
    , x = 1, . . . , k. (3.8b)
    By (3.8b), the likelihood ratio can alternatively be described as the ratio of the grade x odds of
    survival and the unconditional odds of survival. By (3.5b), (3.8a) also implies
    Pr[X = x| D] =
    Pr[X = x]
    p + (1 p) (x)
    , x = 1, . . . , k, (3.9a)
    and, by taking the sum of all Pr[X = x| D]
    1 =
    k

    x=1
    Pr[X = x]
    p + (1 p) (x)
    . (3.9b)
    This observation suggests that the information borne by the likelihood ratio is very closely
    related to the information inherent in the PD curve. More specically, we obtain the following
    characterisation of (3.9b) which is basically the likelihood ratio version of (3.5a).
    Proposition 3.2 Let
    x
    > 0, x = 1, . . . , k be a probability distribution. Assume that x (x)
    is a positive function for x = 1, . . . , k. Consider the equation
    k

    x=1

    x
    p + (1 p) (x)
    = 1. (3.10a)
    Then with regard to solutions p [0, 1] of (3.10a) other than p = 1 the following statements
    hold:
    14
    Figure 1: Illustration for the proof of proposition 3.2. The three possibilities for the shape of the
    graph of the function dened by (3.11a).
    0.0 0.2 0.4 0.6 0.8 1.0
    0
    .
    0
    0
    .
    5
    1
    .
    0
    1
    .
    5
    2
    .
    0
    p
    f
    (
    p
    )
    Case A
    Case B
    Case C
    (i) Assume that x (x) is a mapping onto a constant, i.e. (x) = for all x = 1, . . . , k.
    Then all p [0, 1] are solutions of (3.10a) if = 1 and there is no solution p [0, 1) if
    = 1.
    (ii) Assume that x (x) is not a mapping onto a constant. Then there exists a solution
    p [0, 1) of (3.10a) if and only if
    k

    x=1

    x
    (x)
    1 and
    k

    x=1

    x
    (x) > 1. (3.10b)
    If there exists a solution p [0, 1) of (3.10a) then this solution is unique. The unique
    solution is p = 0 if and only if
    k

    x=1

    x
    (x)
    = 1. (3.10c)
    Proof. Statement (i) is obvious. With regard to statement (ii), dene the function f : [0, 1]
    15
    (0, ) by
    f(p) =
    k

    x=1

    x
    p + (1 p) (x)
    . (3.11a)
    Observe that f is twice continuously dierentiable in p with
    f

    (p) =
    k

    x=1
    ((x) 1)
    x
    (p + (1 p) (x))
    2
    and (3.11b)
    f

    (p) = 2
    k

    x=1
    ((x) 1)
    2

    x
    (p + (1 p) (x))
    3
    . (3.11c)
    From (3.11a) and (3.11b) we obtain
    f(0) =
    k

    x=1

    x
    (x)
    , f(1) = 1, and f

    (1) =
    k

    x=1

    x
    (x) 1. (3.11d)
    (3.11c) implies f

    (p) > 0 because (x) is not constant by assumption. Hence f is strictly convex
    in p. The strict convexity of f implies that the shape of the graph of f is determined by (3.11d)
    and that only the following three cases can occur:
    case A: f(0) < 1 and f

    (1) > 0,
    case B: f(0) 1 and f

    (1) > 0, or
    case C: f(0) > 1 and f

    (1) 0.
    A stylised illustration of the three dierent possible shapes of the graph of f is shown in gure 1.
    Only in case B there is a second (and only one) intersection at a p < 1 of the horizontal line
    through 1. By (3.11d), case B is equivalently described by (3.10b). The second intersection of
    the horizontal line through 1 occurs if and only if f(0) = 1 which is equivalent to (3.10c). q.e.d.
    At rst glance, proposition 3.2 might appear as an unnessecarily complicated way to describe
    the interplay of unconditional rating prole, likelihood ratio, and unconditional PD. However,
    proposition 3.2 becomes interesting when we try to construct the joint distribution of a bor-
    rowers rating X at the beginning of the observation period and the borrowers state S at the
    end of the period from an unconditional rating prole and a candidate likelihood ratio (which
    might have been estimated separately). In this context, proposition 3.2 tells us that the con-
    struction will work only if condition (3.10b) is satised. In contrast, by proposition 3.1 (iii) the
    construction is always possible if one combines an unconditional rating prole with a candidate
    PD curve (assuming that all its components take values between 0 and 1).
    Actually, from proposition 3.2 it is not yet clear that it gives indeed rise to a fully specied joint
    distribution of rating X and default or survival state S. This, however, is conrmed by the next
    proposition.
    Proposition 3.3 Let
    x
    > 0, x = 1, . . . , k be a probability distribution. Assume that x (x)
    is a positive function for x = 1, . . . , k and that equation (3.10a) has a solution 0 < p < 1. Then
    there exists a unique joint distribution of X and S such that x (x) is the likelihood ratio
    associated with the joint distribution in the sense of equation (3.7).
    16
    Table 7: Unconditional default rates and accuracy ratios for the 2009 all corporate and non-
    nancial corporate data from table 2.
    Sample Default rate Accuracy ratio
    All corporates 2009 3.99% 82.7%
    Non-nancial corporates 2009 5.44% 83.8%
    Proof. Dene
    Pr[D] = p,
    Pr[X = x| D] =

    x
    p + (1 p) (x)
    , x = 1, . . . , k,
    Pr[X = x| N] =

    x
    (x)
    p + (1 p) (x)
    , x = 1, . . . , k.
    (3.12)
    Then both Pr[X = x| D] and Pr[X = x| N] are proper probability distributions on {1, . . . , k}.
    Hence, by proposition 3.1 there is a unique distribution of X and S such that Pr[X = x| D] and
    Pr[X = x| N] are its conditional rating proles and p is its unconditional PD. By construction,
    (3.7) holds. q.e.d.
    3.3. Estimation period example
    In this section, we illustrate the concepts introduced in sections 3.1 and 3.2 by revisiting the
    S&P data for 2009 presented in section 2. For the sake of a more convenient notation we map
    the S&P rating symbols CCC-C, B-, B, . . ., AA+, AAA onto the numbers 1, . . . , 17 (hence grade
    17 stands for the least risky grade AAA).
    Columns 2 and 5 of table 5 show the unconditional rating proles x Pr[X = x], {1, . . . , 17}
    [0, 1] for the all corporates and non-nancial corporates samples respectively.
    Table 7 shows the empirical unconditional default rates and accuracy ratios for our estimation
    data (i.e. the 2009 S&P data). The accuracy ratios were calculated according to (3.6).
    Table 8 presents both the empirically observed grade-level default rates and the smoothed PD
    curves (according to appendix B, with the values from table 7 as targets) for the 2009 S&P
    data. It is hard to assess directly from the numbers how well or badly the smoothed curves t
    the empirical data. Therefore we derive implied default proles and compare them by means of
    a -squared test to the observed default proles.
    Table 9 shows the empirically observed and implied default proles for the 2009 S&P data.
    Equation (3.5b) was applied to calculate the implied proles. Clearly the t is worse for the all
    corporates data. This impression is conrmed by application of the -squared test as described
    in section 2.2. To apply the test, choose the implied proles as Null hypothesis distributions and
    test against the default numbers for 2009 tabulated in the upper and lower panels of table 2.
    The test results (based on Monte-Carlo approximation) are a p-value of 9% for the t of the
    implied all corporates default prole and a p-value of 23% for the t of the implied non-nancial
    17
    Table 8: Grade-level default rates and smoothed conditional PDs (PD curves) for the 2009 all
    corporate and non-nancial corporate data from table 2. All numbers in %.
    All corporates Non-nancial corporates
    Rating grade Default rate Smoothed PD Default rate Smoothed PD
    AAA 0.000 0.003 0.000 0.001
    AA+ 0.000 0.006 0.000 0.002
    AA 0.000 0.012 0.000 0.004
    AA- 0.000 0.025 0.000 0.009
    A+ 0.294 0.047 0.000 0.015
    A 0.392 0.091 0.000 0.031
    A- 0.000 0.173 0.000 0.068
    BBB+ 0.402 0.299 0.000 0.132
    BBB 0.185 0.495 0.000 0.249
    BBB- 1.089 0.797 0.000 0.443
    BB+ 0.000 1.138 0.000 0.673
    BB 1.017 1.518 0.833 0.965
    BB- 0.907 2.280 1.127 1.583
    B+ 5.479 3.943 5.141 3.085
    B 9.959 7.999 9.862 7.451
    B- 17.162 19.557 19.305 21.512
    CCC-C 48.421 48.355 51.497 55.396
    corporates default prole. Hence the t for the non-nancial corporates is satisfactory while for
    all corporates the t could be rejected as too poor at 10% type-I error level. However, given the
    inversions of default rates in the all corporates data it might be hard to get a much better t
    with any other forced monotonic PD curve estimate. We therefore adopt both the all corporate
    and the non-nancial smoothed PD curves from table 8 as starting points for the PD curve
    calibration examples described in section 4 below.
    4. Calibration approaches
    The result of the estimation period is a fully specied (and smoothed) model for the joint
    distribution of a borrowers beginning of the period rating X and end of the period solvency
    state S. In this section, we discuss how to combine the estimation period model with observations
    from the beginning of the forecast period in order to predict the grade-level default rates that
    are observed at the end of the forecast period. This process is often referred to as calibration of
    the PD curve.
    18
    Table 9: Empirical and implied default proles for the 2009 all corporate and non-nancial
    corporate data from table 2. All numbers in %.
    All corporates Non-nancial corporates
    Rating grade Empirical prole Implied prole Empirical prole Implied prole
    AAA 0.0000 0.0010 0.0000 0.0001
    AA+ 0.0000 0.0009 0.0000 0.0001
    AA 0.0000 0.0094 0.0000 0.0012
    AA- 0.0000 0.0261 0.0000 0.0027
    A+ 0.4274 0.0685 0.0000 0.0068
    A 0.8547 0.1989 0.0000 0.0330
    A- 0.0000 0.4041 0.0000 0.0937
    BBB+ 0.8547 0.6355 0.0000 0.1942
    BBB 0.4274 1.1436 0.0000 0.4495
    BBB- 2.1368 1.5642 0.0000 0.6695
    BB+ 0.0000 1.2934 0.0000 0.6728
    BB 1.2821 1.9143 0.9756 1.1292
    BB- 1.7094 4.2968 1.9512 2.7405
    B+ 10.2564 7.3802 9.7561 5.8539
    B 20.5128 16.4769 20.9756 15.8468
    B- 22.2222 25.3242 24.3902 27.1784
    CCC-C 39.3162 39.2628 41.9512 45.1271
    Notation. All objects (like probabilities and the likelihood ratio) from the estimation period are
    labelled with subscript 0. All objects from the forecast period are labelled with subscript 1.
    In the following we will make use, in particular, of assumptions on the invariance or specic
    transformation between estimation and forecast period of
    the conditional rating proles Pr
    0
    [X = x| D] and Pr
    0
    [X = x| N] (for x = 1, . . . , k),
    the PD curve x Pr
    0
    [D| X = x], and
    the likelihood ratio x
    0
    (x).
    Imagine we are now at the beginning of the forecast period. The borrowers states of solvency
    at the end of the period are yet unknown. The objective of the forecast period is to predict the
    default rates to be observed at the end of the period for the rating grades 1, . . . , k by conditional
    PDs (PD curve) Pr
    1
    [D| X = x], x = 1, . . . , k. There are dierent forecast techniques for the
    conditional PDs. The selection of a suitable technique, in particular, depends on what we already
    know at the beginning of the forecast period about the joint distribution of a borrowers rating
    X at the beginning of the period and and the borrowers solvency state S at the end of the
    period. We will look in detail at the following two possibilities:
    The unconditional rating prole Pr
    1
    [X = x], x = 1, . . . , k is known. This is likely to be
    19
    the case for a newly developed rating model if all borrowers can be re-rated with the new
    model in a big-bang eort before the beginning of the forecast period. It will also be the
    case if an existing rating model is re-calibrated. Even if in the case of a new rating model
    no timely re-rating of the whole portfolio is feasible, it might still be possible (and should
    be tried) to re-rate a representative sample of the borrowers in the portfolio such that a
    reliable estimate of the unconditional rating prole is available. Where this is not possible,
    the rating model should be used in parallel run with the incumbent rating model until
    such time as the full rating prole of the portfolio has been determined. Only then a PD
    curve forecast with some chance of being accurate can be made. This might be one of
    the reasons for the credible track record requirement of the Basel Committee (BCBS,
    2006, paragraph 445). However, we will see in section 4.4 that as soon as a forecast of the
    unconditional PD is given a meaningful if not accurate PD curve forecast can be made
    without knowledge of the actual unconditional rating prole. This forecast could be used
    for a preliminary calibration during the Basel II track record period.
    An estimate of the unconditional PD p
    1
    for the forecast period is available. This forecast
    could be a proper best estimate, a pessimistic estimate for stress testing purposes, or
    a long-run estimate for the purpose of a through-the-cycle (TTC) calibration
    2
    . How to
    forecast the unconditional default rate of the portfolio is not a topic of this paper. For
    to come up with a reliable estimate is actually a major and non-trivial task that might
    be done by means of a regression of historical portfolio-wide default rates on a suitable
    set of predictive co-variates. See, for instance, Engelmann and Porath (2012) for such an
    approach.
    These two possibilities are not exclusive nor do they necessarily occur together. That is why, in
    the following, we discuss four cases:
    Case 1. Neither the unconditional rating prole nor the unconditional PD for the forecast
    period are known.
    Case 2. The unconditional rating prole for the forecast period is known but no indepen-
    dent estimate of the unconditional PD is available.
    Case 3. The unconditional rating prole for the forecast period is not known but an
    independent estimate of the unconditional PD is available.
    Case 4. The unconditional rating prole for the forecast period is known and an indepen-
    dent estimate of the unconditional PD is available.
    For each of the four cases we will present one or more techniques to forecast a set of model
    components needed to specify a full model. See proposition 3.1 for the main possibilities to specify
    a full proper one-period model of borrowers beginning of the period rating and end of the period
    solvency state. We will illustrate the forecast techniques presented with numerical exemples
    based on the S&P data from table 2. In none of the four cases there is sucient information
    from the forecast period available to completely specify a model. That is why assumptions about
    inter-period invariance of model components play an important role in the forecast process.
    2
    See Heiteld (2005) for a detailed discussion of point-in-time (PIT) and TTC rating and PD estimation ap-
    proaches. See Loer (2012) for the question of how much TTC agency ratings are.
    20
    4.1. Invariance assumptions
    Forecasting without assuming that some of the features observed in the estimation period are
    invariant (i.e. unchanged) between the estimation and forecast periods is impossible. Ideally, any
    assumption of invariance should be theoretically sound, and it should be possible to verify it by
    back-testing. In this section, we briey discuss which invariance assumptions for the one-period
    model from section 3 we should closer look at in this section.
    It is obvious that no invariance assumptions must be made on objects that can be observed
    or reliably estimated in a separate forecast exercise at the beginning of the forecast period:
    As explained above, in particular, the actual unconditional rating prole of the port-
    folio should be known at the beginning of the observation period.
    Although we will see in section 4.3 that under strong enough invariance assumptions
    a meaningful forecast of the unconditional default rate can be extracted from the
    actual unconditional rating prole it is strongly recommended not to rely on such a
    forecast but to try and independently forecast the unconditional default rate.
    As the future solvency states of the borrowers in the portfolio are not yet known at
    the beginning of the forecast period, assuming that both conditional rating proles are
    invariant could make sense. However, this assumption is likely to make for inconsistency
    issues due to equation (3.4a) in case of a known actual unconditional rating prole.
    Assuming that the likelihood ratio is invariant is less restrictive than the assumption of
    invariant conditional rating proles. It turns out, however, that if the actual unconditional
    rating prole is known the likelihood ratio invariance assumption implies a forecast of the
    unconditional default rate that might be in conict with other forecasts of the uncondi-
    tional default rate.
    Instead of assuming that both conditional rating proles are invariant, one could also
    assume that that only one of the two is invariant. In case of the assumption of an invariant
    survival rating prole this turns out to be impracticable because too often no proper model
    will be the result. That is why we do not discuss the details of this invariance assumption
    in this paper.
    Assuming the default rating prole as invariant is a much more promising approach. As a
    matter of fact we will see that for new models it might even be the only logically consistent
    approach.
    Invariance assumptions might be weakened be restating them as shape invariance assump-
    tions.
    For instance, a common approach is to assume that the shape of the PD curve is
    preserved between the estimation and the forecast periods. This can be accomplished
    by scaling the PD curve with a constant multiplier that is determined at the beginning
    of the forecast period (see, e.g., Falkenstein et al., 2000, page 67). (3.4b) shows that
    the scaled PD curve strongly depends on the estimation period unconditional PD.
    Hence making use of the scaled PD curve for forecasts in the forecast period might
    contaminate the forecast with the estimation period unconditional PD which might
    21
    be quite dierent from the forecast period unconditional PD. We include the scaled
    PD curve approach nonetheless in the subsequent more detailed discussion because
    of its simplicity and popularity.
    Scaling the likelihood ratio instead of the PD curve avoids the contamination issue we
    have observed for the scaled PD curve. The price to pay for this is that this approach
    is less straightforward than the scaled PD curve approach.
    4.2. Case 1: No unconditional rating prole and no unconditional PD given
    From a risk management point of view it is undesirable to have no current data at all. In a
    stable economic environment, this approach might be justiable nonetheless. One could assume
    that the model from the estimation period works without any adaptations also for the forecast
    period. Of course, at the end of the forecast period, we can then back-test the default prole
    from the observation period against the grade-level default frequencies observed. Formally, the
    assumption made in case 1 may be described as
    Pr
    1
    [X = x] = Pr
    0
    [X = x], x = 1, . . . , k, (4.1)
    Pr
    1
    [D| X = x] = Pr
    0
    [D| X = x], x = 1, . . . , k. (4.2)
    Table 3, combined with table 8, indicates that it would not have been a good idea to try
    and predict the grade-level S&P default rates of 2010 and 2011 with the PD curve from 2009.
    Alternatively, one might try and come up with plausible assumptions on the forecast period
    unconditional rating prole or unconditional PD which would bring us into case 2, case 3, or
    case 4.
    4.3. Case 2: Unconditional rating prole but no unconditional PD given
    In this case, the unconditional rating prole Pr
    1
    [X = x], x = 1, . . . , k can directly be observed at
    the beginning of the forecast period. We consider three approaches to prediction in the forecast
    period that may lead to proper models for the forecast period:
    Assume that the PD curve is invariant, i.e. (4.2) holds.
    Assume that both conditional rating proles are invariant, i.e.
    Pr
    1
    [X = x| D] = Pr
    0
    [X = x| D], x = 1, . . . , k, and
    Pr
    1
    [X = x| N] = Pr
    0
    [X = x| N], x = 1, . . . , k.
    (4.3)
    Assume that the likelihood ratio is invariant, i.e.

    1
    (x) =
    0
    (x), x = 1, . . . , k. (4.4)
    Note that (4.4) is implied by (4.3) and, hence, is a weaker assumption.
    22
    Assumption (4.2). As mentioned in section 4.1, it might not be the best idea to work under
    this assumption because there is a risk to contaminate the forecast with the estimation period
    unconditional default rate p
    0
    . However, by proposition 3.1 the combination of unconditional
    rating prole with any PD curve creates a unique proper model of a borrowers beginning of
    the period rating and end of the period state of solvency. In particular, by (3.5a) this approach
    implies a forecast of the unconditional default rate in the forecast period:
    p
    1
    =
    k

    x=1
    Pr
    0
    [D| X = x] Pr
    1
    [X = x]. (4.5)
    Assumption (4.3). Equation (3.4a) functions here as a constraint. The unknown unconditional
    PD p
    1
    and the two conditional proles therefore must satisfy
    Pr
    1
    [X = x] = p
    1
    Pr
    0
    [X = x| D] + (1 p
    1
    ) Pr
    0
    [X = x| N], x = 1, . . . , k. (4.6a)
    Hence, as all three proles Pr
    1
    [X = x], Pr
    0
    [X = x| D, and Pr
    0
    [X = x| D are known, we have
    k equations for the one unknown p
    1
    . In general, it seems unlikely that all the k equations can
    be simultaneously satised if only one variable can be freely chosen. However, we can try and
    compute a best t by solving the following least squares optimisation problem:
    p

    1
    = arg min
    p
    1
    [0,1]
    k

    x=1
    _
    Pr
    1
    [X = x] p
    1
    Pr
    0
    [X = x| D] (1 p
    1
    ) Pr
    0
    [X = x| N]
    _
    2
    p

    1
    =
    k

    x=1
    _
    Pr
    1
    [X = x] Pr
    0
    [X = x| N]
    _ _
    Pr
    0
    [X = x| D] Pr
    0
    [X = x| N]
    _
    _
    Pr
    0
    [X = x| D] Pr
    0
    [X = x| N]
    _
    2
    .
    (4.6b)
    Observation (4.6b) is interesting because it indicates a technique to extract a forecast of the
    unconditional PD from the unconditional rating prole at the beginning of the forecast period
    that also avoids the contamination issue observed for the invariant PD curve. It should be
    checked whether the forecast PD p

    1
    is indeed in line with the prole x Pr
    1
    [X = x]. This can
    readily be done because with p

    1
    from (4.6b) we obtain an implied unconditional rating prole
    Pr

    1
    [X = x] = p

    1
    Pr
    0
    [X = x| D] + (1 p

    1
    ) Pr
    0
    [X = x| N], x = 1, . . . , k. (4.7)
    This can be
    2
    -tested against the grade-level frequencies of borrowers at the beginning of the
    forecast period. If the hypothesis that x Pr
    1
    [X = x] is just a random realisation of x
    Pr

    1
    [X = x] cannot be rejected we can proceed to predict the PD curve on the basis of x
    Pr

    1
    [X = x] by using (3.5b):
    Pr
    1
    [D| X = x] =
    p

    1
    Pr
    0
    [X = x| D]
    Pr

    1
    [X = x]
    , x = 1, . . . , k. (4.8)
    The optimisation problem (4.6b) is convenient for deriving a forecast of p
    1
    from the unconditional
    rating prole because it yields a closed-form solution. In principle, there is no reason why the
    least squares should not be replaced with a say least absolute value optimisation. This would
    result in a slightly dierent forecast of p
    1
    . However, as we will check the appropriateness of the
    p
    1
    forecast by applying a -squared test as described in section 2.2, it seems natural to also look
    at the variant of (4.6b) where the -squared statistic is directly minimised. Proposition C.1 in
    appendix C shows that this minimisation problem is well-posed and has a unique solution.
    23
    Table 10: Forecasts of 2010 and 2011 S&P unconditional default rates and p-values for goodness
    of t tests of 2010 and 2011 unconditional rating proles. The forecast approaches are
    described in section 4.3.
    Forecast for 2010
    Sample All corporates Non-nancial corporates
    Observed default rate 1.14% 1.43%
    Forecast DR p-value Forecast DR p-value
    Invariant PDs (4.2) 4.32% Exact 5.96% Exact
    Least squares (4.6b) 4.80% 0.0037 6.83% 0.3338
    Least
    2
    (C.3) 5.35% 0.0051 7.17% 0.3467
    Invariant LR (4.4) 5.38% Exact 7.19% Exact
    Forecast for 2011
    Sample All corporates Non-nancial corporates
    Observed default rate 0.75% 0.95%
    Forecast DR p-value Forecast p-value
    Invariant PDs (4.2) 3.75% Exact 4.87% Exact
    Least squares (4.6b) 3.50% < 10
    10
    3.60% 1.5 10
    5
    Least
    2
    (C.3) 2.84% < 10
    10
    3.26% 1.6 10
    5
    Invariant LR (4.4) 2.79% Exact 3.27% Exact
    Assumption (4.4). Like for assumption (4.3), it is not a priori clear that a proper model
    of a borrowers rating prole and solvency state can be based on the unconditional prole
    x Pr
    1
    [X = x] and the likelihood ratio
    0
    (x). The necessary and sucient condition for the
    likelihood ratio to match the rating prole is provided in equation (3.10b) of proposition 3.2,
    with
    x
    = Pr
    1
    [X = x] and (x) =
    0
    (x).
    If condition (3.10b) is satised then proposition 3.3 implies that there is a unique model of a
    borrowers rating and solvency state with characteristics Pr
    1
    [X = x] and
    0
    (x). The uncondi-
    tional PD in this model is determined as the unique solution p
    1
    of equation (3.10a), and we can
    calculate the PD curve by (3.8a):
    Pr
    1
    [D| X = x] =
    p
    1
    p
    1
    + (1 p
    1
    )
    0
    (x)
    , x = 1, . . . , k. (4.9)
    Summary. Table 10 displays some forecast results that were calculated with the approaches
    described in this section. Forecast values for the 2010 and 2011 S&P unconditional default rates
    are presented together with assessments of the goodness of t of the actual unconditional rating
    proles by the implied or assumed unconditional rating proles. It is immediately clear from
    table 10 that the forecasts of the unconditional default rates are much too high in all cases.
    That is why we did not bother to show the grade-level forecast default rates or any other model
    24
    Table 11: S&P grade-level smoothed likelihood ratios (dened by (3.7)) for corporates in 2009,
    2010 and 2011. Source: Own calculations.
    All corporates Non-nancial corporates
    Rating grade 2009 2010 2011 2009 2010 2011
    AAA 1,501.55 53,720.74 20,221.19 6,813.88 20,028,740.00 131,709.90
    AA+ 715.28 20,477.36 7,734.58 3,017.50 5,562,260.00 56,168.55
    AA 353.69 8,583.13 3,410.45 1,304.81 1,253,332.00 21,913.78
    AA- 167.12 3,099.52 1,365.00 646.25 367,937.00 9,281.64
    A+ 88.18 1,161.49 547.00 372.94 138,158.70 4,404.54
    A 45.53 436.37 227.20 182.81 38,238.02 1,645.46
    A- 23.98 177.13 100.78 84.70 9,612.43 601.66
    BBB+ 13.89 83.35 51.03 43.44 2,988.50 255.66
    BBB 8.37 40.00 27.65 23.04 945.70 114.38
    BBB- 5.17 19.85 15.00 12.94 335.61 53.48
    BB+ 3.61 12.23 9.58 8.49 168.65 31.20
    BB 2.70 8.28 6.74 5.91 93.27 20.33
    BB- 1.78 4.93 4.24 3.58 42.88 11.52
    B+ 1.01 2.46 2.19 1.81 15.21 5.07
    B 0.48 0.96 0.79 0.71 3.59 1.44
    B- 0.17 0.27 0.20 0.21 0.57 0.28
    CCC-C 0.04 0.05 0.04 0.05 0.05 0.04
    characteristics as the t would have been equally poor.
    We have argued above that assumption (4.2) is suboptimal for risking undesirable impact on
    the forecast of the estimation period unconditional default rate. Assumption (4.4) is the most
    promising of the three assumptions we have explored because it guarantees exact t of the un-
    conditional rating prole and avoids contamination of the forecast. Assumption (4.3) is stronger
    than (4.4) because it implies (4.4). In principle, assumption (4.3) will hardly ever provide a
    proper model because it is rather unlikely that that the overdetermined equation (4.6a) has an
    exact solution. By (4.6b) or proposition C.1, however, we can determine an approximate t that
    could turn out to be statistically indistinguishable from the rating prole at the beginning of the
    forecast period which would make assumption (4.3) a viable approach, too. Indeed, according
    to table 10 this is the case for the 2010 forecast for the non-nancial corporates.
    As contamination by the 2009 unconditional default rate is prevented under assumptions (4.3)
    and (4.4), it is interesting to speculate why the implied default rate forecasts are so poor nonethe-
    less. The natural conclusion is that the assumptions are simply wrong for the S&P data. Indeed,
    as table 11 demonstrates for the likelihood ratio, with hindsight it is clear that the invariance
    assumptions made in this sections do not hold. An alternative and complementary explanation
    could however be that the S&P ratings made in 2009 and 2010 are over-pessimistic and for this
    reason generate too high default rate forecasts. This explanation is supported by the observation
    25
    that in 2009 the downgrade-to-upgrade ratio for the S&P corporate ratings was 3.99 (S&P, 2012,
    Table 6) which could presumably not even be compensated by the 2010 downgrade-to-upgrade
    ratio of 0.74. Possibly, the truth is a mixture of these two explanations.
    4.4. Case 3: No unconditional rating prole but unconditional PD given
    In this case, we assume that a forecast unconditional PD 0 < p
    1
    < 1 is given that is likely
    to dier from the estimation period unconditional PD. But the unconditional current rating
    prole is assumed not to be known. This would typically be the case if a rating model was newly
    developed and it was not possible to rate all the borrowers in the portfolio in one big-bang eort.
    The new ratings would then only become available in the course of the regular annual rating
    process. This is clearly suboptimal, in particular with a view on the validation of the new rating
    model, but sometimes unavoidable due to limitation of resources.
    In this situation, proposition 3.1 suggests assumption (4.3) as the only possibility to infer a full
    model of a borrowers beginning of the period rating and end of the period state of solvency.
    If, however, it is sucient to obtain an estimate of the forecast period PD curve then it is
    solely the estimation period likelihood ratio x
    0
    (x) that one needs to know in addition to
    the unconditional PD p
    1
    . This follows from equation (4.9). Elkan (2001, theorem 2) stated this
    observation as change in base rate theorem.
    4.5. Case 4: Unconditional rating prole and unconditional PD given
    In this case, it is assumed that the unconditional rating prole Pr
    1
    [X = x], x = 1, . . . , k can
    directly be observed at the beginning of the forecast period, and it is also assumed that a
    forecast unconditional PD 0 < p
    1
    < 1 is given that is likely to dier from the estimation period
    unconditional PD. Like in case 2, there are several approaches to prediction in the forecast
    period that may lead to proper models for the forecast period:
    Assume that the default rating prole is invariant, i.e.
    Pr
    1
    [X = x| D] = Pr
    0
    [X = x| D], x = 1, . . . , k. (4.10)
    Assume that the discriminatory power of the model as measured by the accuracy ratio
    (see (3.6)) is invariant, i.e.
    AR
    1
    = AR
    0
    . (4.11)
    Note that (4.11) is a weaker assumption than (4.3) because (4.11) is implied by (4.3).
    Assume that the estimation period PD curve can be linearly scaled to become the forecast
    period PD curve, i.e. there is a constant c
    PD
    > 0 such that
    Pr
    1
    [D| X = x] = c
    PD
    Pr
    0
    [D| X = x], x = 1, . . . , k. (4.12)
    Assume that the estimation period likelihood ratio can be linearly scaled to become the
    forecast period likelihood ratio, i.e. there is a constant c
    LR
    > 0 such that

    1
    (x) = c
    LR

    0
    (x), x = 1, . . . , k. (4.13)
    26
    In principle, a fth approach is cogitable, namely to assume that the survivor rating prole does
    not change from the estimation period to the forecast period. However, it is very unlikely that
    this approach results in a proper forecast period model with a proper default rating prole. That
    is why we do not discuss this approach.
    Assumption (4.10). This assumption is not necessarily viable as again (3.4a) must be satised.
    It turns out that assumption (4.10) makes for a proper model of a borrowers rating and state
    of solvency if and only if we have for all x = 1, . . . , k
    p
    1
    Pr
    0
    [X = x| D] Pr
    1
    [X = x] and p
    1
    _
    1 Pr
    0
    [X = x| D]
    _

    _
    1 Pr
    1
    [X = x]
    _
    . (4.14a)
    If (4.14a) holds then by (3.5b) we obtain the following equation for the PD curve:
    Pr
    1
    [D| X = x] =
    p
    1
    Pr
    0
    [X = x| D]
    Pr
    1
    [X = x]
    . (4.14b)
    Actually, there are two slightly dierent approaches to implement assumption (4.10):
    (i) Use a smoothed version of the estimation period default prole that could be derived via
    equation (3.5b) from a smoothed PD curve which in turn might have been determined
    by QMM as described in section 3.1.
    (ii) Use the observed estimation period default prole and the given forecast period uncondi-
    tional prole to determine by means of (3.5c) an implied raw survivor prole. Based on this
    survivor prole and the observed estimation period default prole deploy equation (3.6)
    to compute a forecast accuracy ratio. Apply then QMM as described in section 3.1 to
    determine a smoothed PD curve for the forecast period.
    Compared to approach (i), approach (ii) has the advantage of always delivering a monotonic PD
    curve. That is why for the purpose of this paper we implement assumption (4.10) in the shape
    of (ii) although anecdotal evidence shows that the performance of (ii) is not necessarily better
    than the performance of (i).
    Assumption (4.11). Actually, even with unconditional rating prole, unconditional PD, and
    accuracy ratio given the joint distribution of a borrowers beginning of the period rating and end
    of the period state is not uniquely determined. We suggest applying QMM as in the estimation
    period (see section 3.3) and described in appendix B to compute a PD curve as a forecast of
    the grade-level default rates. There is, however, the problem that QMM requires the rating
    prole conditional on survival as an input which cannot be observed or implied at this stage.
    But QMM is fairly robust with regard to the coding of the rating grades used as input to
    the algorithm. That is why approximating the rating prole conditional on survival with the
    unconditional rating prole (known by assumption) seems to work well.
    Assumption (4.12). The constant c
    PD
    is determined by equation (3.5a):
    c
    PD
    =
    p
    1

    k
    x=1
    Pr
    0
    [D| X = x] Pr
    1
    [X = x]
    (4.15a)
    27
    Table 12: 2010 and 2011 grade-level forecast default rates for S&P all corporates ratings. P-
    values are for the
    2
    -tests of the implied default proles. All values in %.
    Default rate Invariant default Invariant AR Scaled PDs Scaled likelihood
    prole (4.10) (4.11) (4.12) ratio (4.13)
    2010: Unconditional default rate 1.141
    AAA 0 0.0013 0.0004 0.0007 0.0005
    AA+ 0 0.0025 0.0009 0.0015 0.0011
    AA 0 0.0048 0.0018 0.0031 0.0023
    AA- 0 0.0097 0.0040 0.0066 0.0049
    A+ 0 0.0177 0.0086 0.0125 0.0093
    A 0 0.0327 0.0183 0.0241 0.018
    A- 0 0.0591 0.0366 0.0458 0.0342
    BBB+ 0 0.0977 0.0652 0.0789 0.059
    BBB 0 0.1555 0.1145 0.1307 0.0979
    BBB- 0 0.2409 0.1955 0.2107 0.1581
    BB+ 0.7874 0.3332 0.2827 0.3006 0.2263
    BB 0.3623 0.4328 0.3806 0.4012 0.3029
    BB- 0.5277 0.6243 0.5631 0.6024 0.4576
    B+ 0 1.0252 0.9460 1.0417 0.8023
    B 0.6881 1.9704 1.8843 2.1134 1.6844
    B- 2.069 4.605 4.5164 5.1671 4.5716
    CCC-C 22.2727 13.4867 14.5179 12.7755 15.576
    P-value Exact 5.34 7.9 4.1 10.5
    2011: Unconditional default rate 0.752
    AAA 0 0.0004 0.0003 0.0006 0.0004
    AA+ 0 0.0009 0.0006 0.0012 0.0009
    AA 0 0.0019 0.0013 0.0024 0.0018
    AA- 0 0.0041 0.0027 0.005 0.0039
    A+ 0 0.0079 0.0058 0.0095 0.0074
    A 0 0.0155 0.0120 0.0183 0.0143
    A- 0 0.0298 0.0236 0.0347 0.0271
    BBB+ 0 0.0519 0.0416 0.0599 0.0468
    BBB 0 0.0867 0.0691 0.0992 0.0777
    BBB- 0.1969 0.1404 0.1147 0.16 0.1256
    BB+ 0 0.2007 0.1662 0.2282 0.1797
    BB 0 0.2677 0.2219 0.3046 0.2405
    BB- 0 0.4006 0.3248 0.4573 0.3635
    B+ 0.3929 0.6913 0.5567 0.7909 0.6378
    B 1.1945 1.4207 1.2710 1.6046 1.3415
    B- 3.9867 3.6221 3.7941 3.9231 3.6627
    CCC-C 15.942 11.9924 13.3871 9.6998 12.7721
    P-value Exact 71.9 89.68 37.65 82.38
    28
    However, if c
    PD
    > 1 the resulting model could be improper because by (4.12) it could turn out
    that Pr
    1
    [D| X = x] > 1 for some x. If the resulting model under assumption (4.12) is proper
    the implied default prole is as follows:
    Pr
    1
    [X = x| D] = c
    PD
    Pr
    0
    [D| X = x] Pr
    1
    [X = x] / p
    1
    , x = 1, . . . , k. (4.15b)
    Assumption (4.13). By (3.9b) we obtain an equation that determines the constant c
    LR
    :
    1 =
    k

    x=1
    Pr
    1
    [X = x]
    p
    1
    + (1 p
    1
    ) c
    LR

    0
    (x)
    . (4.16a)
    Note that
    lim
    c
    k

    x=1
    Pr
    1
    [X = x]
    p
    1
    + (1 p
    1
    ) c
    0
    (x)
    = 0 and lim
    c0
    k

    x=1
    Pr
    1
    [X = x]
    p
    1
    + (1 p
    1
    ) c
    0
    (x)
    = 1/p
    1
    > 1.
    Hence equation (4.16a) has always a unique solution c
    LR
    > 0. By proposition 3.3 then we know
    that under assumption (4.13) we have a proper model of a borrowers rating and default state.
    In addition, by proposition 3.2 the resulting forecast likelihood ratio
    1
    (x) = c
    LR

    0
    (x) satises
    the inequalities
    k

    x=1
    Pr
    1
    [X = x]

    1
    (x)
    > 1 and
    k

    x=1
    Pr
    1
    [X = x]
    1
    (x) > 1.
    This implies the following inequality for c
    LR
    :
    1

    k
    x=1
    Pr
    1
    [X = x]
    0
    (x)
    < c
    LR
    <
    k

    x=1
    Pr
    1
    [X = x]

    0
    (x)
    . (4.16b)
    (4.16b) is useful because it provides initial values for the numerical solution of (4.16a) for c
    LR
    .
    Once c
    LR
    has been determined (3.9a) and (3.8a) imply the following equations for the default
    prole and the PD curve under assumption (4.13):
    Pr
    1
    [X = x| D] =
    Pr
    1
    [X = x]
    p
    1
    + (1 p
    1
    ) c
    LR

    0
    (x)
    , x = 1, . . . , k, (4.16c)
    Pr
    1
    [D| X = x] =
    p
    1
    p
    1
    + (1 p
    1
    ) c
    LR

    0
    (x)
    , x = 1, . . . , k. (4.16d)
    Summary. Tables 12 and 1 show the results of an application of the approaches presented
    above to forecasting the 2010 and 2011 grade-level default rates of the S&P all corporates and
    non-nancial corporates portfolios, based on estimates made with data from 2009. To allow for
    a fair performance comparison, we have made use of prophetic estimates of the 2010 and 2011
    unconditional default rates, by setting the value of p
    1
    to the observed unconditional default rate
    of the respective year and sample.
    In order to express the performance of the dierent approaches in one number for each approach,
    we have used the forecast PD curves to derive forecast default proles by means of (3.5b). The
    29
    forecast default proles can be -squared tested against the observed grade-level default numbers
    from table 2. The p-values of these tests are shown in the last rows of the panels of table 12 and
    1 respectively. Recall that higher p-values mean better goodness of t.
    Tables 12 and 1 hence indicate that under the constraints of this section (unconditional rating
    prole and default rate are given) the scaled likelihood ratio approach (4.13) and the invariant ac-
    curacy ratio approach (4.11) work best, followed by the invariant default prole approach (4.10).
    This observation, however, does not allow an unconditional conclusion that scaled likelihood
    ratio or invariant accuracy ratio are the best approaches to PD curve calibration.
    In particular, when a new rating model is developed one has often to combine data from sev-
    eral observation periods in order to create a suciently large training sample. Estimating the
    likelihood ratio from such a combined sample would implicitly be based on the assumption of
    an invariant likelihood ratio. Hence it would be strange to modify the likelihood ratio ratio via
    scaling in the forecast period. This consistency issue is obviously avoided with the invariant
    default prole and the invariant accuracy ratio approaches. As we have seen, to implement
    the invariant accuracy ratio approach we need to approximate the forecast period survivor
    prole by the forecast period unconditional rating prole. This approximation could be poor if
    the forecast period unconditional default rate is high. Hence, depending on what approach had
    been followed in the estimation period and how big the forecast period unconditional default
    rate is, the invariant default prole approach (4.10) could be preferable for the forecast period
    despite its only moderate performance in our numerical examples.
    5. Conclusions
    Accurate (re-)calibration of a rating model requires careful consideration of a number of ques-
    tions that include, in particular, the question of which model components can be assumed to
    be invariant between the estimation period of the model and the forecast period. Looking at
    PD curve calibration as a problem of forecasting rating-grade level default rates, we have dis-
    cussed a one-period model framework that is suitable for the description of a variety of dierent
    forecasting techniques.
    We have then proceeded to present a number of PD curve calibration techniques and explore
    the conditions under which the techniques are t for purpose. We have tested the techniques
    introduced by applying them to a publicly available dataset of S&P rating and default statistics
    that can be considered typical for the scope of application of the techniques.
    One negative and one positive nding are the main results of our considerations:
    The popular technique of scaling the PD curve in order to (re-)calibrate a rating model
    to a dierent target unconditional PD is likely to deliver suboptimal results because it
    implicitly mixes up the unconditional PD of the estimation period and the target PD.
    As shown by example, two techniques that avoid mixing up the unconditional PDs from the
    estimation and the forecast periods are promising alternatives to scaling the PD curve.
    These techniques are scaling the likelihood ratio and quasi moment matching of target
    PD and estimation period accuracy ratio.
    30
    A. Appendix: The binormal model with equal variances
    In this paper, we apply quasi moment matching (QMM) as suggested by Tasche (2009) for the
    smoothing of PD curves. QMM requires the numerical solution of a two-dimensional system of
    non-linear equations. The solution of such an equation system in general is much facilitated if a
    meaningful initial guess of the solution can be provided. The binormal model we discuss in the
    following delivers such a guess. In addition, the binormal model provides the main motivation
    of the QMM approach.
    Formally, the binormal model with equal variances is based on the following assumption.
    Assumption A.1 X denotes the continuous score of a borrower at the beginning of the obser-
    vation period.
    The distribution of X conditional on the event D (the borrower defaults during the obser-
    vation period) is normal with mean
    D
    and variance
    2
    > 0.
    The distribution of X conditional on the event N (the borrower remains solvent during the
    whole observation period) is normal with mean
    N
    >
    D
    and variance
    2
    > 0.
    p (0, 1) is the borrowers unconditional PD (i.e. the unconditional probability that the
    borrower defaults during the observation period).
    Denote by f
    D
    and f
    N
    respectively the conditional densities of the binormal score X. Hence by
    assumption A.1 we have
    f
    D
    (x) =
    1

    2
    exp
    _

    (x
    D
    )
    2
    2
    2
    _
    ,
    f
    N
    (x) =
    1

    2
    exp
    _

    (x
    N
    )
    2
    2
    2
    _
    .
    (A.1)
    In the continuous case specied by assumption A.1, Bayes formula implies a PD curve x
    Pr[D| X = x] similar to the discrete formula (3.4b):
    Pr[D| X = x] =
    p f
    D
    (x)
    p f
    D
    (x) + (1 p) f
    N
    (x)
    (A.2a)
    =
    1
    1 + exp( + x)
    , (A.2b)
    =

    2
    D

    2
    N
    2
    2
    + log
    _
    1 p
    p
    _
    , (A.2c)
    =

    N

    D

    2
    . (A.2d)
    Note that from (A.2b) it follows that
    d Pr[D| X = x]
    d x
    = Pr[D| X = x]
    _
    1 Pr[D| X = x]
    _
    . (A.3)
    Hence the absolute value of the slope of the PD curve (A.2b) attains its maximum if and only
    if Pr[D| X = x] = 1/2 and then the maximum absolute slope is /4.
    31
    Denote by X
    D
    and X
    N
    independent random variables with X
    D
    N(
    D
    , ) and X
    N
    N(
    N
    , ).
    Then, under assumption A.1, we also obtain a simple formula
    3
    for the discriminatory power of
    the score X if it is measured as accuracy ratio (see, for instance, Tasche, 2009, section 3.1.1):
    AR = Pr[X
    D
    < X
    N
    ] Pr[X
    D
    > X
    N
    ] (A.4a)
    = 2
    _

    N

    D

    2
    _
    1. (A.4b)
    In addition, it is easy to show how the unconditional mean and variance
    2
    of the score X
    can be described in terms of the means and variances of X conditional on default and survival
    respectively:
    = p
    D
    + (1 p)
    N
    , (A.5a)

    2
    =
    2
    +p (1 p) (
    D

    N
    )
    2
    . (A.5b)
    A close inspection of equations (A.4b), (A.5a) and (A.5b) shows that the conditional variance

    2
    and the conditional means
    D
    and
    N
    can be written as functions of the unconditional mean,
    the unconditional variance and the accuracy ratio:
    c =

    2
    1
    _
    AR+1
    2
    _
    ,

    2
    =

    2
    1 +p (1 p) c
    2
    ,

    N
    = +p c,

    D
    = (1 p) c.
    (A.6)
    From this, it follows by (A.2c) and (A.2d) that also the coecients and in (A.2b) can be
    represented in terms of the unconditional mean of X, the unconditional variance
    2
    of X, and
    the discriminatory power AR of X. In particular, we have the following representation of in
    terms of the accuracy ratio and the dispersion of the conditional score distributions:
    =

    2
    1
    _
    AR+1
    2
    _

    . (A.7)
    These observations suggest the following three steps approach to identifying initial values for
    the QMM approach to PD curve smoothing:
    1) Calculate the mean and the standard deviation of the unconditional rating prole.
    2) Use and together with the unconditional PD p and the accuracy ratio AR implied by
    the rating prole and the observed grade-level default rates to calculate the conditional
    standard deviation and the conditional means
    D
    and
    N
    according to (A.6).
    3) Use equations (A.2c) and (A.2d) to determine initial values for and .
    The initial values found by this approach will be the closer to the true values, the closer the
    conditional rating proles are to normal distributions.
    3
    denotes the standard normal distribution function (x) =
    1

    e
    1/2 y
    2
    dy.
    32
    B. Appendix: Quasi moment matching
    Equation (A.2a) shows that the unconditional PD has a direct primary impact on the level of
    the PD curve. Equation (A.7) suggests that the AR of a rating model has a similar impact on
    the maximum slope of the PD curve. The two observations together suggest that in general a
    two-parameter PD curve can be tted to match given unconditional PD and AR.
    It may be argued that for a suitably developed rating model based on carefully selected risk
    factors, the associated PD curve must be monotonic for economic reasons. Under the assumption
    that the PD curve is montonic, Tasche (2009, section 5.2) suggested the following robust version
    of the logistic curve (A.2b) for tting the PD curve:
    P[D| X = x]
    1
    1 + exp
    _
    +
    1
    _
    F
    N
    (x)
    __,
    F
    N
    (x) = Pr[X x| N].
    (B.1)
    The term
    1
    _
    F
    N
    (x)
    _
    in (B.1) transforms the in general non-normal distribution of the ratings
    conditional on survival into another distribution that is approximately normal even if the rating
    distribution is not continuous. However, in the discontinuous case F
    N
    (x) = 1 may occur which
    would entail P[D| X = x] = 0. A suitable work-around to avoid this is to replace the distribution
    function F
    N
    by the average

    F
    N
    of F
    N
    and its left-continuous version:

    F
    N
    (x) =
    Pr[X < x| N] + Pr[X x| N]
    2
    . (B.2)
    Dene, in addition to F
    N
    , the distribution function F
    D
    of the rating variable X conditional on
    default by
    F
    D
    (x) = P[X x| D]. (B.3)
    and denote by X
    D
    and X
    N
    independent random variables that are distributed according to F
    D
    and F
    N
    respectively.
    For quantifying discriminatory power, we apply again the notion of accuracy ratio (AR) as
    specied in Tasche (2009, eq. (3.28b)):
    AR = 2 P[X
    D
    < X
    N
    ] + P[X
    D
    = X
    N
    ] 1
    = P[X
    D
    < X
    N
    ] P[X
    D
    > X
    N
    ]
    =
    _
    P[X < x| D] dF
    N
    (x)
    _
    P[X < x| N] dF
    D
    (x).
    (B.4a)
    See Hand and Till (2001, section 2) for a discussion of why this denition of accuracy ratio (or
    the related denition of the area under the ROC curve) is more expedient than the also common
    denition in geometric terms. Denition (B.4a) of AR takes an ex post perspective by assuming
    the obligors states D or N at the end of the observation period are known and hence can be
    used for estimating the conditional (on default and survival respectively) rating distributions
    F
    D
    and F
    N
    .
    In the case where X is realised as one of a nite number of rating grades x = 1, . . . , k, the
    33
    accuracy ratio can be calculated from the PD curve as follows:
    AR =
    1
    p (1 p)
    _
    2
    k

    x=1
    _
    1 P[D| X = x]
    _
    P[X = x]
    x1

    t=1
    P[D| X = t] P[X = t]
    +
    k

    x=1
    P[D| X = x]
    _
    1 P[D| X = x]
    _
    P[X = x]
    2
    _
    1.
    (B.4b)
    Where p stands for the unconditional PD as described in (3.2).
    Then quasi moment matching, for the purpose of this paper means the following procedure:
    1) Fix target values p
    target
    and AR
    target
    for the unconditional portfolio PD and the accuracy
    ratio of the rating model.
    2) Substitute p
    target
    and AR
    target
    for the left-hand sides of equations (3.5a) and (B.4b) re-
    spectively.
    3) Represent P[D| X = x] in (3.5a) and (B.4b) by the robust logistic curve (B.1) (with F
    N
    replaced by

    F
    N
    ).
    4) Choose initial values for the parameters and according to (A.2c), (A.2d) and (A.6).
    5) Solve numerically the equation system for and .
    C. Appendix: Minimising the Pearson statistic
    Proposition C.1 Assume that in (2.2b) the class probabilities are given as
    q
    i
    = p g
    i
    + (1 p) h
    i
    , i = 1, . . . , k, (C.1)
    where g
    i
    > 0 and h
    i
    > 0, i = 1, . . . , k, are probability distributions and 0 p 1. Assume
    that g
    i
    = h
    i
    for at least one i. Let furthermore class frequencies n
    1
    , . . ., n
    k
    be given and let
    n =

    k
    i=1
    n
    i
    . Dene the function f(p) by
    f(p) =
    k

    i=1
    _
    n
    i
    n(p g
    i
    + (1 p) h
    i
    )
    _
    2
    n(p g
    i
    + (1 p) h
    i
    )
    . (C.2)
    Then there is a unique p

    [0, 1] such that


    f(p

    ) = min
    0p1
    f(p). (C.3)
    We have
    p

    = 0 if and only if
    k

    i=1
    (h
    i
    g
    i
    ) n
    2
    i
    (nh
    i
    )
    2
    0, (C.4a)
    0 < p

    < 1 if and only if


    k

    i=1
    (h
    i
    g
    i
    ) n
    2
    i
    (nh
    i
    )
    2
    < 0 <
    k

    i=1
    (h
    i
    g
    i
    ) n
    2
    i
    (ng
    i
    )
    2
    , (C.4b)
    34
    p

    = 1 if and only if
    k

    i=1
    (h
    i
    g
    i
    ) n
    2
    i
    (ng
    i
    )
    2
    0. (C.4c)
    Proof of proposition C.1. The function f is twice continuously dierentiable with respect to
    p:
    f

    (p) =
    k

    i=1
    (h
    i
    g
    i
    ) n
    2
    i
    _
    n(p g
    i
    + (1 p) h
    i
    )
    _
    2
    ,
    f

    (p) =
    k

    i=1
    (h
    i
    g
    i
    )
    2
    n
    2
    i
    _
    n(p g
    i
    + (1 p) h
    i
    )
    _
    3
    > 0.
    This last equation implies that f is strictly convex. This implies the assertions because the
    conditions (C.4a), (C.4b), and (C.4c) are phrased in terms of f

    (0) and f

    (1). q.e.d.
    References
    BCBS. International Convergence of Capital Measurement and Capital Standards. A Revised
    Framework, Comprehensive Version. Basel Committee on Banking Supervision, June 2006.
    J.R. Bohn and R.M. Stein. Active Credit Portfolio Management in Practice. John Wiley &
    Sons, Inc., 2009.
    G. Casella and R.L. Berger. Statistical Inference. Duxbury Press, second edition, 2002.
    C. Elkan. The foundations of cost-sensitive learning. In B. Nebel, editor, Proceedings of the
    Seventeenth International Joint Conference on Articial Intelligence, IJCAI 2001, pages 973
    978. Morgan Kaufmann, 2001.
    B. Engelmann and D. Porath. Do not Forget the Economy when Estimating Default Probabili-
    ties. Willmott magazine, pages 7073, February 2012.
    E. Falkenstein, A. Boral, and L.V. Carty. Riskcalc
    TM
    for Private Companies: Moodys Default
    Model. Rating methodology, Moodys Investors Service, May 2000.
    R.A. Fisher. On the interpretation of
    2
    from contingency tables, and the calculation of p.
    Journal of the Royal Statistical Society, 85(1):8794, 1922.
    D.J. Hand. Construction and Assessment of Classication Rules. John Wiley & Sons, Chichester,
    1997.
    D.J. Hand and R.J. Till. A simple generalisation of the area under the ROC curve for multiple
    class classication problems. Machine Learning, 45(2):171186, 2001.
    E.A. Heiteld. Dynamics of rating systems. In Studies on the Validation of Internal Rating
    Systems. Revised version, pages 1027. Basel Committee on Banking Supervision, Working
    Paper No. 14, 2005.
    35
    M.P. Konrad. The Calibration of Rating Models. PhD thesis, Wirtschaftswissenschaftliche
    Fakultat der Westfalischen Wilhelms-Universitat M unster, 2011.
    G. Loer. Can rating agencies look through the cycle? Review of Quantitative Finance and Ac-
    counting, 2012. URL http://link.springer.com/article/10.1007/s11156-012-0289-9.
    Moodys. Annual default study: Corporate Default and Recovery Rates, 1920-2011. Special
    comment, Moodys Investors Service, February 2012.
    K. Pearson. On the criterion that a given system of deviations from the probable in the case
    of a correlated system of variables is such that it can be reasonably supposed to have arisen
    from random sampling. Philosophical Magazine, Series 5, 50(302):157175, 1900.
    S&P. Default, Transition, and Recovery: 2009 Annual Global Corporate Default Study And
    Rating Transitions. Report, Standard & Poors, March 2010.
    S&P. Default, Transition, and Recovery: 2010 Annual Global Corporate Default Study And
    Rating Transitions. Report, Standard & Poors, March 2011.
    S&P. Default, Transition, and Recovery: 2011 Annual Global Corporate Default Study And
    Rating Transitions. Report, Standard & Poors, March 2012.
    D. Tasche. Validation of internal rating systems and PD estimates. In G. Christodoulakis and
    S. Satchell, editors, The Analytics of Risk Model Validation, pages 169196. Academic Press,
    2008.
    D. Tasche. Estimating discriminatory power and PD curves when the number of defaults is
    small. Working paper, Lloyds Banking Group, 2009.
    M. van der Burgt. Calibrating low-default portfolios, using the cumulative accuracy prole.
    Journal of Risk Model Validation, 1(4):1733, 2008.
    A.W. van der Vaart. Asymptotic Statistics. Cambridge University Press, 1998.
    H.L. van Trees. Detection, Estimation, and Modulation Theory, Part I. John Wiley & Sons,
    1968.
    36

    You might also like