Improving Cash Flow Using Credit Management
Improving Cash Flow Using Credit Management
Improving Cash Flow Using Credit Management
Cash flow is the life blood of all businesses and is the primary indicator of business health. It is generally acknowledged as the single most pressing concern of most small and medium-sized enterprises (SMEs), although even finance directors of the largest organisations emphasise the importance of cash, and cash flow modelling is a fundamental part of any private equity buy-out. In a credit crunch environment, where access to liquidity is restricted, cash management becomes critical to survival. In its simplest form, cash flow is the movement of money in and out of your business. It is not profit and loss, although trading clearly has an effect on cash flow. The effect of cash flow is real, immediate and, if mismanaged, totally unforgiving. Cash needs to be monitored, protected, controlled and put to work. There are four principles regarding cash management: Cash is not given. It is not the passive, inevitable outcome of your business endeavours. It does not arrive in your bank account willingly. Rather it has to be tracked, chased and captured. You need to control the process and there is always scope for improvement. Cash management is as much an integral part of your business cycle as, for example, making and shipping widgets or preparing and providing detailed consultancy services. Good cash flow management requires information. For example, you need immediate access to data on: your customers creditworthiness your customers current track record on payments outstanding receipts your suppliers payment terms short-term cash demands short-term surpluses investment options current debt capacity and maturity of facilities longer-term projections. You must be masterful. Managing cash flow is a skill and only a firm grip on the cash conversion process will yield results. Professional cash management in business is not, unfortunately, always the norm. For example, a survey conducted by the Better Practice Payment Group in 2006 highlighted that one in three companies do not confirm their credit terms in writing with customers. And many finance functions do not maintain an accurate cash flow forecast (which is crucial, as well see later). Good cash management has a double benefit: it can help you to avoid the debilitating downside of cash crises; and it can grant you a commercial edge in all your transactions. For example, companies able to aggressively manage their inventory may require less working capital and be able to extend more competitive credit terms than their rivals.
Working capital
Working Capital reflects the amount of cash tied up in the business trading assets. It is usually calculated as: stock (including finished goods, work in progress and raw materials) + trade debtors - trade creditors. It is made up of three components: Days sales outstanding (DSO, or `debtor days) is an expression of the amount of cash you have tied up in unpaid invoices from customers. Most businesses offer credit in order to help customers manage their own cash flow cycle (more on that shortly) and that uncollected cash is a cost to the business. DSO = 365 x accounts receivable balance/annual sales. Days payable outstanding (DPO or creditor days) tells you how youre doing with suppliers. The aim here is a higher number, if your suppliers are effectively lending you money to buy their services, thats cash you can use elsewhere in the business. DPO = 365 x accounts payable balance/annual cost of goods sold. Finally, your days of inventory (DI). This is tells you how much cash you have tied up in stock and raw materials. Like DSO, a lower number is better. DI = 365 x inventory balance/annual sales. Almost all businesses have working capital tied up in receivables and inventory. But not all of them. Many of the UKs big supermarkets chains, for example, have negative working capital. Customers pay in cash at the tills, but stock is provided by suppliers on credit, often on very generous terms. That means that at any given time, the supermarket has excess cash which can be used to earn interest or be invested in new store roll-outs, for example. Thats one reason their business model is so successful and demonstrates the importance of cash flow management. Working capital consultancy REL conducts an annual survey of Europes biggest businesses. In its 2008 report, it said
that in response to the global recession, they were paying suppliers more slowly to artificially bolster their balance sheets. `But in doing so theyre often damaging supplier relationships and creating gains that cant be sustained over time, claimed the report. `A typical European company [in 2008 was] taking over 45 days to pay its suppliers - nearly a day and a half longer than last year. So simply cutting down on your DSO or increasing your DPO are not necessarily good long-term solutions. Smart management of cash flow cycle, including tighter business processes and better credit management, is essential. .
Inflows
Cash inflow is money coming into your business: money from the sale of your goods or services to customers money on customer accounts outstanding bank loans interest received on investments investment by shareholders in the company.
Outflows
Cash outflow is, naturally, what you pay out: purchasing finished goods for re-sale purchasing raw materials to manufacture a final product paying wages paying operating expenses (such as rent, advertising and R&D) purchasing fixed assets paying the interest and principal on loans taxes.
Credit decisions
Dun and Bradstreet has calculated that more than 90% of companies grant credit without a reference. If credit terms and conditions are not agreed in advance and references checked, you risk trading with `cant pay customers as well
as `wont pay ones. Salespeople, in particular, need to remember that a sale is not a sale until its been paid for and extending credit haphazardly might look good for their figures (and the P&L, at least initially), but can be disastrous for cash flow. (See section three on credit management for more details.)
Bad debts
Late payment sometimes escalates to become a bad debt. If you are making 1.5% profit on sales, an uncollected debt of 1,500 nullifies 100,000 worth of sales. Worse still, poor credit management means that you will have to expend additional time and resources to collect debts, so even if you are paid eventually, you will incur costs that are `hidden around those accounts. This scenario is not uncommon in business. On the other hand, the absence of any doubtful, as opposed to bad, debt may mean that you have been missing out on business by being overcautious. Remember that bad debts sometimes arise from disputes over the goods or services you have supplied. Thats why it is important to develop good interpersonal relationships with customers and their accounts teams; and why you should have a clear and rapid disputes escalation process, ensuring senior decision makers can resolve perceived problems to ensure problem accounts are quickly resolved.
The key to improving your ability to collect overdue accounts is to get organised. Use aged debtor analysis. Maintain a list of accounts receivable due and past due. Senior management can use it to monitor trends and control weaknesses, while credit controllers have a ready made to-do list of customers to chase. List accounts in order of size and due date, first ranking largest debt first and second ranking earliest date first. Accountancy software will typically automate this task, but even using the SORT function in a spreadsheet will work. Learn the debtors payment cycle. When dealing with large companies, find out the last day for getting an invoice approved and included in the payment run. Call a couple of days before that date to make sure that they have all the documentation from you that they need. Anticipate where you can. Consider giving a reminder call the week before your payment is due, especially if you have identified a specific group of customers which tends to pay late. Start with a serious letter. If a problem emerges, pay a solicitor to write one for you. If you want to get results, get serious from the start. Use personal visits. Letters are generally the least effective method of chasing debts (although legal letters do have more impact); emails and telephone calls score better; but personal visits are the most effective. If you have a problem with payment, talk to the person who is responsible for buying your goods or services. Point out that if the credit limit is breached, you may have to withhold your goods in future if payment is not made.
3. Credit management
Most business-to-business companies extend credit to their customers. It is often a crucial tool for attracting customers. How you manage that process is a fundamental part of cash flow management. People who owe you money, debtors, are a vital part of cash inflow and poorly managed credit can mean delays in converting sales to cash or, more seriously, trading with customers who are unable or unwilling to pay.
Credit policy
Your companys credit policy is important. It should not be arrived at by default. The board should determine your companys credit criteria, which credit rating agency you use, who is responsible for checking prospective and existing customer creditworthiness, the companys standard payment terms, the procedure for authorising any exemption and the requirements for regular reporting. The policy should be written down and kept up to date with current creditworthiness of specific customers, especially ones with large lines of credit or that increase their orders, plus warnings or notes of current poor experience. The policy should be disseminated to all sales staff, the financial controller and the board.
Credit in practice
Start your credit decision making process when first meeting with new prospective customers or clients. If necessary, consider allowing small orders to get underway quickly. This may be a reasonable level of risk and may ensure that new business is not lost. In a sales negotiation it is professional, not `anti-selling, to be upfront about terms for payment. Use an `Account Application Form that includes a paragraph for the buyer to sign, agreeing to comply with your stated payment terms and conditions of sale. On a `welcome letter restate the terms and conditions. Your `Order Confirmation forms can stress your terms and conditions. Invoices should show the payment terms boldly on the front and re-state the date the payment is due.
Its worth bearing in mind that lax credit decisions are often exploited by fraudsters. The famous `long fraud involves a customer making a series of small purchases which are paid for in full. Gradually, the supplier gains confidence and extends more and more credit. The fraudster then places a very large order, and disappears with the goods. But it neednt be fraud: a company with its own problems might attempt to trade out of trouble and go bust leaving you with massive unpaid invoices. Its a good idea to prioritise your research. The 80/20 rule suggests that 20% of your customers will generate 80% of your revenue, so list accounts in descending order of value and give the top slice a full credit check and regular review. The smaller ones do need attention, but are a lower priority, unless monitoring reveals poor payment performance.
Credit insurance
While a clear, well communicated and properly enforced credit policy will help ensure you convert sales to cash, there is an increasingly wide range of ways of managing credit and exposure to customers. Credit insurance is one such example. Taking out this kind of policy will cover either individual accounts or a businesss entire turnover. It is most commonly used in international trading, where chasing and recovering cash from customers is much harder, justifying the costs. But it can apply to any situation where large amounts of credit are extended. Not surprisingly, it is much harder, and much more expensive, to get credit insurance when the risks increase. Both the credit crunch and the recession massively increase the risk of customers defaulting on monies owed, so it is becoming much harder to obtain cover. Ask yourself: do you check the financial standing of all new customers before executing the first order? do you periodically review the financial standing of existing customers, especially those increasing their order size? do you use the telephone when checking trade references to ensure youre getting a frank opinion that might not be committed to paper? do you incentivise salespeople by cash in, rather than sales made? Do you include risk metrics in their performance targets? who supervises credit decisions and research? Who ensures the prompt collection of monies due and who is accountable if the credit position gets out of hand? do you measure the performance of your risk and credit control teams and are their incentives linked to those metrics? are your normal credit limits explicit both in terms of total indebtedness for each customer and payment period?
The accounts receivable to sales ratio looks at your investment in accounts receivable in relation to your monthly sales. Tracking this figure will help you to identify recent changes in accounts receivable. The accounts receivable to sales ratio is calculated by dividing your accounts receivable balance at the end of any given month by your total sales for the month. accounts receivable Accounts receivable to sales ratio = --------------------------------------------------------------------------------------current sales for the month A ratio of more than one readily shows that accounts receivable are greater than current monthly sales. This indicates that if this figure persists, month on month, you will soon run into cash flow problems.
An accounts payable ageing report looks almost like an accounts receivable ageing schedule. However, instead of showing the amounts your customers owe you, the payables ageing schedule is used for listing the amounts you owe your various suppliers; a breakdown by supplier of the total amount on your accounts payable balance. Most businesses prepare an accounts payable ageing schedule at the end of each month. A typical accounts payable ageing schedule consists of six columns as per the example for accounts receivable opposite. The number of columns, however, can be adjusted to meet your reporting needs. For instance, you might prefer listing the outstanding amounts in 15 day intervals rather than 30 day intervals. You should take into account your suppliers terms of trade, to which you will already have agreed. For example ( The accounts payable ageing schedule is a useful tool for analysing the makeup of your accounts payable balance. Looking at the schedule allows you to spot problems in the management of payables early enough to protect your business from any major trade credit problems. For example, if G.R.H. Unlimited was an important supplier for Technical Office Supplies Ltd, then the past due amounts listed for G.R.H. Unlimited should be paid in order to protect the trade credit established. The schedule can also be used to help manage and improve your businesss cash flow. Using the example schedule above, Technical Office Supplies will need to generate at least 11,32 5 in income to cover the current months purchases on account. Where possible you might want to think about supplier consolidation, as bigger orders should allow you to increase your power when negotiating payment terms.
Projected outgoings
Operating expenses include payroll and payroll taxes, utilities, rent, insurance and repairs and maintenance and, like the cost of goods sold, can be fixed or variable. Rent, for example, is likely to be the same amount each month, and youll probably have plenty of notice of any change. However, payroll, goods in or utilities may vary in line with your sales projections and have a seasonal aspect. Purchasing new assets for the company tends to occur when the business is expanding or when machinery needs replacing. Cash outflow in this area is generally large and irregular. Examples of fixed asset expenditure would be on new company cars, computers, vans and machinery. In a situation where banks are reluctant to provide additional funding, it makes sense to delay some of your major investments. It is critical to have a purchasing policy in place that will ensure no significant expenses are made without being approved. A `No Purchase Order, No Pay policy can be implemented if not already in place. Also, management accounting techniques like Activity Based Costing (ABC) will help you identify overheads or expenses that can be eliminated if your company is going through a rough time. Projecting for debt payments is the easiest category to predict when preparing the cash flow budget. Mortgage payments and lease hire payments will follow the schedule agreed with the lender. Only payment against an overdraft, for example, will be variable by nature.
closing balance is determined by combining the opening balance with the second periods anticipated cash inflows and cash outflows. The closing balance for the second period then becomes the third months opening cash balance and so on. If a cash flow gap, where the balance is negative at any time, is predicted early enough, you can take cash flow management steps to ensure that it is closed, or at least narrowed, in order to keep your business going. These steps might include: increase sales by lowering prices, or increasing marketing or utilisation rates (although this could worsen the gap if your cash flow management is already poor) increase margins by cutting costs and/or raising prices (although you need to be mindful of putting off customers or squeezing suppliers) tighten cash processes such as collections or inventory management decrease anticipated cash outflows by cutting back on inventory purchases or cutting operating expenses such as wages postpone a major purchase sell assets (but not those core assets essential to the business unless you can arrange a sale-and-leaseback deal on, for example, property) roll over a debt repayment (much tougher in a credit crunch) seek outside sources of cash, such as a short-term loan.
Surpluses
If your business creates a cash surplus, you have important choices: deposit the surplus cash, either overnight or on term deposit with a bank or with a proprietary money fund, to earn interest until you are ready to use it elsewhere use the cash to fund capital investment for development and expansion in line with your longer-term corporate strategy pay out money to stakeholders pay creditors early to enhance your credit credentials for the future pay down debt to improve your balance sheet gearing ratio and make future interest payments more manageable. If you choose this route, then there are considerations of whether there is a premium to be paid for early repayment and whether it restricts your future flexibility given the scarcity of credit.
Shortages
If there is a requirement for additional funds, either to meet short-term payments or for longer-term development, there are several sources of new funds: An overdraft facility. You should negotiate with the bank to agree acceptable limits and agree competitive interest rates. Youll be paying a premium over the base rate, so haggle. In fact, during the credit crunch, many companies have found that this type of unsecured lending (where the bank has no asset to claim if you default) is increasingly
rare. A short-term borrowing facility. The bank will allow you to draw down a specific amount to be repaid in a specified number of days. The limits to the facility, the repayment periods and the interest rates will be negotiated with the bank. The interest on a short-term facility may be more favourable than for an overdraft, but again, banks have become increasingly risk-averse and may prefer a more structured option. A revolving credit facility. Again, you will agree acceptable limits to the facility and agree competitive interest rates. The facility will enable you to make withdrawals at short notice. It will also enable you to make unscheduled repayments whenever you have a cash surplus. The saving on interest owed may outweigh the interest that could have been earned from a separate investment. Fixed-term loans. The finance can be loan debt or, for much larger amounts, a bond issue. The interest rate can be fixed or variable. Haggle the premium and, indeed, do not be afraid to shop around. Although you will want to maintain a good relationship with your bank, there are now many competing sources of sound finance on the market, especially since the de-mutualisation of many of the building societies. It is simply good business to take the time to establish fresh links with some of these. Fresh equity, either from a private placing of shares, in private businesses, an injection of capital by existing or new investors, or a public offering. This is an important source of funds and can be essential if the debt-equity ratio is to be maintained at acceptable levels.
Asset sales
Selling non-core assets could be an appealing solution if additional cash is required. For example, Lloyds TSB managed to gain additional funds through the sale of its Abbey Life business to Deutsche Bank in July 2008. Likewise, in 2008 RBS were looking to sell its ABN Amros Australian and New Zealand operations in the hope of securing 4 billion. And sale leaseback transactions can allow businesses to raise money by selling assets while retaining use. In 2007 HSBC
agreed the sale and leaseback of its global headquarters, which raised 1.09 billion.
Current ratio
This liquidity ratio is calculated by dividing current assets by current liabilities. It measures the ability to pay bills. Low risk Average risk High risk Over 1.5 1.01.5 Under 1.0
Current assets (cash + stocks + trade debtors) divided by current liabilities (amounts due under 1 year)
Debt/equity (gearing)
This assesses how heavily the company is relying on external funding to support the business. Low Average High Under 50% 50-90% Over 90% Debt (loans, overdraft, etc.) divided by equity (shareholders funds) x 100 An alternative definition is debt divided by (debt plus equity), which would modify the last table from the previous page: Low Average High Under 33% 33-47% Over 47%
Profit/sales
To assess profit margin of sales after costs.
again, does not factor debt repayments into his financial planning. Worse, because the new customer promises a lot of business, he expects credit hell pay for the recoveries and repairs in arrears at the end of each month. In the first week of accepting the new contract, ten recoveries are made. Fixing these cars takes longer than expected and, to maintain credibility with his big new customer, he stops taking on work from local customers and diverts effort into upholding the terms of his new contract. That means his regular sources of cash business have declined, again, lack of a cash flow forecast means he doesnt know how this will affect his viability. In week two, one of Alberts mechanics breaks his arm and is off sick, still being paid, but generating no income. He still has the workshop to run and the breakdowns to attend to. One of his recovery trucks is now lying idle, however. The servicing work is mounting up and he is deluged with breakdown requests. There is nothing for it but to hire more staff. He calls an agency and take on two more mechanics, paying weekly wages. Week three: the trucks are in full use, but Albert is splitting his time between recovery work and the few remaining servicing jobs hed already booked in. At the end of this week, cash is getting tight. Weekly wages must still be paid, but the reduction in cash customers means the bank account is emptying fast. Work in the office is still mounting up and he decides to take on an office manager so he can concentrate on the workshop to address complaints from several regulars for delays in servicing their vehicles. Meanwhile, Alberts Autos has seen no cash yet from the recoveries customer, who is so delighted with Alberts service that he proposes to increase the contract to cover an even greater distance. If Albert does not take this contract, future work from the contact may revert to just the odd recovery. If he accepts the deal, he may go out of business: cash outflows (wages, loan repayments, operating expenses) are spiralling out of control, but he has yet to be paid for the recoveries to date and the cash work from servicing is drying up. In considering whether to take the work he would have to consider not only whether or not he wants a bigger garage, but also whether he wants to expose his company to increased borrowing to finance the means of providing the recovery service (trucks, spares, trained mechanics). He would also probably lose his remaining local servicing customers. And if the recoveries work declines lets assume in month four, there are 25% fewer breakdowns locally hell be facing the same high costs, but with drastically lower cash coming in. Worse, if Alberts contact went out of business himself, Alberts Autos would still have all the costs, but no income. Expanding this business requires a solid business plan and cash flow forecast. The problem is that growth is rarely of an incremental, easily absorbed nature. It usually represents a step-change. Any manager has to ask whether they can cope, and model the cash flows to show how. In the example above, by focusing solely on the need to meet the recovery work, Albert has allowed a cash crisis to develop unnoticed and unchecked. Take the time to think big; plan your new projected cash flows, identify the shortfalls, identify the risks and secure ample lines of credit. If the step-change is revealed to be too great you will at least have spotted it in time and you can plan a different route.
8. Conclusion
As outlined at the outset, cash flow is the life blood of all businesses, and it becomes even more important when going through an economic downturn. Cash flow has to be managed. Cash management is as much an integral part of the business cycle as any other part of the process. The effect of cash is real, immediate and, if mismanaged or not managed, it is very unforgiving. It can be your ruination, but with care will be your servant and reward. Hopefully, this booklet has helped to illuminate where cash comes from and where it goes in the cash flow cycle, and provided you with insight into the basics of cash and credit management.
Introduction Over the past decade, banks have devoted many resources to developing internal risk models for the purpose of better quantifying the financial risks they face and assigning the necessary economic capital. These efforts have been recognized and encouraged by bank regulators. For example, the 1997 Market Risk Amendment (MRA) to the Basle Capital Accord formally incorporates banks internal, market risk models into regulatory capital calculations. That is, the regulatory capital requirements for banks market risk exposures are explicitly a function of the banks own value-at-risk (VaR) estimates. A key component in the implementation of the MRA was the development of standards, such as for model validation, that must be satisfied in order for banks models to be used for regulatory capital purposes. Recently, there has been a flurry of developments in the field of credit risk modeling, as evidenced by the public release of such models by a number of financial institutions; see J.P. Morgan (1998) and Credit Suisse Financial Products (1997) for examples. Credit risk is defined as the degree of value fluctuations in debt instruments and derivatives due to changes in the underlying credit quality of borrowers and counterparties. Recent proposals, such as by the International Swap Dealers Association (ISDA, 1998) and the Institute of International Finance Working Group on Capital Adequacy (IIF, 1998), argue that credit risk models should also be used to formally determine risk-adjusted, regulatory capital requirements. However, the development of the corresponding regulatory standards for credit risk models is much more challenging than for market risk models. Specifically, a major impediment to model validation (or backtesting as it is popularly known) is the small number of forecasts available with which to evaluate a models forecast accuracy. That is, while VaR models for daily, market risk calculations generate about 250 forecasts in one year, credit risk models can generally produce only one forecast per year due to their longer planning horizons. Obviously, it would take a very long time to produce sufficient observations for reasonable tests of forecast accuracy for these models. In addition, due to the nature of credit risk data, only a limited amount of historical data on credit losses is available and certainly not enough to span several macroeconomic or credit cycles. These data limitations create a serious difficulty for users own validation of credit risk models and for validation by third-parties, such as external auditors or bank regulators. Using a panel data approach, we propose in this paper several evaluation methods for
credit risk models based on cross-sectional simulation techniques that make the most use of the available data. Specifically, models are evaluated not only on their forecasts over time, but also on their forecasts at a given point in time for simulated credit portfolios. Once a models credit loss forecasts corresponding to these portfolios are generated, they can be evaluated using a variety of statistical tools, such as the binomial method commonly used for evaluating VaR models and currently embodied in the MRA. Note that, since simulated data are used, the number of forecasts and observed outcomes can be made to be as large as necessary. Although this resampling approach cannot avoid the limited number of years of available data on credit defaults and rating migrations, it does provide quantifiable measures of forecast accuracy that can be used for model validation, both for a given model and across models. These evaluation methods could be used by credit portfolio managers to choose among credit risk models as well as to examine the robustness of specific model assumptions and parameters. Supervisors could use these methods to monitor the performance of banks credit risk
1 See
Altman and Saunders (1997) for a survey of developments over the past twenty years.
management systems, either alone or relative to peer group performance. The paper is organized as follows. Section II provides a general description of credit risk models and highlights two main difficulties with conducting model validation: the lack of credit performance data over a sufficiently long time period and uncertainty about which statistical methods to use in evaluating the models forecasts. Section III presents the proposed evaluation methodology; i.e., the cross-sectional simulation approach and various statistical tools for forecast evaluation. Section IV concludes with a summary and discussion of future research.
General Issues in Credit Risk Modeling The field of credit risk modeling has developed rapidly over the past few years to become a key component in the risk management systems at financial institutions. 1 In fact, several financial institutions and consulting firms are actively marketing their credit risk models to other institutions. In essence, such models permit the user to measure the credit risk present in their asset portfolios. (Note that such models generally do not measure market-based risk factors, such as interest rate risk.) This information can be directly incorporated into many components of the users credit portfolio management, such as pricing loans, setting concentration limits and measuring risk-adjusted profitability. As summarized by the Federal Reserve System Task Force on Internal Credit Risk Models (FRSTF, 1998) and the Basle Committee on Banking Supervision (BCBS, 1999), there exists a wide variety of credit risk models that differ in their fundamental assumptions, such as their definition of credit losses; i.e., default models define credit losses as loan defaults, while
2 Note
that some credit risk models directly forecast the entire loss distribution, while others assume a parametric form for the distribution and forecast its relevant parameters. In this paper, we refer more generally to the output of these approaches as forecasted distributions.
mark-to-market or multi-state models define credit losses as ratings migrations of any magnitude. However, the common purpose of these models is to forecast the probability distribution function of losses that may arise from a banks credit portfolio.2 Such loss distributions are generally not symmetric. Since credit defaults or rating changes are not common events and since debt instruments have set payments that cap possible returns, the loss distribution is generally skewed toward zero with a long right-hand tail. Although an institution may not use the entire loss distribution for decision-making
purposes, credit risk models typically characterize the full distribution. A credit risk models loss distribution is based on two components: the multivariate distribution of the credit losses on all the credits in its portfolio and a weighting vector that characterizes its holdings of these credits. Let N represent the number of credits in a banks portfolio, and let A t, an (Nx1) vector, represent the present discounted value of these credits at time t. If the banks holdings of these credits is denoted as the (Nx1) vector wb, then the value of bank bs credit portfolio at time t is Pbt = wbAt. Once Pbt has been established, the object of interest is _Pbt+1, the change in the value of the credit portfolio from time t to time t+1, which is a function of the change in the value of the individual credits; i.e., A credit risk model, say model m, is characterized by its forecast of _Pbt+1 over a specified horizon, which is commonly set to one year. That is, the model generates a forecast F m _Pbt_1 of the cumulative distribution function of portfolio losses based on the portfolio weights wb and
3 For
simplicity we have assumed that a banks exposures or portfolio weights are known. If, as in some credit risk models, exposures have a random component, then the object of interest, , does not change, but the F m distribution of the weights, wb, must also be considered.
the distribution function of the (Nx1) random variable _At+1.3 This ability to measure credit risk clearly has the potential to greatly improve banks risk management capabilities. With the forecasted credit loss distribution in hand, the user can decide how best to manage the credit risk in a portfolio, such as by setting aside the appropriate loan loss reserves or by selling loans to reduce risk. Such developments in credit risk management have led to suggestions, such as by ISDA (1998) and IIF (1998), that bank regulators permit, as an extension to risk-based capital standards, the use of credit risk models for determining the regulatory capital to be held against credit losses. Currently, under the Basle Capital Accord, regulated banks must hold 8% capital against their risk-weighted assets, where the weights are determined according to very broad criteria. For example, all corporate loans receive a 100% weight, such that banks must hold 8% capital against such loans. Proponents of credit risk models for regulatory capital purposes argue that the models could be used to create riskweightings more closely aligned with actual credit risks and to capture the effects of portfolio diversification. These models could then be used to set credit risk capital requirements in the same way that VaR models are used to set market risk capital requirements under the MRA. However, as discussed by FRSTF (1998) and BCBS (1999), two sets of important issues must be addressed before credit risk models can be used in determining risk-based capital requirements. The first set of issues corresponds to the quality of the inputs to these models, such as accurately measuring the amount of exposure to any given credit and maintaining the internal consistency of the chosen credit rating standard. For example, Treacy and Carey (1998) discuss some of the difficulties in creating and maintaining internal ratings systems. Although such issues are challenging, they can be addressed by various qualitative monitoring procedures, both internal and external. The second set of issues regarding model specification and validation are much more difficult to address, however. The most challenging aspect of credit risk modeling is the construction of the distribution function of the (Nx1) random variable _At+1. Changes in the value of these credits will be due to a variety of factors, such as changes in individual loans credit status, general movements in market credit spreads and correlations between portfolio assets. Thus, in general, a variety of modeling assumptions and parameter values are involved in the construction of a credit risk models forecasted distribution. However, testing the validity of these model components is limited, mainly because the historical data available on the performance of different types of credits generally do not span sufficiently long time periods. To evaluate or any other distribution forecast, a relatively large number of Fm _Pbt_1 observations is needed. This evaluation issue is less of a concern for evaluating distribution
forecasts generated by VaR models. Such models generate daily forecasts of market portfolio changes, which can then be evaluated relative to the actual portfolio changes. As specified in the MRA, 250 observations (about one year) are currently used in the regulatory evaluation of such models; see Lopez (1999a,b) for a further discussion of the evaluation of VaR models. However, it is difficult to gather a large number of observed credit losses with which to evaluate credit risk models, because these models have much longer forecast horizons, typically one year. Thus, one year generates only one observation of credit losses. To literally replicate the evaluation procedure specified in the MRA, it would take an unrealistic 250 years of observations to gather
4 Note
that evaluations of models based on different forecast horizons are not directly comparable. That is, the statistical properties of monthly and annual data can be sufficiently different to prevent inference for annual data to be drawn from an evaluation conducted with monthly data.
the number specified in the MRA. Even if credit risk models of shorter horizons (say, one month) were used for regulatory evaluations, it would still take over 20 years to gather the specified number of observations.4 It is this limited ability to validate credit risk models forecasted loss distributions with actual credit losses observed over multiple credit cycles that causes severe problems in using them for determining risk-based capital requirements. In this paper, using a panel data approach, we present simulation-based evaluation methods for credit risk models that address this regulatory concern more directly than has been done to date. Our objective is to develop evaluation methods that provide quantifiable performance measures for credit risk models, even in light of the short history of available credit risk data. Although the lack of data is an insurmountable issue, our proposed methods make the most use of the available data for backtesting purposes. Of course, as more data becomes available, these methods become even more useful. In addition, several of the statistical tools employed in these evaluation methods are familiar to regulators, since they are commonly used in the evaluation of VaR models. In the following section, both the intuition and the mathematical details of these methods are described.
III. Evaluation Methodology Based on Simulated Credit Portfolios As mentioned, the data limitations for evaluating credit risk models are considerable. In terms of a panel dataset, credit data is generally plentiful in the cross-sectional dimension (i.e., N is usually large since many credits are available for study), but scarce in the time dimension. This limitation has led the users of credit risk models to construct alternative methods for validating these models. For example, credit risk models have been evaluated using stress testing. For this method, a models performance is evaluated with respect to event scenarios, whether artificially constructed or based on historical outcomes. One possible stress scenario would be the simultaneous default of several sovereign borrowers. Once the scenarios are specified, the models forecasts under these circumstances are examined to see if they intuitively make sense. Although such a practice may provide a consistency check regarding the models various assumptions, these scenarios generally do not occur. Recently, researchers have begun comparing the forecasts from different credit risk models given similar assumptions about their underlying parameters; see, for example, Crouhy and Mark (1998), Gordy (1998), and Koyluoglu and Hickman (1998). However, as per the evaluation of VaR models, the ability to compare a credit risk models forecasts to actually observed outcomes is more desirable. For example, Nickell, Perraudin, and Varotto (1998) use actual prices for a set of publicly traded bonds to compare the performance of credit loss forecasts from two types of credit risk models. In this paper, we present evaluation methods that
specifically focus on quantitative comparisons between credit loss distribution forecasts and observed credit losses. A. Intuition from time-series analysis As outlined in Granger and Huang (1997), methods commonly used for model specification and forecast evaluation in time-series analysis can be adapted for use with panel9 data analysis, such as credit risk modeling. The general idea behind forecast evaluation in time-series analysis is to test whether a series of out-of-sample forecasts (i.e., forecasts of observed data not used to estimate the model) exhibit properties characteristic of accurate forecasts. For example, an important characteristic of point forecasts is that their errors (i.e., the differences between the forecasted and observed amounts) be independent of each other, which is an outcome of properly specified models. See Diebold and Lopez (1996) for further discussion. This idea can be extended to the cross-sectional element of panel data analysis. In any given year, out-of-sample predictions for cross-sectional observations not used to estimate the model can be used to evaluate its accuracy. As long as these additional out-of-sample observations are drawn independently from the cross-sectional sample population, the observed prediction errors should be independent. Standard tests for the properties of optimal predictions can then be used to test the cross-sectional models accuracy. For evaluating credit risk models, we propose to use simulation methods to generate the additional observations of credit portfolio losses needed for model evaluation. That is, the models in question can be used to forecast the corresponding loss distributions for the simulated portfolios, and these forecasts and corresponding observed losses can then be used to evaluate the accuracy of the models. Since simulation techniques are used, we can generate as many observations as we might need. The simulation method used here to generate these additional credit portfolios is simply resampling with replacement from the original panel dataset of credits; see Carey (1998) for a related methodology in which credit portfolios are randomly drawn from a dataset to construct a nonparametric distribution of credit losses for specific portfolio strategies.
5 In
practice, two types of credits are commonly analyzed using credit risk models. The first type are basic debt instruments, such as loans or debentures. The second type is the expected future exposures arising from derivative positions; i.e., forecasts of the expected size of the credit exposure. In this paper, we take such expected future exposures as inputs to the credit risk models. 6 Note that for the purposes of model validation, the forecast evaluation should not be greatly affected by using a simple, binary weighting scheme. Of course, other weighting schemes could be used, if so desired. Certainly, the weighting scheme is of major importance when credit risk models are actually being used for risk management purposes. 7 Note that only unique credit portfolios should be used; if a duplicate were to be drawn, it should be discarded and replaced with another. 8 The number of possible different portfolios that could be resampled from a set of N credits using a binary weighting scheme is 2N. The choice of the number of simulations, denoted as R, should be less than that number. The 1,000 simulations suggested here is appropriate for occasions where N A 10; i.e., 1000 < 1024 = 210. Additional research is needed to fully understand the impact of the R, N, and _ resampling parameters on the
B. Evaluation Methods for a Credit Risk Model Having generated (T * R) resampled credit portfolios, we can generate the corresponding predicted cumulative density functions of credit losses, denoted with i=1,...,R and F m _Pit_1 t=1,...,T, for model m. To evaluate the models forecast accuracy, we can examine the properties of certain characteristics of these distributions, such as their means. Below, we focus on three hypothesis testing methods, which are also used for evaluating VaR models.
10 As
mentioned previously, credit losses are due to either defaults or rating migrations (for example, a credit downgrade from AAA to A). Defaults and the associated losses can readily be observed, but rating migrations and
corresponding losses may not be. In this paper, we assume that such losses are observable. If they are not or are the outcomes of pricing models, additional uncertainties are introduced into the credit model evaluations, as further discussed in Section D.
12 (i). Evaluating forecasts of expected loss We can evaluate the accuracy of the models predicted expected losses by comparing them to the actual observed losses on the resampled portfolios. The predicted expected loss in year t for portfolio i at time t+1, denoted , is simply the expected value of . If the mit F m _Pit_1 specified credit risk model is accurate, then the difference between the predicted expected losses and the observed losses should be zero on average across the full simulated sample. Let denote the observed loss on resampled portfolio i in year t+1. Note that L it+1 Lit_1 _ w_ i _At_1 can be defined in terms of either losses due to credit defaults or credit migrations.10 The null hypothesis that the prediction errors emit+1 = Lit+1 - have mean zero implies mit that the model is accurate. A large number of test statistics are available to test this hypothesis. For example, we can use the intuition of Mincer-Zarnowitz regressions to test this property. Specifically, for the regression L (where is an error term), it should it_1 _ _ _ _ mit _ _it_1 _it_1 be the case that _=0 and _=1, which implies that the mean of the prediction error is zero. (ii). Evaluating forecasted critical values A commonly-used output from credit risk models is a specified quantile or critical value of the forecasted loss distribution; i.e. the dollar amount of losses that will not be exceeded with a given probability. The evaluation of these forecasted critical values, denotedCV m _,_Pit_1 for the upper _% critical value from with i=1,...,R and t=1,...,T, can be conducted Fm _Pit_1 LR(_) _ 2 log _y(1_ _)T_R_y _ log _y(1__)T_R_y , along the lines of the evaluation of VaR estimates from market risk models. Nickell, Perraudin, and Varotto (1998) present comparative evidence of this type. Specifically, we can use the binomial method to evaluate these forecasted critical values. Under the assumption that these forecasted critical values are accurate, the exceptions -occasions when actual credit losses exceed the forecasted critical values -- can be modeled as draws from an independent binomial random variable with a probability of occurrence equal to the specified _ percent. We can test whether the percentage of observed exceptions, denoted _ , equals _ using the likelihood ratio statistic where y is the number of exceptions in the sample. This LR(_) statistic is asymptotically distributed as a _ random variable, and the null hypothesis that _ = can be rejected at the
2 (1)
_ five percent level if LR(_) > 6.64. As noted by Kupiec (1995) and Lopez (1999a,b), the power of such tests can be quite low in small samples. However, since we can control the sample size by increasing R within certain limits, this should not be such a concern. (iii). Evaluating forecasted loss distributions Given that credit risk models generate full distributions of credit losses, it is reasonable to evaluate the accuracy of the (T * R) distribution forecasts themselves. In the context of evaluating VaR models, Crnkovic & Drachman (1996) propose specific tests for evaluating forecasted distributions; see Diebold, Gunther & Tay (1997) and Berkowitz (1999) for further discussion. The object of interest for such hypothesis tests is the observed quantile q mit+1, which
is the quantile under corresponding to the observed credit loss L it+1; i.e., Fm _Pit_1
11 Note
that comparisons between default models and multi-state models based credit migrations are complicated by their different definitions of credit losses. The comparisons of forecast accuracy can be conducted since both types of models generate credit loss distributions, but the results may not be as meaningful as comparisons across models with the same loss definition.
14 . These hypothesis tests examine q whether the observed quantiles derived mit_1 _ Fm Lit_1 under a models distribution forecasts exhibit the properties of observed quantiles from accurate distribution forecasts. Specifically, since the quantiles of independent draws from a distribution are uniformly distributed over the unit interval, the observed quantiles under model m should also be independent and uniformly distributed. These two properties are typically tested separately.
C. Evaluation Methods Across Credit Risk Models As suggested by ISDA (1998), there exist credit risk models of different levels of sophistication. Users may, over time, wish to upgrade the model they use or simply change an assumption in their model. The simulation method described above can be used to make meaningful comparisons on the relative performance between credit risk models. 11 As before, we generate the resampled portfolios and the corresponding, forecasted loss distributions from both models, denoted and , for i= 1,...,R and t=1,...,T. Here again, we can compare various F1it F
2it
elements of these distributions, such as their expected losses (or means) and specific tail regions, using slightly different statistical tools.
(i). Evaluating forecasts of expected loss To evaluate the two competing sets of expected loss forecasts, we can examine the two sets of forecast errors between the realized and expected losses, denoted e and 1it_1 _ Lit_1 _ 1it e . As described by Granger and Huang (1997), a number of tests are available 2it_1 _ Lit_1 _ 2it for evaluating these forecast errors. (a). The count method Let e and denote the squared forecast errors for resampled portfolio i. The null 2 1it_1 e 2
2it_1
hypothesis that the two credit risk models are equally accurate across the R resampled portfolios in a given year is expressed as H A simple test of this hypothesis is to count 0: E e 2 1_ E e 2 2. the number of times that is greater than , which we denote as p 1e . If the null hypothesis 2 1it_1 e 2
2it_1
is correct, then the random variable p1/R should have a normal distribution with a mean of 0.5 and a variance of 1/(4*R). We can reject the null hypothesis at the five percent level if the observed value of p1/R lies outside the range [0.5-R-1/2, 0.5+R-1/2]. Although this is not particularly powerful test, it is robust to any covariance between the two forecast errors as well as any heteroskedasticity of the individual errors.
(b). Sum/difference regressions A second test for evaluating the models predicted expected losses is based on analyzing the sum and difference of the forecast errors. For a given year, let S it+1 be the sum of the e1it+1 and e2it+1 errors, and let Dit+1 be their difference. The regression S is then it_1 _ _ _ _Dit_1 _ _it_1 run, and the null hypothesis that _ = 0 is examined using the t-test based on Whites robust standard errors. If we reject the null hypothesis, then the two models are not equally accurate, and the model with the lower forecast error variance should be considered to be more accurate.
(c). Analysis under a general loss function In the discussion of the count method above, the quadratic loss function f(x) = x 2 is implied. Diebold and Mariano (1995) suggest testing the same null hypothesis under the general loss function g(x); i.e., H They propose various asymptotic and 0: _ E g e1 _ E g e2 . finite-sample statistics to test this null hypothesis. If we reject the null hypothesis, then the two models generate forecasts of differing quality, and we should select the model whose forecasts have the lowest value under the loss function g(x). (ii). Evaluating forecasted critical values The evaluation criteria described in the section above focuses specifically on the forecasted expected losses under two competing models. However, the relative performance of other aspects of the distribution forecasts, such as their forecasted _% critical values, may also be of interest. The binomial test used to evaluate the critical value forecasts from one models forecasted distributions can be adapted to examine the performance of the two competing forecasts. Specifically, a Bonferroni bounds test with size bounded above by k% can be conducted to test the null hypothesis that the forecasted _% critical values from each model are accurate and provide the expected _% coverage. For this test, we conduct binomial tests individually for each set of forecasts with the smaller size of k/2%. The null hypothesis that the two models are equally accurate is rejected if either set of forecasts rejects the binomial null hypothesis. If the null hypothesis is rejected by just one set of forecasts, then the other set can be said to be more accurate. Note that this multi-model testing is conservative, even asymptotically. 17 (iii). Evaluating forecasted loss distributions Similarly, Bonferroni bounds tests based on the hypothesis tests proposed by Crnkovic & Drachman (1996), Diebold et al. (1997) and Berkowitz (1999) can be constructed for comparing two sets of forecasted loss distributions. See Diebold, Hahn and Tay (1998) for further discussion of multivariate density forecast evaluation. D. Limitations to the Proposed Evaluation Methodology The proposed simulation approach permits the comparison of a models forecasted credit loss distributions to actually observed outcomes, as in the standard backtests performed for VaR estimates. However, a few important limitations must be kept in mind when using this approach. First, changes in the value of a credit portfolio are mainly driven by three factors: changes in credit quality; changes in the value of credits of given quality (possibly due to changes in credit spreads); and changes in the portfolio weights of the credits. The first two elements, changes in credit quality and valuation, are captured in this methodology by _At+1, the changes in the value of the credits. Thus, model performance can be evaluated along these dimensions using the resampling approach outlined here, with one notable exception. Many credit instruments, especially potential future exposures, do not have observable market prices and must instead be marked-to-model or priced according to a valuation model. This process introduces a source of potential error not captured by this methodology. As to the third element, this methodology
generally takes actual and simulated portfolio weights as fixed parameters over the planning horizon; thus, changes in portfolio weights are not directly addressed by this methodology. Second, credit risk models are essentially panel data models used for datasets where the 18 number of assets N is much greater than the number of years T. This limited amount of credit default and migration data over time complicates the forecast evaluation of these models because the results are inextricably tied to the prevailing macroeconomic and credit conditions over the T year period. Our approach permits model validation using quantitative measures of model performance, but these measures must obviously be interpreted with care if T is small and does not span a business or credit cycle. As more years of data become available, the resampling of credit portfolios under different economic conditions provides for more extensive evaluation of a credit model's accuracy. As data for the time dimension becomes available, a credit risk models performance can be evaluated according to both dimensions using these criteria. For example, resampled portfolios can be randomly constructed for a randomly selected year, and the models performance could be examined as described. This approach could be interpreted as evaluating a models conditional performance. Alternatively, resampled portfolios and the forecasted outcomes could be examined across the available time period. As more years of data are available, we can be more confident that we are evaluating a models unconditional performance. Further study is needed to understand the properties of such evaluations. Finally, as presented, this resampling approach is wedded to the original set of N credits. Since institutions actively manage their existing credit portfolios and originate new credits, this limitation essentially causes the evaluation of the models performance to be static. Although static diagnostic tools may be useful under some circumstances, the implications for credit risk models used by firms with dynamic credit exposure are less clear. In such cases, it is reasonable to sample for each year t from the Nt credits held at the end of that year; this generalization is 19 straightforward. Conclusions In general, the evaluation of credit risk models will always be more difficult than market risk models because of their underlying time horizons. The evaluations of the distribution forecasts generated by both of these types of models require a relatively large number of forecasts and observed outcomes. Certainly, the daily horizon underpinning market models guarantees a steady stream of observations over which to evaluate forecasts. However, the yearly horizon commonly used for credit risk models does not. Thus, qualitative methods, such as stress-testing and sensitivity analysis, will always be important in the evaluation of credit risk models. In this paper, we propose evaluation methods based on statistical resampling that can provide quantitative measures of model accuracy for credit risk models. These methods provide performance evaluation in a cross-sectional environment. The proposed statistical tools are relatively simple; are well known in the forecast evaluation and risk management literatures; and are general enough to be used on any type of credit risk model. Although important caveats must be attached to any inference drawn from this type of evaluation, at least some is now available where previously there was little. Several aspects of the proposed evaluation methodology require additional research. For example, the impact of specific parameters, such as the number of credits to be included in a simulated portfolio and the nature of the simulated portfolios weights, must be better understood. However, most of the future research in this area should be on actual comparisons of credit risk models over various credit datasets.