Beyond Roe How To Measure Bank Performance 201009 en

B E YO N D RO E H OW TO M E A S U R E BANk pERfORMANcE
SEpTEMBER 2010
AppENDix TO T H E REpORT ON EU BANkiNg STRUcTURES
BEYOND ROE HOW TO MEASURE BANK PERFORMANCE Appendix to the report on EU banking structures
In 2010 all ECB publications feature a motif taken from the 500 banknote.
S E P T E M B E R 2010
European Central Bank, 2010 Address Kaiserstrasse 29 60311 Frankfurt am Main Germany Postal address Postfach 16 03 19 60066 Frankfurt am Main Germany Telephone +49 69 1344 0 Website http://www.ecb.europa.eu Fax +49 69 1344 6000 All rights reserved. Reproduction for educational and non-commercial purposes is permitted provided that the source is acknowledged. ISBN 978-92-899-0322-6 (online)
CONTENTS
1 EXECUTIVE SUMMARY 2 WHAT IS PERFORMANCE MEASUREMENT? WHAT IS IT USED FOR? 2.1 Denition and approaches to performance measurement 2.2 The scope of performance measurement analysis and its reection in metrics or models applied 3 WHAT IS WRONG WITH ROE? 3.1 What did RoE tell us? Empirical evidence from large banks RoE trend and changes 3.2 What drives RoE? 3.3 Caveats to relying on RoE in the assessment of bank performance 4 REFINEMENTS AND ISSUES TO CONSIDER 5 ANNEXES 1 8 2 8 3 Approach, questionnaire, workshop participants and respondents Questionnaire on performance measures References 37
37 38 42
11
15 18 20 26
4.1 Renements in terms of scope 26 4.2 Renements in terms of properties of performance measures 2 8 5 ADDITIONAL FACTORS AND ALTERNATIVE MEASURES OF PERFORMANCE 5.1 Why do business models matter? 5.2 Economic capital models and stress test results are key elements of a performance measurement framework 5.3 Governance and performance: incentive schemes based on value creation may have an effect on performance 5.4 Governance and performance: disclosure and market discipline 6 CONCLUSION: WHICH IS THE WAY FORWARD?
30 30
33
34 35 36
ECB Beyond RoE How to measure bank performance September 2010
1 EXECUTIVE SUMMARY
What is an acceptable level of return on equity (RoE) for a bank? This question is likely to play a pivotal role in the post-crisis debate among banking executives, investors and regulators. Following the spectacular losses in the nancial crisis and the massive government intervention, there is little public support for banks returning RoE ratios of well above 20%, as these have mostly proved to be unsustainable. Recent events have shown that the most common measure for a banks performance, i.e. RoE, is only part of the story, as a good level of RoE may either reect a good level of protability or more limited equity capital. In addition, although the traditional decomposition of the RoE measure (i.e. looking at banks operational performance, risk prole and leverage) may have been useful to assess banks performance during benign times, this approach has clearly not proven adequate in an environment of much higher volatility such as during the global nancial crisis, where RoE uctuations have been caused entirely by operational performance, which does not aid our understanding of the potential trade-off between risk and return in performance. This may actually explain why some of the high-RoE rms have performed particularly poorly over the crisis, dragged down by a rapid leverage adjustment. Against this backdrop, there is obviously room for taking a step back from the rather consensual market valuation of performance through RoE and carrying out a more comprehensive assessment of banks performance. This sets the context for the current report by the BSC that aims: (i) to analyse issues relating to bank performance measurement and to examine why the commonly used RoE measure may not be sufcient to characterise banks performance; (ii) to look at what may be missing in this type of approach; and (iii) to identify potentially complementary approaches to RoE. The ndings and conclusions in this report are based on information drawn from a variety of sources, including a review of the academic and practitioner publications on the topic, an analysis of main RoE drivers on the basis of a set of bank case studies, a workshop with market participants, a survey of relevant practices among nine market participants, and the expertise of supervisory authorities and central banks. The capacity to generate sustainable protability is used in this report as a denition for describing banks performance. Protability is essential for a bank to maintain ongoing activity and for its investors to obtain fair returns; but it is also crucial for supervisors, as it guarantees more resilient solvency ratios, even in the context of a riskier business environment. Indeed, retained earnings appear to be one of the most important drivers of Tier 1 ratios. The main drivers of banks protability remain earnings, efciency, risk-taking and leverage. Various stakeholders (e.g. depositors, debt or equity holders and managers) emphasise different aspects of protability. These views need to be taken into account by market participants (i.e. analysts, rating agencies, consultants and supervisors) when looking at ways of measuring bank performance that meet their needs. For this, each different group of market participants has its own preferred set of indicators. In order to demystify RoE, the report details the misconceptions and limitations of its use on the basis of case studies differentiating between banks driven largely by investment activities and banks driven largely by traditional deposit-lending activities. The analysis points to an initial limitation of RoE, namely that it is not risk-sensitive. A decomposition of RoE shows that a risk component represented by leverage can boost RoE in a substantial manner. Other risk elements, on the other hand, are missing in the RoE gure (e.g. the proportion of risky assets and the solvency situation). RoE is therefore not a stand-alone performance measure,
1 EXECUTIVE SUMMARY
and decomposition or further information is necessary to identify the origin of developments and possible distortions over time. The recent crisis has shown how RoE failed to discriminate the best performing banks from the others in terms of sustainability of their results. RoE is a short-term indicator and must be interpreted as a snapshot of the current health of institutions. It does not take into account either institutions long-term strategy or the long-term damages caused by the crisis. Its weaknesses are even more obvious in times of stress, when there is a climate of uncertainty surrounding the medium-term protability of institutions. In challenging times, extraordinary elements may become very signicant, but fail to appear in reported RoE measures. As a matter of fact, RoE does not reect the sustainable performance of the bank, if the change comes from a one-shot element that cannot be reproduced in the future. RoE from continuing operations proves to be more relevant for comparing institutions and assessing operating performance accurately. RoE measures more generally fail to take into account measures with a long-term impact (e.g. restructurings and consolidation), thereby posing an additional challenge to performance analysis. Finally, RoE measures can be misleading, be manipulated or provide wrong incentives as they are inuenced by quite strong seasonal
factors, rely on data and expose banks to higher unexpected risk levels. In order to come up with a more informed assessment of banks performance, RoE must be rened. In particular, desirable features of banks performance measures may cover comparability, stability over time and the capacity to be forward-looking as well. In that context, complementary measures to RoE, such as risk returns, funding capacity, assets and own funds quality, cost of equity and capital allocation across business lines may be of some help. Alternative approaches to measuring banks performance may require a deeper analysis of the way in which banks run their business and make use of their stress-testing results, or even further enhancement of their high-level discussions with supervisors on consistency between performance and business strategy. This may eventually call for more transparency from banks on their protability structure, and some adjustment in the governance process, as suggested in the proposals for enhancing Basel II. Among other things, these measures comprise a reassessment of the risk function with respect to its independence and the available tools and an adequate level of risk awareness at the top-tier management level. As a result, there may be some opportunity here for regulators to address these issues with bank managers. To summarise, the main messages of this analytical work are as follows:
1. RoE may be less of a performance benchmark than a communication tool in the relationship between banks and markets. 2. A comprehensive performance analysis framework needs to go beyond that kind of indicator though not excluding it and provide for a more informed assessment, using banks business-based data and qualitative information. 3. The consistency of risk appetite with the business structure and strategy appears to be one of the most crucial elements in assessing an institutions capacity to deliver performance in the future. Against this backdrop, sustainable indicators constructed on the basis of economic capital models and nancial planning frameworks inside the banks may become even more
1 EXECUTIVE SUMMARY relevant. For instance, risk-adjusted types of returns indicators, such as RAROC, may benet from higher disclosure and better explanation to the markets, or at least to the supervisors. 4. Desirable features for banks performance measures should encompass more aspects of the performance than just protability embedded in a pure market-oriented indicator such as RoE. In particular, it may be useful to take account of the quality of assets, the funding capacity and the risk associated with the production of value. In that context, a good performance measurement framework should incorporate more forward-looking indicators and be less prone to manipulation from the markets. 5. In the context of achieving a comprehensive analysis for all business areas, data availability and comparability are key factors. This may call for enhanced disclosure (both towards the supervisors and, where possible, towards the public) and improved market discipline. 6. As regards governance, the adoption of a more comprehensive and more forward-looking assessment of performance may be a rst step towards intensifying the dialogue with the banks top-tier of management, related to the coherence between economic performance, business model and supervisory and nancial stability issues. The report adopts the following structure: Chapter 2 starts by setting the context for measuring bank performance: bank performance is dened and the main drivers of protability are outlined. In particular, this chapter identies the different angles under which performance measurement can be approached. Chapter 3 illustrates, on the basis of empirical evidence, the misconceptions and limitations of RoE measures. Chapter 4 outlines the reports suggestions for renements in both scope and properties of performance measurement, and addresses issues to consider when applying it. Chapter 5 elaborates on various additional factors and alternative ways of measuring performance, before Chapter 6 concludes.
2 WHAT IS PERFORMANCE MEASUREMENT? WHAT IS IT USED FOR?

2.1 DEFINITION AND APPROACHES TO PERFORMANCE MEASUREMENT This report analyses bank performance in terms of its capacity to generate sustainable protability. Protability is a banks rst line of defence against unexpected losses, as it strengthens its capital position and improves future protability through the investment of retained earnings. An institution that persistently makes a loss will ultimately deplete its capital base, which in turn puts equity and debt holders at risk. Moreover, since the ultimate purpose of any prot-seeking organisation is to preserve and create wealth for its owners, the banks return on equity (RoE) needs to be greater than its cost of equity in order to create shareholder value. Although banking institutions have become increasingly complex, the key drivers of their performance remain earnings, efciency, risk-taking and leverage. In detail: while it is clear that a bank must be able to generate earnings, it is also important to take account of the composition and volatility of those earnings. Efciency refers to the banks ability to generate revenue from a given amount of assets and to make prot from a given source of income. Risk-taking is reected in the necessary adjustments to earnings for the undertaken risks to generate them (e.g. credit-risk cost over the cycle). Leverage might improve results in the upswing in the way it functions as a multiplier but, conversely, it can also make it more likely for a bank to fail, due to rare, unexpected losses. There are a multitude of measures used to assess bank performance, with each group of stakeholders having its own focus of interest. Box 1 gives an indicative, but non-exhaustive list of indicators commonly used to measure bank performance.
Box 1
AN OVERVIEW OF PERFORMANCE MEASURES FOR FINANCIAL INSTITUTIONS Among the large set of performance measures for banks used by academics and practitioners alike, a distinction can be made between traditional, economic and market-based measures of performance. Traditional measures of performance Traditional performance measures are similar to those applied in other industries, with return on assets (RoA), return on equity (RoE) or cost-to-income ratio being the most widely used. In addition, given the importance of the intermediation function for banks, net interest margin is typically monitored. The return on assets (RoA) is the net income for the year divided by total assets, usually the average value over the year. return on assets = net income / average total assets RoE is an internal performance measure of shareholder value, and it is by far the most popular measure of performance, since: (i) it proposes a direct assessment of the nancial return of a shareholders investment; (ii) it is easily available for analysts, only relying upon public information; and (iii) it allows for comparison between different companies or different sectors
of the economy. RoE is sometimes decomposed into separate drivers: this is called the Dupont analysis, where RoE = (result/turnover)*(turnover/total assets)*(total assets/equity). The rst element is the net prot margin and the last corresponds to the nancial leverage multiplier. return on equity = net income / average total equity The cost-to-income ratios shows the ability of the institution to generate prots from a given revenue stream. Impairment charges are not included in the numerator. cost-to-income ratio = operating expenses / operating revenues Finally, the net interest margin is a proxy for the income generation capacity of the intermediation function of banks. net interest margin = net interest income / assets (or interest-bearing assets) Economic measures of performance The economic measures of performance take into account the development of shareholder value creation and aim at assessing, for any given scal year, the economic results generated by a company from its economic assets (as part of its balance sheet). These measures mainly focus on efciency as a central element of performance, but generally have high levels of information requirements. Two sets of indicators can then be identied amongst economic measures of performance: 1) Indicators related to the total return of an investment, based on the concept of an opportunity cost; the most popular one being economic value added (EVA). EVA = return on invested funds (weighted average cost of capital * invested capital) (weighted average cost of debt * net debt) (Developed by Stern and Stewart in 1991, EVA takes into account the opportunity cost for stockholders to hold equity in a bank, measuring whether a company generates an economic rate of return higher than the cost of invested capital in order to increase the market value of the company.) 2) Indicators related to the underlying level of risk associated with banks activity. According to Kimball (1998), for a bank to be successful in its operations, managers must weigh complex trade-offs between growth, return and risk, favouring the adoption of risk-adjusted metrics. RAROC (risk-adjusted return on capital, i.e. the expected result over economic capital) allows banks to allocate capital to individual business units according to their individual business risk. As a performance evaluation tool, it then assigns capital to business units based on their anticipated economic value added.1
1 There are many different measures and different types of indicators under the generic name of RAROC: RORAA (return on risk-adjusted assets), RAROA (risk-adjusted return on assets), RORAC (return on risk-adjusted capital).
The theoretical RAROC can be extracted from the one-factor CAPM as the excess return on the market per unit of market risk (the market price of risk). This measure shares in common with the EVA that it takes into account the banks cost of capital. But RAROC goes further because it adjusts the value-added in relation to the capital needed. However, literature is quite critical of this measure as a tool to analyse performance, essentially due to its thorough accounting basis 2, while it is then difcult to calculate RAROC without having access to internal data. Furthermore, it appears that RAROC may be appropriate for activities with robust techniques for measuring statistical risk, such as credit activity. On the contrary it may be less relevant for market activities, given that the value-at-risk (VaR) is still a very imperfect measurement of risk. Market-based measures of performance Market-based measures of performance characterise the way the capital markets value the activity of any given company, compared with its estimated accounting or economic value. The most commonly used metrics include: the total share return (TSR), the ratio of dividends and increase of the stock value over the market stock price; the price-earnings ratio (P/E), a ratio of the nancial results of the company over its share price; the price-to-book value (P/B), which relates the market value of stockholders equity to its book value; the credit default swap (CDS), which is the cost of insuring an unsecured bond of the institution for a given time period.
2 See, for example, Weissenrieder, F. (1997) and Fernandez, P. (2002).
Inevitably, different stakeholders in a bank view performance from different angles. For example, depositors are interested in a banks long-term ability to look after their savings; their interests are safeguarded by supervisory authorities. Debt holders, on the other hand, look at how a bank is able to repay its obligations; a concern taken up by rating agencies. Equity holders, for their part, focus on prot generation, i.e. on ensuring a future return on their current holding. This focus is reected in the valuation approaches of banks analysts, who try to identify the
fundamental value of the rm. Managers, too, seek prot generation, but are subject to principal-agent considerations and need to take employee requests into consideration. The view of bank consultancies might also encompass the internal struggle of managers. This report encompasses a broad range of views drawn from a variety of sources. It includes a review of the literature on the topic, examines related case studies, assesses the conclusions of a workshop held with market participants and
10
the results of a survey of relevant practices carried out among nine market participants, and draws on the expertise of supervisory authorities and central banks.1
the bank analysts, consultants also consider liquidity indicators and the systemic signicance of the bank to be less informative, although they acknowledge that these indicators could have been helpful around the time of the crisis. Rating agencies follow a more holistic approach, in line with their objective of assigning grades for the overall assessment of the banks. They consider all types of prudential returns (e.g. capital, asset quality, liquidity) to be integral in measuring the performance of a bank. They also assign equal weight to efciency indicators and revenue/cost composition. Moreover, they take a more dynamic approach, paying attention to changes in the level and composition of revenue and cost elements, as well as trying to incorporate market-based indicators into their analysis. Interestingly, none of the market analysts appear to have adequately incorporated the systemic relevance of a bank into their considerations of performance measurement. Given that banks interlinkages have been one of the key factors behind the crisis spreading to different countries and sectors, it is even more important to remember that bank performance cannot only be assessed at the individual level. This highlights the importance of developing a macro-prudential framework to incorporate elements of performance measurement. The different perspectives of performance measurement that have already been identied (see Box 1) are also reected in the different metrics chosen by the various analysts. It is worth noting, however, that such choices are generally dictated by the availability and quality of the data. Most analysts are therefore calling for a general improvement in the quality and disclosure of certain indicators of performance, which may thus play a central role in the future.
1 Experts from UniCredit, JP Morgan, Morgan Stanley, Bank of Valetta, Cheuvreux, KBW, Oliver Wyman, McKinsey, Fitch Ratings and Moodys Investor Services as well as from EU central banks or supervisory authorities provided input to the questionnaire or workshop. Among those interviewed.
2.2 THE SCOPE OF PERFORMANCE MEASUREMENT ANALYSIS AND ITS REFLECTION IN METRICS OR MODELS APPLIED It is crucial to identify the scope of performance measurement analysis, since this can indicate where potential alternatives to traditional metrics, such as the RoE, may be preferable. In this respect, bank analysts 2 tend to consider efciency, asset quality and capital adequacy indicators as key elements of banks performance measures. Hence, explicit indicators of credit risk and shock absorption capacity are considered essential in assessing the performance of a bank and encompassing risk in the analysis. Their analyses also rely upon detailed revenue and cost indicators (e.g. the structure, sustainability and rate of change of revenue and cost items), as well as market-based indicators of protability and valuation (e.g. P/E, P/BV). On the other hand, in assessing banks performance, bank analysts tend not to use liquidity indicators, market-based indicators of credit risk, the systemic signicance of the bank and efciency indicators related to capital, primarily because these indicators provide less reliable information. With efciency indicators, for example, it is often difcult to gauge the actual amount of capital allocated to each line of business, whereas with market-based indicators, the problem is more that they mirror other indicators and are already reected in the banks valuation. Bank consultants that were interviewed seem to adopt a narrow denition of performance measures. They place efciency indicators both traditional and capital-adjusted at the core of their performance analysis and consider revenue, asset quality and capital adequacy as secondary measures. Interestingly though, they consider market-based indicators, including bond spreads and CDS, to be useful. As with
11
Bank analysts are comfortable with traditional indicators of revenue and credit risk (see Table 1). In addition, they have introduced the concept of tangible equity into their assessments, which may better reect the impact of the crisis. It should also be noted that some of the respondents expressed reservations about the interpretation of CDS spreads, since such spreads may be illiquid and reect speculative activity. Regarding market-based indicators, bank analysts also regarded unsecured debt indicators as informative, whereas subordinated liabilities were deemed to suffer from bond specicities and illiquidity. Moreover, some specic issues addressed by bank analysts may be important in measuring performance. First, the assessment of the sustainability of bank revenues, in which the focus is generally on the share of non-recurring revenues (e.g. trading income, non-retail fee income and other income), the volatility of revenues and the evolution of net interest and fee income over the cycle, as well as the business strategy and the position against peers.3 Second,
the challenge of incorporating off-balance sheet activities into the performance assessment, although there are efforts to incorporate off-balance sheet activities into all three pillars under the proposals for enhancements to the Basel II framework. Many respondents explicitly acknowledged that the inclusion of off-balance sheet activities is very difcult, if not impossible, primarily because of the different way these activities are recorded. Some respondents indicated that, to some extent, off-balance sheet activities are already incorporated in risk-weighted assets for operational and market risk, whereas others noted that an additional qualitative judgement was always necessary. Consultants regard the monitoring of developments in net interest income as central, but view the volatility and potential
3 In the case of non-listed entities, the most common approach is to benchmark against comparable banks, contrasting the development of their key gures and indicators (e.g. debt spread, CDS) against peers, complemented by an assessment of the business model and a valuation based on assets, equity and deposits.
Table 1 Analyst indicator preference ranking by category
Category of indicators Revenue and cost
Type Net interest income metrics - including after the deduction of impairment charges - net interest income/interest-bearing assets Share of key income sources - Share of trading income Return on tangible equity Cost-to-income - Cost-to-income including impairments Return-on-risk-weighted assets Return on equity (RoE) Return on assets (RoA) Price-to-tangible equity (P/TE) Credit default swap (CDS) Price-to-earnings (P/E) Senior debt spread Distance to default (DD) Impairment charges as a percentage of total loans Coverage ratio Non-performing loans (NPLs) ratio - Net NPLs/regulatory own funds
Bank analysts 1
Consultants 1
Rating agencies
1 2 2 1 2 2 1 3 3 1 2 3 1 2 2 1 3 1 3 2 3 1 2 3 1 2 4 3 1 2
Efciency
Market-based
Credit risk
Note: The ranks were derived by averaging respondents replies to question 2 of the questionnaire (see Annex 2).
12
pro-cyclicality of income sources (e.g. share of trading income) as being less important. Regarding efciency indicators, they give more emphasis to risk-adjusted metrics, such as the return-on-risk-weighted assets and the cost-to-income ratio after the incorporation of the credit risk cost (i.e. impairment charges) in the numerator (i.e. operating expenses). Consultants seem to distrust traditional indicators, such as the RoE and the simple cost-to-income ratio. Regarding market-based and credit-risk indicators, their views are very similar to those of bank analysts. To assess the sustainability of bank revenues, consultants identify either the share of core banking income (i.e. net interest, as well as fee and commission income) or the share of non-recurring revenue (e.g. income from fees, excluding fees related to loans, trading income and other one-off gains). This analysis is supplemented by considering the volatility of revenues and the breakdown of the key income drivers. Regarding off-balance sheet activities, one respondent suggested that they could be taken into account through an annotated simulation and scenario analysis. Rating agency analysts try primarily to assess the sustainability of income sources: the net interest margin reects the ability to generate recurring income, whereas the share of trading income indicates sensitivity to volatile market conditions. Regarding efciency indicators, rating agency analysts also consider the cost-to-income ratio, as do the bank analysts and consultants. In addition, they favour the return on risk-weighted assets and return on tangible equity metrics. Rating agencies, more than bank analysts and consultants, try to adjust efciency metrics with the risks incurred (e.g. risk-weighted assets (RWA)) and the banks capacity to absorb shocks (tangible equity in lieu of prudential own funds). As expected, they monitor closely credit default swaps and the spread on senior debt. Although they regard subordinated debt and hybrid capital spreads as less informative, spreads are generally relied upon as a means to ag potential issues.
Regarding credit risk indicators, rating agency analysts consider the coverage ratio as the most important, a preference that reects their consideration of the shock absorption capacity. Next in line are the NPLs ratio and the level of impairment charges as a percentage of loans. They also incorporate the ratio of net non-performing loans over regulatory own funds (once again, the shock absorption metric). To assess the sustainability of bank revenues, rating agencies measure the volatility and quality of revenues and try to identify one-off gains/losses. Alternatively, to smooth the impact on volatility, one agency mentioned that they take three-year averages of the various metrics, with the scores sometimes adjusted as a result of forward-looking assessments. In cases of non-listed entities, agencies look at the spread on senior debt and the credit default swap. In addition, they examine the business model of the entity. Lastly, they also try to incorporate off-balance sheet activities through a qualitative assessment, although they acknowledge that this is a challenging exercise. They also consider the overall quality of risk management processes and practices in making an informed judgement on off-balance sheet exposures. Regarding proprietary models, information from respondents was rather sparse. Banks usually rely on either multi-stage dividend discount models (DDM) or multi-stage discounted cash-ow (DCF) models, applied to different business lines, whereby growth rates are estimated by analysts and the cost of equity is retrieved via a capital asset pricing model (CAPM) based on historical data. Exit values are usually calculated via segment/peer group analysis. The sum of the parts analysis is also used for bank valuation, with the capital allocation aspect constituting an additional benet. Bank analysts also use balance score card models to incorporate qualitative aspects. The rating agencies highlight the more holistic/ inclusive approach offered by using sets of proprietary indicators and models, but also the more forward-looking nature of these
13
measures/models. A further advantage lies in the analysis of the underlying drivers of protability and in the analysis of the vulnerability of banks performance regarding market developments, which is seen as the key difference between standard (i.e. backward-looking) indicators and the aforementioned proprietary indicators and models. Rating agencies also use proprietary indicators/models focused on particular dimensions of risk, such as liquidity or credit risk, for specic asset classes. Overall, consulting these different banks stakeholders has conrmed that there is a variety of indicators used to measure banks performance beyond RoE, although RoE remains one of the most surveyed. Although there is some heterogeneity in the computation and use of these indicators, and despite a lack of data available or the poor quality of the data, the most important indicators include: efciencybased indicators (such as cost-to-income, cost-of-equity, return-on-RWA, return-ontangible-equity), indicators of capital adequacy and asset quality (non-performing loans) as well as revenue structure and sustainability (share and volatility of non-recurring revenue, net interest-bearing income, etc.). Obviously, market-based indicators are always taken into account, but economic-based measures, such as EVA or RAROC, are not often selected as they are considered complex and difcult to assess correctly. As a result, a consistent framework for measuring banks performance may incorporate more insider data than those used for RoE, but may also provide a good equilibrium between providing a reasonable level of insider information (namely as regards business strategy and risks associated with each business line) and relatively simple and comparable indicators.
14
3 WHAT IS WRONG WITH ROE?

3.1 WHAT DID ROE TELL US? EMPIRICAL EVIDENCE FROM LARGE BANKS ROE TREND AND CHANGES SCOPE Over the crisis, RoE may have provided misleading information in differentiating good performers from bad ones. In that context, an analysis was performed on a sample of 12 large European and US banks to point out the dynamic of RoE (evolution and drivers) over the different phases of the crisis. These banks were divided into two sub-groups: universal banks and banks driven by investment activity (see Table 2). This classication was carried out according to the composition of banks net revenue. 4 Due to the small sample size and the expert-based interpretation criterion used for differentiating banks within the sample, the analysis in this section can only be indicative. The analysis rst looked at the evolution of banks RoE since the end of 2002, using semiannual data (or, where available, quarterly data) from Bloomberg, which were subsequently annualised. HOW ROE CHANGED BEFORE AND DURING THE CRISIS A weak discrimination in normal times, a great dispersion during the crisis As Chart 1 shows, RoE increased steadily from the end of 2002 until the rst half of 2007. During this period, the mean was not distorted by outliers (i.e. banks with extraordinarily high or low results), which indicates a high homogeneity of RoE levels. Indeed, the
Table 2 The sample composition
Banks driven to a large extent by investment activity Morgan Stanley Goldman Sachs Deutsche Bank UBS Credit Suisse Universal banks (i.e. banks driven to a large extent by traditional deposit-lending activity)

dispersion (as measured by the interquartile range) remained low and more or less constant. This trend changed in the second half of 2007, when RoE registered a sharp downturn. The fall in protability was coupled with a greater dispersion among banks, owing to a few banks facing severe losses. The increase in dispersion of RoE was mainly driven by the worst performing banks. A slight upswing of RoE occurred in the rst half of 2009, indicating a possible beginning of recovery. Investment banks profitability proved to be much more volatile than universal banks during the crisis As Chart 2 indicates, universal banks and investment banks have shown a different evolution of RoE, with investment banks experiencing higher volatility than the universal banks. It is worth noting, however, that the evolution of sub-groups means resembled the course of sub-group medians, which shows that the RoE of banks was distributed symmetrically around their sub-groups means. This changed
Banks were identied as driven by investment activity when the share of commission and trading income was higher than the share of net interest and other income in most periods (annual data from Bankscope). There are yet some issues to this approach. High commission income may stem from custodian activity. Trading income is volatile it can be high in some periods and low in others. Moreover, if trading income is negative and commission income positive, they cancel each other out (however, the division based on squared shares gives pretty much similar results). It is not clear what constitutes other income, especially in investment banks. It would probably be better to see the structure of income by business lines, but these gures are hard to compare, because each bank has its own names and denitions of business lines.
Royal Bank of Scotland (RBS) Bank of America Barclays BNP Paribas HSBC Santander UniCredit
15
Chart 1 The evolution of RoE of 12 international banks

(percentages) interquartile range all sample banks median all sample banks mean all sample banks 30 25 20 15 10 5 0 -5 -10 -15 -20 2003 2004 2005 2006 2007 2008 2009 Source: Bloomberg. 30 25 20 15 10 5 0 -5 -10 -15 -20 2010
Chart 2 The evolution of RoE in sub-groups

(percentages) median investment banks mean investment banks median universal banks mean universal banks 40 30 20 10 0 -10 -20 -30 2003 2004 2005 2006 2007 2008 2009 Source: Bloomberg. 40 30 20 10 0 -10 -20 -30 2010
a little during the crisis in the sub-group of universal banks, where the mean was pushed down by an outlier with strongly negative nancial results. In general, the performance of investment banks was better during the boom period beginning in mid-2005, whereas the universal banks withstood the crisis in a sounder manner. Indeed, investment banks protability proved to be much more volatile than universal banks during the crisis. SOME LESSONS FROM A COMPARED EVOLUTION OF ROE AND TRADITIONAL MARKET INDICATORS: A POOR WARNING SIGNAL AND A WEAK POWER OF DISCRIMINATION BETWEEN BANKS The price-earnings ratio (P/E) and the price-tobook ratio (P/B) are broadly used to assess market performance of banks. Although these indicators are supposed to be narrowly correlated with RoE, one driver is expected earnings and these ratios are expected to be leading indicators of economic performance. The P/E calculated with expected earnings5 did not seem able to predict, with any signicant advance, risks that were mounting in the system (see Chart 3). Moreover it did not differentiate clearly between banks business models, so that market valuations seemed keener to herd estimations. Instead this indicator gave a broad signal about
the level of condence of the market in relation to the growth of future earnings which started to deteriorate before the turmoil. Moreover, investors seem to suffer from short-termism in their assessment of banks, since a time series analysis of the ratio linking
5 Earnings refer to two-year forward expectations taken from the IBES database for a sub-sample of European banks.
Chart 3 Evolution of price/expected earnings

(left-hand scale: ratio; right-hand scale: index (base 100 December 2003)) interquartile range P/E expectation (2 years expected earnings average) median P/E expectation (2 years expected earnings average) expected earnings (2 years average) (right-hand scale) stock price (right-hand scale) 16 14 12 10 8 6 4 2 0 2004 2005 2006 2007 2008 2009 Source: Thomson Reuters Datastream. 180 160 140 120 100 80 60 40 20 0 2010
16
Chart 4 Ratio of expected price to book value versus expected RoE (two-year average)
(x-axis: ratio; y-axis: percentages) x-axis: expected P/B y-axis: expected RoE pre subprime post between subprime 35 30 25 20 15 10 5 0 0.0 35 30 25 20 15 10 5 0 3.5
held quite well before the crisis and is evidence that markets did not differentiate valuations according to business models or source/stability of revenues and were interested primarily in bottom line results. The nancial crisis did not change signicantly the conclusions of this analysis, but, since the crisis, markets appear to have relied more heavily on the concept of tangible equity than on the broader concept of book value, as outlined in earlier parts of this report. P/E, calculated on actual earnings, decreased steadily in the period from 2003 to 2007 (see Chart 5) in spite of increasing actual protability, reecting higher risk premia or lower expectations for earnings growth, factored in actual stock prices. However, its dispersion around the mean decreased in the run-up to the crisis, which shows that the market has a limited capacity to distinguish clearly between banks. Furthermore, in the period before the turmoil, there was no evidence of decoupling. In addition, the P/E becomes meaningless in times of stress, when nancial results tend towards zero, since this ratio can increase sharply without economic sense (if results drop faster than stock prices), as shown in Chart 5. Thus, almost all
Chart 6 Evolution of the price-to-book ratio (P/B)
(left-hand scale: ratio; right-hand scale: percentages) interquartile range P/B mean P/B median P/B RoE (right-hand scale)
0.5
1.0
1.5
2.0
2.5
3.0
Source: IBES - Thomson Reuters Datastream. Note: Quarterly data for a sample of nine European banks over the period December 2003 to June 2010.
current stock prices of banks to their one-year expected book value correlates positively with the two-year average of the respective banks expected RoE (see Chart 4). This relationship
Chart 5 Evolution of the price-earnings ratio (P/E)
(left-hand scale: ratio; right-hand scale: percentages) interquartile range P/E mean P/E median P/E RoE (right-hand scale) 30 25 20 15 10 5 0 -5 -10 2003 2004 2005 2006 2007 2008 2009 Source: Bloomberg. 30 25 20 15 10 5 0 -5 -10 2010
2.5 2.0 1.5 1.0 0.5 0.0 2003 2004 2005 2006 2007 2008 2009 Source: Bloomberg.
30 25 20 15 10 5 0 -5 -10 2010
17
analysts switched from the P/E to the P/B ratio during the crisis to assess a banks strength to bear losses and impairment on intangible assets as goodwill, given the volatility of results. The P/B ratio has a similar shape to the RoE for large banks and has been closely monitored since the start of the crisis. Even if Chart 6 shows a slight lag between the protability and the P/B turnaround in 2007, both indicators are closely linked.
WHAT ARE THE MAIN DRIVERS EXPLAINING ROE CHANGES OVER THE CRISIS? The results of the decomposition analysis, presented in Chart 7 and 8, show that the factor changes increased strongly during the crisis. Furthermore, whereas, before the crisis, different factors changed in different directions, during the crisis, they mostly behaved in a similar manner to one another. In the sub-group of universal banks, the main factor behind the downslide of RoE was the decreasing pre-tax prot margin, which shows the importance of loan-loss provisions (the cost of credit risk that materialised during the crisis). The other important reason for the RoE deterioration (and the main driver in investment banks) was the generally lowered protability and operational efciency of banking activity, as shown by the sinking prot margin and asset turnover. Indeed, for investment write-downs on
3.2 WHAT DRIVES ROE? DECOMPOSING ROE TO IDENTIFY ITS MAIN DRIVERS The decomposition of RoE was based on a formula multiplying four factors:
operating income pre tax profit ROE = operating income * net revenue net revenue assets * * equity assets
Chart 7 The decomposition of RoE changes for banks driven by investment activity
(percentages)
where operating income is stated before deduction of loan-loss provisions, and pre-tax prot is the result before taxes and after loan-loss provisions.
300
asset turnover pre-tax profit margin operating margin financial leverage RoE (right-hand scale) 35 +935 30 25 20 15 10 5 0 -5 -10 -15
In this formula, the rst factor represents the pre-tax prot margin; the second, the operating margin; the third, the asset turnover; and the fourth, nancial leverage. It is worth remembering that, all other things being equal, higher nancial leverage, while pumping up RoE, increases solvency risk. Since RoE and some of the factors showed both negative and positive values, their changes were calculated using the formula (Rt is the ratio value in period t, |.| is the absolute value): Rt Rt 1 | Rt |
250 200 150 100 50 0 -50 -100 -150 -200 -250
-20 -1835 -25 -300 2002 2003 2004 2005 2006 2007 2008 2009 2010 Source: Bloomberg. Note: The RoE level is presented on the right-hand scale in percentage; the changes of factors are presented on the left-hand scale.
d (Rt) =
18
Chart 8 The decomposition of RoE changes for universal banks

(percentages) pre-tax profit margin operating margin asset turnover financial leverage RoE (right-hand scale) 24 22 250 20 18 200 16 14 150 12 10 100 8 6 50 4 2 0 0 -2 -50 -4 -6 -100 -8 -150 -10 2002 2003 2004 2005 2006 2007 2008 2009 2010 300 Source: Bloomberg. Note: The RoE level is presented on the right-hand scale in percentage; the changes of factors are presented on the left-hand scale.
securities, due to the mark-to-market effect being accounted at the revenue level, the fall in asset turnover has been more severe for investmentoriented banks. Meanwhile, leverage continued to increase slightly in 2008, mitigating the RoE collapse. This continuing process of leveraging has been mostly involuntary, since it resulted from the deterioration of the capital base and the (re)consolidation of off-balance sheet commitments. Therefore, the leverage effect has not yet had a signicant impact on protability. Traditionally, RoA is considered a more reliable protability indicator than RoE, in terms of efciency performance, since it is adjusted for the leverage effect (RoA=RoE/ leverage). Nevertheless, this ratio is quite at across time, especially for the universal banks, (see Chart 9), and so it did not provide much information that could have helped to predict a protability reversal before the crisis. In the case of investment-driven banks, RoA has even been steadily rising since 2003, contributing positively to the increase in RoE. All the same, on an individual basis, banks with the highest RoA proved to be the most resilient amid the crisis. It is worth noting that the RoA of investment-oriented banks (unlike RoE) was consistently lower than that of universal banks. STYLISED CASE OF A STRONG PERFORMER DURING THE CRISIS The selected Bank A is a universal bank that seems to have come through the nancial crisis with only limited damage (see Chart 10). Its protability, as measured by RoE, has decreased since early 2008, but is still over 17%, which is one of the best results among large European banks. The positive performance of Bank A during the crisis was driven mainly by operating margin and asset turnover. Financial leverage was only used, to a limited extent, to counter the fall in RoE. Compared with its peers, the clearest deviation is the development of the operating margin.
Chart 9 Evolution of large banks RoA

(percentages) median investment banks mean investment banks median universal banks mean universal banks 1.5 1.5
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
-0.5
-1.0
-1.0 2003 2004 2005 2006 2007 2008 2009 2010
Source: Bloomberg.
19
Chart 10 Contributions to changes in Bank As RoE over the period from 2005 to 2009
(percentages) pre-tax profit margin operating margin asset turnover leverage RoE (right-hand scale) 15 10 5 0 -5 -10 -15 2005 2006 2007 2008 2009 23 22 21 20
Chart 11 RoE evolution and its breakdown into RoA and leverage the case of Bank B
(percentages) Bank B median investment banks mean investment banks 40 20 0 -20 -40 -60 40 20 0 -20 -40 -60 -80 2010 1.5 1.0 0.5 0.0 -0.5 -1.0 -1.5 2003 2004 2005 2006 2007 2008 2009 2010 50 45 40 35 30 25 20 15 10 5 0 2010
19 -80 18 17 16 15 2010 1.5 1.0 0.5 0.0 -0.5 -1.0 -1.5 2003 2004 2005 2006 2007 2008 2009
Source: Bloomberg. Note: The RoE level is presented on the right-hand scale in percentage; the changes of factors are presented on the left-hand scale.
STYLISED CASE OF A WEAK PERFORMER DURING THE CRISIS Investment Bank B was selected to present the case of a bank that performed poorly during the crisis. A simple decomposition of Bank Bs RoE into asset efciency and risk-taking (leverage) highlights some unbalances in the drivers of RoE. Indeed, RoA was quite at between 2002 and 2007, whereas the leverage increased by 80% over the same period (see Chart 11). Moreover, these evolutions were very atypical in comparison with other large investmentoriented banks: while reporting almost the same level of RoE, Bank B has recorded a lower RoA than its peers since 2004, which has been offset by a much higher leverage.
50 45 40 35 30 25 20 15 10 5 0 2003 2004 2005 2006 2007 2008 2009 Source: Bloomberg.
3.3 CAVEATS TO RELYING ON ROE IN THE ASSESSMENT OF BANK PERFORMANCE The above analysis, together with input from market participants, points to the following limitations of RoE as a measure of performance:
1 ROE IS NOT RISK-SENSITIVE The rst criticism to make about RoE is that this indicator lacks attachment to risk, i.e. leverage, funding and liquidity prole. As highlighted in the decomposition of RoE, a risk component represented by leverage can boost RoE in a substantial manner, as shown by the empirical study above. Other risk elements are also missing in the RoE gure, such as the quality of assets, the cost of risk, the risk concentration, and the solvency situation. RoE is denitely not a stand-alone performance measure and, at the
20
very least, needs to be decomposed to establish where most of its changes come from and, eventually, to identify distortions over time. Indeed, as RoE may be articially swelled by a worsening in solvency, it has to be linked to capital ratios. In 2008, for example, many banks mitigated the fall in their protability due to the erosion of the capital base. In fact, the crisis led to a greater divergence of capital ratios. In addition, long-term as well as crisisinduced consolidation in the banking sector had an impact on banks capital and made an historical comparison or analysis difcult, as it is necessary to adjust data to get pro-forma gures. A POINT-IN-TIME MEASUREMENT WITHOUT SIGNALLING POWER AND FORWARD-LOOKING VIEW The recent crisis has shown how RoE failed to discriminate between the best performing banks and the others (in the sense of banks being able to generate sustainable prots) since, a quarter before the crisis, gures pointed to a great homogeneity in terms of banks protability (a high level of RoEs). In some cases, the banks with the highest RoE were those worst hit by the crisis. Thus, RoE did not make it possible to identify the best performing banks in terms of sustainability of their results. RoE is a short-term indicator and must be interpreted as a snapshot of the current shape of institutions. It does not take into account either the institutions longterm strategy or long-term damages caused by the crisis. Its weaknesses become even more obvious in times of stress, when there is a climate of uncertainty in the medium-term shape and protability of institutions. Extraordinary elements can distort earnings in a non-sustainable way In times of crisis, extraordinary elements (e.g. major holdings sales) may become very signicant. Where changes come from one-shot elements that cannot be reproduced in the future, reported RoE does not reect the sustainable performance of the bank. In such cases, RoE from continuing operations proves to be more 2
reliable in terms of making comparisons between institutions and assessing operating performance. Long-term issues are not taken into account During a crisis, most banks undertake many actions, such as restructuring, to enable them to return to a position where they are generating value in the long term. These actions generally imply negative effects in terms of costs and entail further pressures on RoE. It is somewhat difcult to gauge the extent to which a decline of RoE is the result of a long-term strategy to improve revenue and capital generation. For instance, some banks have shown some merits in long-term restructuring, de-risking and improvement in capital ratios, but all these also have a negative effect on immediate performance. In addition, consolidation makes it more difcult to analyse performance. From the accounting perspective, the effect on equity is generally negative in the short term, since the price paid is often higher than the positive impact expected on future results. But, in the medium term, the effect could be positive from the point of view of nancial strength (i.e. wider capital base). In the long run, however, it is hard to predict the global effect, as consolidation may modify the business model of a group. Another example of long-term issues concerns cost efciency. From a cost-to-income perspective, cutting staff expenses has positive effects on efciency ratios, but this strategy can be unprotable in the long term, given the importance of human capital in the banking sector. A brain drain can create irreversible damages and prevent a return to success. Banks which seek to retain or attract talent and key employees have a high efciency ratio weighing on their immediate protability. In the end, RoE does not take the long-term damages caused by the crisis into account. For example, even if an investment bank has been severely hit by the crisis, RoE projection for the next year is good, since risks will have switched
21
to the lending activity, where the cost of risks is unlikely to improve materially for several quarters. According to analysts, investmentoriented banks are going to perform better, as the bulk of write-downs appear to have already been booked for this banking segment. But a sick bank is in a challenging situation, due to continuing outows in wealth management activity and the fact that a lot of clients and staff will have moved away towards sounder competitors. Having lost large market shares in the area of asset management and prime brokerage services, this bank will have to rebuild itself and re-attract customers. Bad RoE impairs the brand image, and it could take many years for poor performers to regain even their previous market shares. 3 ROOM FOR MANIPULATION AND WRONG INCENTIVES
instrument can be accounted differently in different institutions. In addition, RoE could be manipulated in the sense that it is possible to account assets in different portfolios, which can impact RoE components in such different ways that comparisons may be misleading. RoE focus provides wrong incentives Since RoE is the most well-known performance indicator widely used by market participants and banks themselves in their disclosures (i.e. at the top line of bank reports), targeting RoE has exposed banks to higher unexpected risk levels and opened the door to a more shortterm-oriented approach to balance sheet management. The importance of seasonal factors The RoE measure is inuenced by quite strong seasonal factors, so that the interpretation of intra-year RoE provides misleading information. Indeed, annualised RoE (not seasonally adjusted) is commonly used in quarterly and semi-annual reports, although using annualised RoE for the rst half of the year is traditionally better (due to dividend payments in the second half of the year and trading losses, which tend to be swelled at the end of year). Thus, the semi-annual or quarterly performance indicators reported by banks are awed, and protability indicators must be seasonally adjusted in order to be analysed properly.
Lack of transparency or inconsistency in disclosure If data are not reliable (e.g. due to a lack of disclosure or a change in the accounting rules), RoE can no longer be used to make meaningful comparisons between banks. Indeed, due to a lack of transparency, unrecognised losses can swell RoE in a fallacious way. In this context, best performers can also be those banks with major unrecognised losses. Moreover, given the differences in accounting standards between countries, any one particular nancial
Box 2
CASE STUDY ON WARNING SIGNALS FROM BALANCE SHEET AND MARKET DATA PRIOR TO THE CRISIS This case study is aimed at assessing if, before the beginning of the current crisis, there was evidence of mounting imbalances or excesses that, unlike RoE, could have signalled where problems were going to show up in the near future. The analysis focuses on six European banking groups and looks at both market and balance sheet data.1
1 Data, provided by Bloomberg, refer to semi-annual balance sheets for a time period spanning from the rst semester of 2003 to the rst half of 2007.
22

Chart A Stock total return index
(index (base 100 September 2003)) UniCredit BNP Santander 250 200 150 100 50 0 2004 Source : Bloomberg. 2005 2006 2007 Deutsche Bank RBS UBS 250 200 150 100 50 0 60 50 40 30 20 10 0 2003
Chart B Leverage
(percentages) UniCredit BNP Santander Deutsche Bank RBS UBS 60 50 40 30 20 10 0 2007
2004
2005
2006
Source : Bloomberg.
1 POOR SIGNAL FROM MARKET DATA Stock return is the most common indicator that is supposed to reect banks nancial health and strength (as assessed by investors). As shown in Chart A, most of the best performing banks in terms of return and level of growth in RoE gures before the crisis are among the worst performers during the crisis. Moreover, stock performances seemed led by the forecast of short-term earnings, as detailed in Section 3.1. 2 THE BALANCE SHEET ANALYSIS
The balance sheet analysis (for details see Section 5.1) is concentrated on the upper part of the income statement. It considers the different sources of revenues, including interest rate-bearing activities, commissions, trading and investments, as well as other revenues. Revenue analysis may lead to a deeper understanding of the protability of different business areas. It takes a longer-term approach and investigates the level and evolution of margins, stock and turnover indicators of the key revenue drivers and looks at them in relation to leverage. Applying the above framework to balance sheet data can be a useful way of measuring the contribution of the different business areas, in particular their evolution over time, and of assessing structural differences between banks business models. However, the unavailability of detailed data does make it difcult to carry out a broad and deep analysis. A thorough assessment of a banks nancial strength therefore requires a careful reading of the whole balance sheet, combined with the use of internal data. Leverage as a quite good warning indicator
The main result for this indicator is that non-sound banks experienced either a high level of leverage in absolute terms (i.e. above 30) or a signicant increase of this indicator over previous years (see Chart B). Furthermore these banks experienced a reduction in the share of loans on total assets, indicating a shift towards other non-interest generating assets. Further analysis of
23
Chart C Interest on operating revenues

(percentages) UniCredit BNP Santander 70 60 50 40 30 20 10 0 2003 2004 2005 2006 Deutsche Bank RBS UBS 70 60 50 40 30 20 10 0 2007
Chart D Cost of risk over interest margin

(percentages) UniCredit BNP Santander 40 30 20 10 0 -10 -20 -30 -40 2003 2004 2005 2006 2007 Deutsche Bank RBS UBS 40 30 20 10 0 -10 -20 -30 -40
Source : Bloomberg.
Source: Bloomberg.
asset composition is not possible due to the unavailability of detailed data. Due to this lack of data, the analysis misses another signicant area of analysis related to off-balance sheet leverage, i.e. the nominal value of derivatives embedded in structured products or in the trading books that adds further to balance sheet leverage. The revenue component analysis highlighting some unbalances
This analysis focuses on the contribution of margins to the protability of the different business areas. For instance, interest rate revenue drivers can be analysed considering interest rate income on total interest bearing assets as the gross margin and loan-loss provision as the cost of credit and total assets at the level of the stock. Portfolio composition and volume of trading can add further insights for the trading area; for commissions generating business, Assets Under Management (AUM) could also be taken into account as the main revenue driver. In this case study, data availability only makes an analysis of the interest rate area possible. The most important issues that differentiate sound from not-sound banks are: The growing importance of trading revenues and commissions in relation to the interest rate margin: Chart C shows that not-sound banks reduced their dependence on the interest rate margin in relation to total revenues by more than 20%. During the crisis, losses came from that area of business through trading losses or provisions related to nancial assets that had generated growing revenues in the past. As regards the interest rate margin, Chart D shows that not-sound banks keep loan-loss provisions in relation to gross interest rate margins below 10%, whereas sound banks keep that ratio at a constant level of more than 15%. In addition, not-sound banks evidenced a low interest margin on assets as they were investing in low spread interest-bearing assets.
24
3 WHAT IS WRONG WITH ROE? Cost efficiency as a source of resilience
An analysis of the cost efciency of banks, in running their business, shows that, for the sample, not-sound banks had a cost/income ratio of near to 65%, compared with 55% for sound banks. When the crisis hit, banks with a weak efciency got into difculties more quickly and consequently reduced employees to keep costs down in line with the new nancial environment. 3 CONCLUDING REMARKS A rst broad ex-post analysis highlights some important elements that could explain the different economic performance of the two sub-groups during the crisis, but an important remark needs to be made regarding the results of the analysis of banking groups, which had similar economic and risk indicators, but behaved very differently during the crisis. The case study showed the need for a more structural analysis of banks performance that deals with the specicity of each business area, taking into account the core protability elements, as in the sum of the part analysis used currently by nancial analysts. Moreover, it stresses the need for a more detailed analysis when the rst analysis does not give a clear picture of the evolution of a bank business model, but, in doing so, there is the risk of insufcient data or comparability problems.
25
4 REFINEMENTS AND ISSUES TO CONSIDER

4.1 REFINEMENTS IN TERMS OF SCOPE The global nancial crisis has shown traditional performance measures to have certain deciencies and reinforced the need to rethink some of the metrics used by the nancial community. The shortcomings identied could be addressed by focusing the scope of analysis of banks performance on those aspects that have previously been disregarded or neglected. According to the results of the survey and the workshop organised by the BSC, there is much more to the measurement of performance than plain protability and more than one metric is required to explain performance changes. The respondents pointed to efciency, asset quality and capital adequacy indicators as integral parts of performance analysis. Hence, a more holistic and forward-looking approach to performance measurement seems to be appropriate. RISK-ADJUSTED MEASURES The crisis has reintroduced risk into the equation of banks performance measures. Hence, one possible renement to performance analysis would be to rely on risk-adjusted returns instead of plain returns. When comparing the protability of universal banks against that of banks driven by investment activities, it is clear that while the latter were more successful in benign times, they suffered larger losses when the crisis hit. The interesting question would be whether the higher risk was sufciently rewarded. This question can be answered by looking at risk-adjusted returns. RAROC allows capital to be assigned to business units and activities of banks in accordance with their anticipated economic value added (EVA). As a consequence, it is essentially a prospective measure that assesses anticipated results by looking at future capital needs. Similarly to EVA, it links a banks prot with its cost of capital. However, it goes further than this by balancing value added against the capital needed. Theoretically, it is the most relevant measure of performance since it balances economic return against risks. But it has an important disadvantage: its calculation requires internal data and it relies heavily on assumptions. ASSET QUALITY Asset quality analysis, as a complementary measure of banks overall performance, has increased in complexity and importance. Because of the asset legacy problem, the quality of banks assets will be a recurrent topic when it comes to discussing performance. Traditional performance analysis does not fully reect the more complex world of accounting and market participants may have difculties understanding the information provided by banks. In order to assess asset quality, analysts and bankers use vintage-based delinquencies, migration matrices (i.e. rollover rate between delinquency buckets) and scenario analysis to project future loss levels for retail portfolios. Another qualitative indicator is the analysis of portfolio quality to identify preimpairment assets, predictability and stability of cash ows. Stress tests help to assess the quality of the assets and support management in dening deleveraging action plans. In this context, the market has widely adopted reconsolidated all off-balance sheet assets. QUALITY OF BANKS CAPITAL IN RELATION TO ITS ASSETS Another important issue highlighted by the crisis is the quality of banks capital in relation to its assets, as well as the misleading information stemming from capital adequacy ratios. Regulatory capital requirements tend to include large hybrid components that would not necessarily qualify as capital in the economic sense since they have limited risk-absorbing capacity. What is more, it has transpired that risk-weighted assets do not reect the degree of risk of some toxic assets and gave little or no weight to credit-risk bearing assets other than loans (e.g. collateralised debt obligations (CDOs)). As a result, some banks had strong capital ratios, but extremely high ratios of total assets-to-equity (which also boosted their RoE). However, in the heat of the crisis, these banks proved particularly vulnerable, which has shaken investors condence in prudential capital measures. Participants in the workshop recommended
26
the usage of tangible equity instead of net equity and core Tier 1 capital instead of total regulatory capital. They also started to observe leverage, which can be regarded as going back to basics. A similar approach was taken by some supervisors, who set target capital adequacy ratios for banks either at a much higher level than 8% or at the basis of Tier 1 capital. The respondents also suggested that all off-balance sheet instruments and vehicles should be reconsolidated (regardless of whether or not they are accounted for in risk-weighted asset calculations). They pointed out that it could be very difcult or even impossible to reach a quantitative estimation of off-balance sheet exposures, such that a qualitative judgment would be necessary. COMPOSITION AND SUSTAINABILITY OF EARNINGS The earnings analysis is linked to the asset quality analysis and is an issue which has drawn a good deal of attention since the crisis began. Persistently high levels of protability can be alarming signals. Very high protability can imply excessive risk-taking and a buildup of vulnerabilities, which would eventually jeopardise sustainable protability. The focus should be on predictability and low volatility of earnings in order to enable performance sustainability. The diversication and specialisation of banks activities is a relevant issue in this respect. There is much literature on this topic, but the empirical evidence is mixed. In theory, diversication (including product diversication) should lead to reduced volatility of earnings. However, earnings arising from non-interest activities of banks are much more volatile than net interest income a large part of these gains is considered non-recurring (trading income, non-retail fee income). It is unclear as to whether the over-the-cycle prots of these non-recurring activities are sufcient to make up for increased volatility. Nevertheless, in times of nancial stress, the recurring components of revenues are carefully valued by analysts. Market participants appear to carefully observe the structure of banks revenues and capital by business line and try to assess the sustainability of their business models. Metrics
adjusted to the features of individual business lines are required to accomplish this goal. LIQUIDITY AND FUNDING CAPACITY The crisis has highlighted the crucial importance of banks liquidity and funding capacity. Market participants are, therefore, likely to take a closer (albeit incomplete) look at the liquidity and funding structure of banks, including measures such as the loans-to-deposit ratio, share of short-term (or wholesale) funding and maturities table, in order to assess the relative funding strength of a bank and its dependency on short-term funding. A banks funding structure is largely determined by its dependency on wholesale market funding, on short-term funding, and its maturity prole. Furthermore, higher funding costs affect a banks protability. A performance analysis should include the examination of a banks funding structure and policy in order to establish its liquidity prole and business sustainability. Market participants also stressed the need for contingency planning in the event of liquidity strains, because even a fundamentally sound bank may face liquidity or funding difculties. With regard to these points, the general lack of disclosure vis--vis funding in European banks limits the quality of the performance analysis of the funding structure. An additional assessment, therefore, can come from the internal funds transfer pricing, i.e. the appropriate pricing of funding between various units to ensure cost allocation and to contain risk-taking. Finally, when judging a banks performance and bank funding, the maturity mismatch in the banking sector would also have to be taken into consideration. Banks carry trade of cheap, short-term borrowing in order to invest in higher yield long-term lending is their core business. This, however, would not be possible without the existence of the current safety net provided by the lender of last resort and the deposit guarantees schemes. When analysing a banks performance or benchmarking the banking sector against other industries, this specic publicly-supported prot enhancement
27
characteristic of the banking sector needs to be taken into account. INTANGIBLES Though banks xed assets are important, the banking sector is a knowledge intensive industry, and nancial knowledge, intellectual resources and other intangible assets are also relevant performance drivers. Banks complexity has increased considerably, and intangible assets have become an important driver of performance. In fact, Ang and Clarks (1997) argument that banks book value should be approximate to their market values is becoming increasingly invalid. Book and market values will differ since the former cannot reect a companys internal values (Abuzayed and Molyneux (2009)).6 Sagar and Rajesh (2008) therefore argue that merely analysing nancial indicators does not constitute an effective strategy for credit institutions, since their performance interlinks nancial indicators with non-nancial indicators. As a result, nancial performance is determined by too many intangible business processes and performance indicators. Zhang and Longyi (2009) regard the traditional measures of performance as inadequate and point out the following deciencies in traditional performance measures: (i) too heavy a focus on nancial indicators, while ignoring the intangible indicators; (ii) too much focus on the internal analysis of operating conditions, while excluding external factors; and (iii) too great an emphasis on traditional assets, while neglecting intangible assets.7 Accounting standards do not properly reect all the intangibles related to nancial activity. Furthermore, the current crisis has highlighted that the evaluation and impairment process of intangibles has changed dramatically. Thus, it is essential to correctly understand the link between business drivers and intangible assessments. One approach to cater for the above identied gaps in information is the use of a Balanced Score Card, a concept developed by Kaplan
and Norton (1992). Sagar and Rajesh (2008) additionally suggest that certain commonly used nancial ratios provide indications of the contribution of intangible assets. For example, the growth rate of deposits is a good indicator of customer condence in the credit institution. Another indicator is the growth rate in loans, which illustrates the customer preference for the services of a specic bank. Yet, Ittner and Larcker (2000) argue that although non-nancial indicators are becoming more important in decision-making and performance analyses, rms should not simply adopt measures used by other rms. They argue that the choice of measures of performance must be linked to factors such as the competitive environment, corporate strategy, organisational objectives and value drivers. In addition, rms must understand that the choice of performance measures is a dynamic process, whereby measures which may be appropriate today may not be in the future and therefore may require constant review as strategies and competitive environments constantly evolve.
4.2 REFINEMENTS IN TERMS OF PROPERTIES OF PERFORMANCE MEASURES The global nancial crisis revealed that performance metrics used by the nancial community lack some desirable features. This shortcoming may have led analysts deploying these metrics to draw false conclusions. Hence, it is worthwhile considering renements to the comparability, transparency and stability of performance metrics as well as the time frame of such analyses.
In this sense, Martin and Salas (2007) argue that the value of a bank is made up of material assets, intangible assets and rents from market power and that their contribution amounts to 55%, 20% and 25%, respectively. Martin, Salas and Saurina (2007) point out that the main limitation of traditional performance measures is the use of raw accounting data. Conventional accounting principles tend to underestimate the stock of intangible assets recorded in the balance sheet. As the expenditure in these intangible assets has increased over time, often at a faster pace than physical assets, accountants and business analysts should pay increasing attention to ways of measuring the total assets by assessing the performance of the bank beyond the gures reported in their balance sheets.
28
A considerable number of pivotal performance metrics, including return on equity and capital adequacy ratios, are incomparable across companies and countries due to specic peculiarities of accounting standards and capital regulations. Even under the common framework of International Financial Reporting Standards (IFRS), companies are allowed to choose among different accounting rules, which, while boosting compliance with the true and fair view criterion, hinder the comparability of nancial reports. A similar scenario applies to the implementation of the Capital Requirement Directive (CRD Basel II) there are several country-specic solutions for capital eligibility, deductions, etc. that inhibit the comparability of regulatory capital and risk-weighted assets. Moreover, it should be borne in mind that even with the best measurement framework, there is considerable bias linked to some of the internal rules or conventions that are used within the banks. Finally, there are often multiple denitions of nancial ratios. Owing to these divergences, data on capital adequacy as well as on protability and asset quality were considered to be awed, which added to the general distrust in nancial reports during the crisis. Workshop participants claimed that as long as denitions and accounting frameworks were incoherent, protability measures would lack accuracy. The consistency of performance measurement over a period of time is also a crucial issue, since mergers and acquisitions may blur the analysis of banking group performance in the absence of a constant perimeter. Finally, market participants raised the issue of stability and predictability of performance metrics across time and the business cycle and they criticised the so-called short-terminism of traditional performance measures (point-in-time measurement). Many studies have shown that during upturns banks become overcondent, which implies lower lending standards or, in other words, an increase in risk-taking. On the contrary, during recessions banks become conservative and tighten up their lending standards. This results in an increase in non-performing loans during downturns in
the cycle, which ultimately leads to an increase in provisions and losses incurred by banks. Correspondingly traditional performance measures exhibit a certain degree of cyclicality (correlation) or procyclicality (causality), which was discussed in the previous section. For instance, the earnings and balance sheet gures of banks can be inated by high consumer price or asset price increases without any change in real performance. Therefore, through-the-cycle measures should also be taken into account. For example, the pre-tax prot margin over the business cycle shows the ability for sustainable internal capital generation. In this context, protability measures should be assessed over sufciently long periods. It is worth reiterating that capital adequacy regulations and accounting standards were proved to boost the cyclical behaviour of banks. The International Accounting Standards Board (IASB) addressed this issue recently by proposing to introduce expected losses over the lifespan of a loan into its valuation.
29
5 ADDITIONAL FACTORS AND ALTERNATIVE MEASURES OF PERFORMANCE

The crisis that affected global nancial stability and the economy in 2007-09 has reinforced the need to rethink some of the approaches adopted by the nancial community in assessing banks performance. To this end, it is important to obtain a comprehensive view of the key factors that may inuence banks performance, including the adequacy of business models in relation to risk appetite, and the question of how this adequacy is handled inside and outside banks through governance processes. Against this backdrop, appropriate benchmarks, sensitivity analyses as well as stress tests ought to be considered in order to assess the real capability of banks to face stressed market conditions and absorb consecutive shocks on the basis of their business strategy and degree of risk tolerance. This section aims to address these factors, to go beyond traditional assessments of bank performance and to focus on key factors that may inuence banks performance, discussing, to the extent possible, alternative approaches of both a quantitative and qualitative nature. One approach is to evaluate whether the business model of a bank allows it to withstand shocks and maintain long-term protability. To this end, it is helpful to identify and monitor key business drivers (e.g. loan and deposit margins) within an analysis framework in line with the sum of the parts used by nancial analysts. In addition, the current environment, whereby business models are going to change as a result of regulatory measures, requires analysts to avoid relying too heavily on past trends, but rather to be more forward-looking. Box 3 provides some empirical evidence, based on stylised case studies, to illustrate the importance of business line revenue analysis in the assessment of banks performance. In particular, it highlights the need, when looking at RoE decomposition, to search deeper in the revenue structure analysis, by placing more emphasis on the upper part of the income statement and by considering the different sources of revenues among interest-rate bearing activities, commissions, trading and investments and other revenues. In particular, assessing the sustainability of bank revenues may require either the share of core banking income (i.e. net interest, commission and fee income) or the share of non-recurring revenue (i.e. income from fees not related to loans) to be identied. This analysis ought to be supplemented by considering the volatility of each revenue component, i.e. by looking at their respective key drivers. This could constitute the rst step towards a sensitivity analysis of banks protability to the development of different business drivers.
5.1 WHY DO BUSINESS MODELS MATTER? The crisis has highlighted the importance of having business models that lead to long-term sustainable activity and protability. Therefore, identifying and assessing the sustainability of key business drivers is essential in assessing the sustainability and resilience of banks protability structure.
Box 3
THE IMPORTANCE OF THE ANALYSIS OF BUSINESS LINE REVENUES IN ASSESSING PERFORMANCE: SOME EMPIRICAL EVIDENCE A zoom in on the composition of revenues and earnings by business line is necessary to analyse accurately the performance of a banking group, namely its capacity to generate sustainable protability. Indeed, RoE provides a global picture of the protability of a bank but does not reect the relative contribution from each activity. However, a decomposition of revenue over a certain time period can reveal some imbalances between recurring and non-recurring revenues, which are more volatile by nature.
30
The case of a large international bank with strong investment banking revenues is illustrative in this respect. From early 2006 to 2007, the net trading income of this bank increased sharply and was multiplied by nearly ve over this short period (see Chart). In the meantime, commissions and fees net income went up steadily, while net interest income remained at. Thus, the rise in protability was mainly driven by the strong and unusual growth of one segment in particular. In the same way, the fall in RoE in 2007 was mostly explained by the bad results in trading by this bank. Thus, since the start of the crisis, RoE and trading income are tightly correlated given the rapid expansion of trading in this banks business model a few years before the crisis.
RoE and revenue breakdown of the case bank

(left-hand scale: million local currency units; right-hand scale: percentages) net interest income net trading income net commissions and fees income RoE (right hand-scale) 15,000 100 10,000 5,000 0 -5,000 -10,000 -15,000 2005 2006 2007 2008 2009 Source: Bloomberg. 50 0 -50 -100 -150 -200 -250 2010
The relative contribution of each segment to total revenues provides an indication of the sustainability of performance regardless of the business model of the bank. However, obviously some banks did not experience such imbalances in their business models before the crisis, as shown in Chart 2. However, it is noticeable that after falling markedly in 2008, the corporate and investment banking segment recovered and even exceeded its pre-crisis level (in terms of the relative part of total revenue) in 2009 in the case of four large banks in our sample. That said, the recovery of protability in 2009 was quite fragile in that it was driven by a highly volatile segment of revenue. Splitting the different sources of revenue is indeed key in the analysis, since banking is no more a monoline activity, but has become a franchise over the past decade, with many businesses driven by different economic risk factors. As a result, a single line expression can sufciently represent the performance of a single line business model, as was the case in past banking activities based on deposits and loans, however nowadays, it is no longer sufcient to identify and distinguish the contribution to risk and performance from the different business lines that are managed within a complex banking group. In this respect, it is crucial to analyse every business line separately and assess their different contributions to the global performance of the bank through a matrix-type analysis framework. The aim of performance measurement through business model analysis is to develop a set of indicators which highlights how a business area generates revenues across time, given its different business dimensions, such as margins, stock, turnover, leverage and cost efciency (see Chart 12). First of all, it may be useful to assess the percentage gross margin for each revenue driver and the respective cost-to-provision ratio; in that way, we can obtain a percentage net margin. Moreover, not only the level of revenue driver (stock) but also the ows must be considered by means of the turnover analysis. Such a comprehensive analysis may provide us with a deeper understanding of each segment of protability in relation to the risks. Performing a time series analysis, even if incomplete, as we have seen in the case study, can provide some insight into some of the imbalances that are forming over time and those results are useful for challenging the risk management policy developed by banks.
31
Chart 12 Balance sheet analysis of the decomposed RoE measure
Non-recurring revenues
Leverage
Recurring revenues
Cost efficiency
Pre - tax profit Total assets Net revenue Operating income RoE = * * * Operating income Equity Total assets Net revenue
Total assets Intermediation income Operating income * * Equity Intermediation income Total assets
Turnover
Stock
Margins Other Examinations Gross Interest margin margin = Loans stock Commissions AUM flows
Off Balance-sheet On Balance-sheet Interest assets Asset management Trading/ investment Other income Loans flow Loans stock AUM flows AUM stock Turnover portfolio Trading portfolio AUM stock Total assets Trading portfolio Total assets Loan stock Total assets
Contribution Interest margin - LLP Intermediation income + Commissions Intermediation income + Trading-Finance Provision Intermediation income + Other Intermediation income
Net trading income Turnover portfolio Analysis of margins from other income drivers
Business driver flow
Business driver stock
Total assets + AUM Total assets * Total assets + AUM Equity Source: ECB.
An improvement in the analysis framework can be obtained through the use of the capital allocated to the different business areas to obtain a measurement of the return on capital, which could be the base for protability risk-adjusted measures. However, capital allocation inside a banking group is information that is not public and not readily available. Moreover, such information could lead to a volatile assessment of different business areas in relation to its denition. In this respect, results stemming from the use of capital allocated must be carefully considered, as they can lead to decisions on business development that could destroy value on a long-term basis.
The aim is to achieve a set of indicators which highlights how a business area generates revenue through the analysis of the different business dimensions, such as margins, stock, turnover, leverage and cost efciency. The analysis of those indicators over time provides some insight into the key drivers of risks and revenues. Such an approach may also constitute the starting-point for a more thorough analysis of the contribution of each business line to the global performance of a bank. The adoption of a wider and more forwardlooking assessment of performance may be the rst step towards establishing a continuous
32
dialogue with the banks management about the coherence between economic results, business models and supervisory/stability issues. In order to achieve a comprehensive analysis for all business areas, data availability and comparability is a key issue. In fact, those business areas for which balance sheet data are not detailed or comparable include trading, asset management and, in general, investment banking, which prohibits a deeper assessment of how the business is run. Another issue which poses a great challenge is the inclusion in the framework of derivatives or off-balance sheet activities; this problem needs to be dealt with in order to fully understand the risks and revenues in those business areas where they are widely used, otherwise the risks could be underestimated and the risk-adjusted protability consequently overestimated. An additional point in the business model assessment for performance is diversication. From a performance perspective, there are both costs and benets associated with diversication. Indeed, it may give rise to economies of scale and of scope and it may allow organisations to reduce earnings volatility by spreading their operations across regions with different economic environments. On the other hand, the costs of diversication result from increased complexity and agency costs. However, diversication benets must be carefully analysed on a long-term basis, since during periods of nancial distress, different business areas show a higher degree of correlation between performance and economic drivers than in normal times. Finally, when assessing the sustainability of bank revenues, either the share of core banking income (i.e. net interest, commission and fee income) or the share of non-recurring revenue (i.e. income from fees not related to loans) needs to be identied. This analysis ought to be supplemented by considering the volatility of each revenue component by looking at its respective key drivers.
5.2 ECONOMIC CAPITAL MODELS AND STRESS TEST RESULTS ARE KEY ELEMENTS OF A PERFORMANCE MEASUREMENT FRAMEWORK Business model analysis includes earnings mix, funding mix and business mix. It also includes the analysis of market shares in key products and segments, segmental analysis, and a qualitative assessment of implications for the banks risk prole. In this respect, constructive dialogue with banks is required. Business model analyses should be accompanied by stress-testing and what-if analyses to check whether the model would be viable if external conditions change signicantly. Economic capital is a key component in understanding the developmental relationship between real risk-taking, capital usage and risk-adjusted performance. In fact, economic capital calculations encapsulate an ambition to describe and measure, on a consistent basis, the range of phenomena that drive a banks risk/ return decisions. A consistent and comprehensive economic capital model accomplishes two goals: 1. It provides a common metric for risk assessment that banks management can use to compare the risk-adjusted profitability and relative value of different businesses with widely varying degrees and sources of risk. 2. It allows banks management and supervisors to evaluate overall capital adequacy in relation to the risk profile and level of risk appetite of the institution. The theory underlying the economic capital framework requires each portfolios capital allocation to reect its contribution to the volatility of the banks market value, as opposed to its own stand-alone volatility. The calculation, therefore, includes both the stand-alone volatility of an exposure and its correlation with value changes for the rest of the portfolio. Against this backdrop, economic capital may be of primary importance to allow for a different treatment for each type of business line and to
33
analyse the contribution from each business line to the shareholders value added of the global activity of a bank. This granularity in the performance measurement is thus essential for adequate transparency, which, in turn, is key to understanding the drivers of banks performance. A granular approach may also provide a framework for a better pricing of risk at the business line level, which is essential in the maximisation process underlying the risk-return arbitrage. Internal economic capital models may also be accompanied by stress-testing and what-if analyses in order to see whether the model would be viable if external conditions changed signicantly. Stress tests or scenario analyses help to illustrate the shock absorption capacity of economic capital. As a consequence, stress tests results may also help banks to x limits in order to align their business plans with the risk appetite of the bank. The increased volatility and uncertainty created during the crisis has reinforced the need to use stress-testing as a forward-looking performance measure. Stress-testing helps identify the key drivers of performance and their sensitivity under various extreme scenarios, whereby assumptions on (historical) correlations should be avoided. Stress-testing can be used to derive impact assessment by using cycle parameters (e.g. downturn probabilities of default, migration matrices, etc.). As a result, stress tests may be essential in the performance analysis framework, as they are deemed to be embedded in the capital and nancial planning of banks (e.g. liquidity and capital planning and allocation; credit and market risk-taking). However, one issue raised by stress tests is the reduction in incentives for investors to conduct their own risk analysis (moral hazard). Yet, stress-testing is still at a relatively preliminary stage of development in banks, namely as part of the nancial planning process, but it could be enhanced in that direction at great benet.
5.3 GOVERNANCE AND PERFORMANCE: INCENTIVE SCHEMES BASED ON VALUE CREATION MAY HAVE AN EFFECT ON PERFORMANCE The crisis has highlighted the importance of introducing qualitative measures of bank governance to assess performance. Different aspects of corporate governance inuence banks performance. This is compounded as priorities vary considerably depending in the individual banks situation. Key governance issues include the balance of powers at senior management level, i.e. the separation of the role of the Chief Risk Ofcer (CRO) and the Chief Financial Ofcer (CFO), which allows for an integrated view of risk as well as an adequate level of knowledge of the board members and their willingness to acknowledge adverse developments and challenges that lie ahead. Though difcult to embed in a quantitative framework, corporate governance should be assessed on a qualitative basis. A particular aspect of bank governance is compensation incentives. In particular, incentives for good performance are created by the banks business models, as many bonuses are directly linked to the creation of risk-adjusted value for a given market or activity. Targets are usually based on a certain level of risk-adjusted protability. Hence, most of the incentive compensation schemes may be a signal of the internal assessment of banks performance over a certain period of time. However, the nancial crisis has highlighted the need to use compensation schemes consistent with risk levels and horizons that are in line with the long-term objectives of the bank. The structure of compensation gave agents, acting on behalf of stockholders, an incentive to adopt risky behaviour in order to maximise short-term prots, whilst ignoring the long-term health of the rm.
34
In global discussions, the tendency is to set compensation incentives in line with risk appetite for a long-term horizon. Indeed, it is preferable to link incentives with longer-term health metrics (e.g. a protability indicator averaged over a ve-year period) instead of with revenues or pure market shares.
5.4 GOVERNANCE AND PERFORMANCE: DISCLOSURE AND MARKET DISCIPLINE In a market economy, the ultimate indicator of performance is market discipline. Firms labelled with actual bad performance lose a part of the market share and can ultimately be excluded from the market, and those with good performance gain in market share and positioning. Policy-makers should, therefore, be concerned about fostering market discipline as an unquestionable market-based indicator of performance. Therefore, the two main drivers of the market economy prots and losses should be allowed to operate unfettered from distortions as far as possible. The prerequisite for effective market discipline is to ensure a meaningful disclosure that allows market participants to conduct their own assessments of banks risk prole and performance. Transparency is pivotal in banks performance, particularly in times of stress. Simple and globally harmonised denitions, as well as high-quality data, would denitely assist in a better enforcement of market discipline. The lack of comparability of ratios, differences in accounting standards and in applying such standards, as well as the quality of data, were proven to be major obstacles to obtaining a clear picture of banks performance. It would also be benecial to enhance transparency of nancial reporting, which does not mean increasing the amount of reported or publicly available information. As a matter of fact, it is quite easy to get lost in the complexity of reporting and disclosures and fail to effectively address their objectives. On the other hand, even though the amount of disclosures may be huge,
some vital information can still be missing, supporting a comprehensive assessment of the performance of banks business models (e.g. economic capital allocation, off-balance sheet activities, more detailed business line information). The lack of transparency has made it impossible for investors to differentiate between severely impacted and relatively unscratched institutions in the midst of the crisis, which might have amplied the liquidity squeeze. Hence, more transparency is being sought through coherence and consistency of high-quality data using simple and globally-harmonized denitions. For instance, the inclusion in the framework of derivatives or off-balance activities may be necessary to fully understand the risks and revenues in those business areas where they are widely used, otherwise the risks could be underestimated and the risk-adjusted protability consequently overestimated. Building an efcient framework for performance analysis would then call for improved disclosure and exchanges, but not only towards investors. Here, there may be some room for a higher level and detail of disclosure towards supervisors namely as regards business model-related information perhaps through the ongoing supervisory review and evaluation processes (SREP) in the context of Internal Capital Adequacy Assessment Processes (ICAAP). Though performance measures are not absent from these frameworks, or from the internal control reports, the key elements of achievement may perhaps lie in a deeper dialogue between supervisors and banks on the adequacy of their performance with the business model and strategy in place. Hence, the quality of the information ensuing from this dialogue may also depend on it being carried out at a sufciently high level. Hence, the involvement of the top-tier management in the internal process of analysing the adequacy of the business model in relation to the risk appetite and risk-taking tendencies of the banking institution may be a key element to ensuring good governance.
35
6 CONCLUSION: WHICH IS THE WAY FORWARD?

In conclusion, it may be worthwhile focusing further work and discussions around three axes. First, RoE may be less of a performance measure than an element of incentive in the relationship between banks and markets. A comprehensive performance analysis framework would then necessarily go beyond that kind of indicator though not excluding it and provide the scope to conduct assessments directly on the basis of banks business data and qualitative information. In particular, the consistency of risk appetite with the business structure and strategy of a bank appears to be one of the most important elements in the assessment of a banks capacity to perform in the future. Against this backdrop, sustainable indicators constructed on the basis of economic capital models and nancial planning frameworks within the banks may be of important use. Second, desirable features for banks performance measures should encompass more aspects of the performance than just protability embedded in a pure market-oriented indicator such as RoE. In particular, it is essential to take account of the quality of assets, the funding capacity and the risk associated with the production of value. In that context, a good performance measurement framework should incorporate more forwardlooking indicators and be less prone to manipulation from the markets. Third, governance and banks risk management processes should be further enhanced: the adoption of a wider and more forward-looking assessment of performance may constitute the rst step to intensifying the dialogue between the banks management and supervisors, and where condentiality issues permit, with market analysts as well. In that context, a comprehensive analysis for all business areas based on data availability and data comparability is key. This may call for enhanced disclosure, both towards the public and the supervisor.
36
ANNEXES 1 APPROACH, QUESTIONNAIRE, WORKSHOP PARTICIPANTS AND RESPONDENTS

One of the objectives of the BSC was to benet from the expertise and insight of market analysts (i.e. investment banks, consultants, rating agencies) with regard to banks performance measures. This was achieved by sending a short questionnaire in the autumn of 2009 to selected market analysts, as well as by organising a workshop at the ECB in December 2009. The questionnaire aimed to compile market analysts views on performance measure metrics in a structured and comparable format. The questionnaire (reproduced in Annex 2) comprised four parts: dening the scope of performance measures analysis; evaluating various performance measures metrics; identifying proprietary indicators and models; and assessing the impact of the global nancial crisis on the analysis of banks performance. QUESTIONNAIRE RESPONDENTS Bank analysts Universal banks (UniCredit, JP Morgan) Investment banks (Morgan Stanley) Commercial banks (Bank of Valetta) Securities companies (Cheuvreux, KBW) Consultants Oliver Wyman McKinsey Rating agencies Fitch Ratings Moodys Investor Services Mr Sam Theodore (currently Manager of the Banking Sector Team at the UK FSA, but with previous ratings experience)
ANNEXES
Altogether, 12 responses were received, with seven from bank analysts, two from consultants and three from rating agencies. The workshop provided the opportunity for a more open exchange of views with market participants with the aim of, inter alia, examining or challenging preliminary questionnaire ndings. Each of the ve presentations was accompanied by a brief discussion, which elaborated on particular issues or points of view. Workshop participants Mr Davide Taliente and Ms Vronique McCarroll from Oliver Wyman Mr Joachim Mller from Crdit Agricole Cheuvreux Mr Michael Dawson-Kropf from Fitch Ratings Mr Carlos Egea from Morgan Stanley Mr Sam Theodore from the UK FSA
37
2 QUESTIONNAIRE ON PERFORMANCE MEASURES

0 0.1 0.2 0.3 INTERVIEWEE Respondent institution: Respondents name: Respondents position:
1 1.1
DEFINING THE SCOPE OF PERFORMANCE MEASURES ANALYSIS
When asked about performance measures of banks, which of the following metrics would you include? As for indicator relevance, please choose a mark according to the highest relevance (1 most relevant, 6 least relevant on an ordinal scale). Indicator relevance 1.1.1 Revenue and cost changes Please choose a mark 1.1.2 Revenue and cost composition and sustainability Please choose a mark 1.1.3 Efciency indicators (e.g. cost-to-income, ROE, ROA) Please choose a mark 1.1.4 Asset quality indicators (provisioning, coverage ratio, NPLs) Please choose a mark 1.1.5 Market-based indicators (e.g. P/E, P/B, etc.) Please choose a mark 1.1.6 Market-based indicators of credit risk response, funding costs (bond spreads, credit default swaps) Please choose a mark 1.1.7 Liquidity indicators Please choose a mark 1.1.8 Capital adequacy indicators Please choose a mark 1.1.9 Efciency indicators related to capital (RAROC, EVA) Please choose a mark 1.1.10 Indicators of banks systemic relevance Please choose a mark 1.1.11 Other Please specify:
2 2.1
EVALUATING PERFORMANCE MEASURE METRICS Please rank the following revenue and cost indicators according to their usefulness when assessing the performance of a bank: As for indicator relevance, please choose a mark according to the highest relevance (1 most relevant, 6 least relevant on an ordinal scale). Indicator relevance Net interest income/total assets Please choose a mark Net interest income/interest-bearing assets Please choose a mark Net interest income after impairment charges/total assets Please choose a mark Total income/total assets Please choose a mark Operating expenses (or staff costs)/total assets Please choose a mark Income from fees and commissions/total income Please choose a mark Other income/total income Please choose a mark Trading income/total income Please choose a mark Other Please specify:
2.1.1 2.1.2 2.1.3 2.1.4 2.1.5 2.1.6 2.1.7 2.1.8 2.1.9
38
ANNEXES 2.2 Please rate how well each of the following efciency indicators reects the performance of a bank. As for indicator relevance, please choose a mark according to the highest relevance (1 most relevant, 6 least relevant). Indicator relevance Cost-to-income ratio Please choose a mark Cost-to-income ratio, including impairment charges Please choose a mark Return on equity Please choose a mark Return on risk-weighted assets Please choose a mark Return on assets Please choose a mark Return on tangible equity Please choose a mark Comments, if any:
2.2.1 2.2.2 2.2.3 2.2.4 2.2.5 2.2.6 2.2.7
2.3
Do you assess the sustainability of bank revenues? If so, what measure(s) do you use? Please specify:
2.4
Please rank the following market-based indicators according to their usefulness when assessing the performance of a bank. As for indicator relevance, please choose a mark according to the highest relevance (1 most relevant, 6 least relevant). Indicator relevance 2.4.1 Price-to-earnings (P/E) Please choose a mark 2.4.2 Price-to-book value (P/B) Please choose a mark 2.4.3 Price-to-tangible equity (P/TE) Please choose a mark 2.4.4 Distance to default Please choose a mark 2.4.5 Expected default frequency Please choose a mark 2.4.6 Implied volatility of banks stock prices Please choose a mark 2.4.7 Spread on senior debt Please choose a mark 2.4.8 Spread on subordinated debt Please choose a mark 2.4.9 Spread on hybrid capital bonds Please choose a mark 2.4.10 Credit default swap Please choose a mark 2.4.11 Other Please specify: 2.4.12 In the case of non-listed banks, do you use any specic metrics? Please elaborate:
2.5
2.5.1 2.5.2 2.5.3 2.5.4 2.5.5 2.5.6
Please rate how well each credit-risk related indicator reects the performance of a bank. As for indicator relevance, please choose a mark according to the highest relevance (1 most relevant, 6 least relevant). Indicator relevance Impairment charges as a percentage of total income Please choose a mark Impairment charges as a percentage of total assets Please choose a mark Impairment charges as a percent of loans (bps on the loan book) Please choose a mark Non-performing loans/total loans Please choose a mark Coverage ratio 1 (i.e. cumulative provisions/non-performing loans) Please choose a mark Net non-performing loans/regulatory own funds Please choose a mark
39
2.5.7 2.5.8 2.5.9
Credit value at risk Leverage ratio Other
Please choose a mark Please choose a mark Please specify:
2.6 2.6.1
How do you incorporate risk elements (credit, market, liquidity) into the assessment of the banks performance? How do you incorporate revenues and risks (market, liquidity, operational) related to off- balance sheet activities (i.e. asset management, derivatives/securitisation, structuring, etc.) into the assessment of banks performance?
Please elaborate:
Please elaborate:
2.7 2.7.1
2.7.2
Do you rely on RoE analysis? If so, which kind of analysis do you perform? DuPont analysis (decomposition into margin (net income-to-operating revenues); turnover (operating revenues-to-assets); and leverage (assets-to-Tier 1 equity or an even ner breakdown) Other
Parameter importance
Please choose a mark Please specify:
2.8
2.8.1 2.8.2 2.8.3 2.8.4 2.8.5 2.8.6 2.8.7 2.8.8
Do you conduct analysis by business line/unit? If so, which are the main business lines you take into consideration? As for indicator relevance, please choose a mark according to the highest relevance (1 most relevant, 6 least relevant). Indicator relevance Retail banking Please choose a mark Corporate/wholesale banking Please choose a mark Investment banking Please choose a mark International operations Please choose a mark Insurance activities Please choose a mark Proprietary trading/treasury Please choose a mark Asset management Please choose a mark Other Please specify:
2.9 2.9.1 2.10
Which indicators do you use to evaluate the business lines? Please list them. Please elaborate: How do you take diversication over different business lines into account and how do you evaluate it? Please elaborate:
40
ANNEXES 3 3.1 PROPRIETARY INDICATORS AND MODELS Do you use proprietary indicators or models for the assessment of the performance of banks? If so, please explain (in the case of a proprietary model, please explain how you estimate inputs (e.g. growth rates, discount rates, etc.). Please elaborate:
3.1.1
3.2 3.2.1
What are the advantages of the aforementioned indicators/models versus standard performance measures? Please elaborate:
4 4.1 4.1.1
IMPACT OF THE CRISIS Has the crisis shown potential aws in certain performance measures? If so, which measures do you believe proved to be insufcient or misleading? Please elaborate:
4.2
Has the crisis prompted you to develop new performance measures or change the computation of existing ones? 4.2.1 Please elaborate: 4.2.2 What are the advantages of the new metrics over standard metrics? 4.2.2.1 Please elaborate:
4.3 4.3.1
What other qualitative aspects (e.g. corporate governance) came to the fore in analysing banks performance since the beginning of the crisis? Please elaborate:
41
3 REFERENCES
Abuzayed, B. and Molyneux, P. (2009), Market Value, Book Value and Earnings: is Bank Efciency a Missing Link?, Managerial Finance, Vol. 35 No 2, pp. 156-179. Ang, J. and Clark, J. (1997), The Market Valuation of Bank shares with Implications for the Value Additively Principle, Financial Markets, Institutions and Investments, Vol. 6, New York University Salmonn Center, pp. 1-22. Dev, A. and Rao, V. (2006), Performance Measurement in Financial Institutions in an ERM Framework: A Practitioner Guide, Risk Books. De Wet, J.H.v.H. and De Toit, E. (2007), Return on equity: A popular, but awed measure of corporate nancial performance, Department of Financial Management, University of Pretoria. Fernandez, P. (2002), EVA and Cash Value Added Do Not Measure Shareholder Value Creation, Valuation Methods and Shareholder Value Creation, 2002 Academic Press, San Diego, CA. Hirschey, M. and Wichern, D.W. (1984), Accounting and Market Value Measures of Protability: Consistency, Determinants and Uses, Journal of Business and Economic Statistics, Vol. 2, No 4, pp. 375-383. Ittner, C. and Larcker, D. (2000), Non Financial Performance Measures: What Works and What Doesnt, Financial Times, Mastering Management Series. Kaplan, R.S. and Norton, D.P. (1992), The Balanced Scorecard Measures that Drive Performance, Harvard Business Review, January-February, pp. 71-79. Kimball, R.C. (1998), Economic Prot and Performance Measurement in Banking, New England Economic Review, Federal Reserve Bank of Boston, issue July, pp. 35-53. Martin, A., Salas, V. and Saurina, J. (2007), Measurement of capital stock and input services of Spanish Banks, Working Paper No 0711, Banco de Espaa. Martin, A. and V. Salas (2007), How do intangibles create economic value: An application to Spanish Bank, Working Paper No 0730, Banco de Espaa. Rappaport, A. (1986), Creating shareholder value, New York: The Free Press. Sagar, R.D. and Rajesh, B. (2008), Incorporating Intangible Aspects in Performance Evaluation of Indian Banks. Zhang, Y. and Longyi, L. (2009), Study on Balanced Scorecard of Commercial Bank in Performance Management System, Nanchang, P.R. China, pp. 206-209. Weissenrieder, F. (1997), Value Based Management: Economic Value Added or Cash Value Added?, FWC AB Study No 1997:3. Worthington, C.A. and West, T. (2001), Economic Valued Added: A Review of the Theoretical and Empirical Literature, Adian Review 9 (1), pp. 67-86.
42

Beyond Roe How To Measure Bank Performance 201009 en

Uploaded by

Copyright:

Available Formats

Beyond Roe How To Measure Bank Performance 201009 en

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Beyond Roe How To Measure Bank Performance 201009 en

Uploaded by

Copyright:

Available Formats

B E YO N D RO E H OW TO M E A S U R E BANk pERfORMANcE

AppENDix TO T H E REpORT ON EU BANkiNg STRUcTURES

ECB Beyond RoE How to measure bank performance September 2010

ECB Beyond RoE How to measure bank performance September 2010

ECB Beyond RoE How to measure bank performance September 2010

ECB Beyond RoE How to measure bank performance September 2010

2 WHAT IS PERFORMANCE MEASUREMENT? WHAT IS IT USED FOR?

ECB Beyond RoE How to measure bank performance September 2010

2 WHAT IS PERFORMANCE MEASUREMENT? WHAT IS IT USED FOR?

ECB Beyond RoE How to measure bank performance September 2010

2 See, for example, Weissenrieder, F. (1997) and Fernandez, P. (2002).

ECB Beyond RoE How to measure bank performance September 2010

2 WHAT IS PERFORMANCE MEASUREMENT? WHAT IS IT USED FOR?

ECB Beyond RoE How to measure bank performance September 2010

Table 1 Analyst indicator preference ranking by category

Category of indicators Revenue and cost

ECB Beyond RoE How to measure bank performance September 2010

2 WHAT IS PERFORMANCE MEASUREMENT? WHAT IS IT USED FOR?

ECB Beyond RoE How to measure bank performance September 2010

3 WHAT IS WRONG WITH ROE?

3 WHAT IS WRONG WITH ROE?

ECB Beyond RoE How to measure bank performance September 2010

Chart 1 The evolution of RoE of 12 international banks

Chart 2 The evolution of RoE in sub-groups

Chart 3 Evolution of price/expected earnings

ECB Beyond RoE How to measure bank performance September 2010

3 WHAT IS WRONG WITH ROE?

ECB Beyond RoE How to measure bank performance September 2010

250 200 150 100 50 0 -50 -100 -150 -200 -250

ECB Beyond RoE How to measure bank performance September 2010

Chart 8 The decomposition of RoE changes for universal banks

3 WHAT IS WRONG WITH ROE?

Chart 9 Evolution of large banks RoA

-1.0 2003 2004 2005 2006 2007 2008 2009 2010

ECB Beyond RoE How to measure bank performance September 2010

50 45 40 35 30 25 20 15 10 5 0 2003 2004 2005 2006 2007 2008 2009 Source: Bloomberg.

ECB Beyond RoE How to measure bank performance September 2010

3 WHAT IS WRONG WITH ROE?

ECB Beyond RoE How to measure bank performance September 2010

3 WHAT IS WRONG WITH ROE?

Chart C Interest on operating revenues

Chart D Cost of risk over interest margin

ECB Beyond RoE How to measure bank performance September 2010

3 WHAT IS WRONG WITH ROE? Cost efficiency as a source of resilience

ECB Beyond RoE How to measure bank performance September 2010

4 REFINEMENTS AND ISSUES TO CONSIDER

ECB Beyond RoE How to measure bank performance September 2010

4 REFINEMENTS AND ISSUES TO CONSIDER

ECB Beyond RoE How to measure bank performance September 2010

4 REFINEMENTS AND ISSUES TO CONSIDER

ECB Beyond RoE How to measure bank performance September 2010

5 ADDITIONAL FACTORS AND ALTERNATIVE MEASURES OF PERFORMANCE

RoE and revenue breakdown of the case bank

5 ADDITIONAL FACTORS AND ALTERNATIVE MEASURES OF PERFORMANCE

Chart 12 Balance sheet analysis of the decomposed RoE measure

Business driver flow

Business driver stock

ECB Beyond RoE How to measure bank performance September 2010