Key Risk Indicators

Section 7

Tools
Tool 7.1 Frequently asked questions on KRIs Tool 7.2 Illustration of a KRI for staff turnover Tool 7.3 Roles and responsibilities for KRIs Tool 7.4 Major steps necessary to generate KRIs Tool 7.5 Insurance specific operational risk KRIs Tool 7.6 Generic operational risk KRIs Tool 7.7 A KRI tool

7 Key Risk Indicators

What are key risk indicators (KRIs)?
KRIs are parameters which can act as indicators and which can be seen to be predictive regarding changes in the risk profile of a business. This enables timely action to be taken to deal with issues arising. KRIs can include: something observed or calculated used to identify a condition or trend; an instrument or gauge measures something and registers the measurement; and something such as a light, sign, or pointer provides information, for example, about which direction to follow, and which serve as signals.

KRIs then are measures which indicate the level of and changes in an organisations risk profile. This is achieved by focusing KRIs on the root causes of potentially significant risk events and exposures, as illustrated below.

Cause1

Effect1

Cause2

Risk event

Effect2

Cause3

Effect3

Key risk indicators

Detective controls

Expected loss events

The key attributes of KRIs are that they: highlight current risk levels by providing a measure of the status of an identified risk and the effectiveness of its control. Risk indicators can provide information which gives a useful ongoing view of the underlying behaviour of the risk profile1; highlight trends and changes in risk level by monitoring changes in risk between formal risk and control assessments; provide early warning signals through predictive risk indicators which highlight changes in the risk environment, control effectiveness and potential risk issues, before they crystallise and result in loss or other exposure;

1 Another type of indicator is a key control indicator (KCI), which is a measure of the effectiveness (e.g. design and performance) of a specific control. Deterioration in KCIs can show an increase in residual risk impact or likelihood. KCIs are relevant to a particular control activity(s).

Page 3 of 16

Section 7 Key risk indicators enable actions that prevent or minimise material loss or incident by prompting timely action on early warning signals; and express escalation criteria for risk management by using thresholds to convert raw indicator data into meaningful risk ratings to aid effective decision making.

Key risk indicators can be classified into two categories, namely: specific indicators, which relate to particular processes within a franchisee, such as the number of reconciling items in a given area; and environmental indicators, which impact the franchisee as a whole, for example, business volume.

How do you identify key risk indicators?

The diagram below illustrates how key risk indicators may be identified, using people risk as an example.

Define risk category

Develop causal map

Establish KRIs

Risk examples

Risk causes

Risk indicators
Staff turnover ratios

Average time to fill Protect staff Poaching by competitors Low job satisfaction Exit interview summaries Inadequate skills and education Poor performance of staff % job offers accepted Benchmark salaries against industry standard Low staff morale Staff survey summaries No of applicants per vacancy No of staff not completing their probationary period

Inability to recruit

People Risk

Inability to retain

The following are considerations in the selection of KRIs: ideally determined for many of the significant risks identified in the risk and control self assessment (self assessment) process; can provide early warning signals to trigger actions that reduce potential risk exposures; some indicators are meaningless on their own and need to be combined with other KRIs. In many cases, it is a group of KRIs that will provide the best management information for a meaningful assessment;

Page 4 of 16

Section 7 Key risk indicators can indicate past, current and projected level of risks and can be used as a criteria to monitor, escalate and manage risk and related actions; and KRIs relevance and change in importance over time.

The appropriate frequency of reporting and monitoring of each identified indicator is also an important consideration. The following (non exhaustive list) provides some sources of information that can help to identify significant risks and aid in KRI identification: historical internal loss events; risk and control self assessment results; internal / external audit findings; regulatory inspection findings; and workshops / discussions with business functions e.g. Human resources (including staff turnover statistics).

Why are they important?

An important objective of key risk indicators is to provide a measure of risk causes in addition to the effects of risk, so aiding robust risk management and enabling timely action. Key risk indicators play an important role in: Risk management namely: the ability of KRIs to predict potential risk hotspots can help a franchisee avoid or minimise losses; KRIs help identify process and/or control weaknesses and thus enable action to be taken to strengthen controls and resolve issues; and targets for KRIs can be set to drive behaviour and desired outcomes for the franchisee.

Risk appetite setting one of the methods to articulate risk appetite, particularly for operational related risk, is through the setting of tolerance and escalation levels for key risk indicators; Regulatory compliance identification and management of KRIs is an area of regulatory focus; and Capital calculation data from established KRIs can be used as one of the inputs into operational risk capital calculations.

What practical steps are necessary for implementation?

From a practical standpoint, KRIs need to be built into (rather than bolted onto) operational and business processes: "opening up" legacy operational and business processes and supporting systems to retroactively build in KRI requirements is likely to be an expensive and potentially impractical proposition for many organisations; as a result, for all but a few absolutely vital KRIs, introducing KRIs for most organisations is most practically undertaken as an integral part of new system development and process transformation initiatives where the cost is likely to be much lower.

Page 5 of 16

Section 7 Key risk indicators Given the above, the following guidance is intended to help organisations understand what is involved from a practical standpoint to implement this KRI section: Staff resource and skills business unit workshops to identify the important key risk indicators, drawing on senior business unit managers and expert staff, facilitated by risk management IT programming resource to develop and integrate KRI reporting briefing of senior management and committees internal auditor or external consultant review to provide technical assistance and support and assurance that KRIs are focussed on key areas and are robustly implemented, such that they support risk monitoring and decision making

Enabling technology MS Excel and/or Access software bespoke operational risk software (developed in house or from third party vendors)

Time initial workshops and IT implementation 3 to 6 months iteration, refinement and assurance over selected KRIs 3 months overall expected implementation time 6 to 12 months

Direct / indirect costs management and staff time per 2 to 3 hour workshops IT staff time for programming risk management time facilitation, documentation and review internal audit / external consultants for support, quality assurance, technical review and independent assurance

Given the differences in scale, sophistication and resources between franchisees, the capacity to establish KRIs, to monitor and react to KRI information, and the overall number of KRIs will vary significantly across organisations. It is therefore anticipated that this section would be of relevance to organisations as follows: Large / composite highly relevant: expected to be in use for business unit management reporting; Medium / multi-line highly relevant: expected to be generally in use for business unit management information; and Small / mono line relevant: but with less extensive KRI reporting.

Page 6 of 16

Section 7 Key risk indicators

Relevant toolkit contents

Relevant toolkit contents for key risk indicators include: Tool 7.1 Frequently asked questions on KRIs: frequently asked questions and helpful pointers with respect to key risk indicators; Tool 7.2 Illustration of a KRI for staff turnover: an example key risk indicator for staff turnover to illustrate the design / use of key risk indicators; Tool 7.3 Roles and responsibilities for KRIs: an illustration of principle roles and responsibilities with respect to key risk indicators and the associated reporting structure; Tool 7.4 Major steps necessary to generate KRIs: identifies six major steps necessary in establishing key risk indicators, with an illustrated example range of KRI perspectives for LMP slip compliance; Tool 7.5 Insurance specific operational risk KRIs: a non-exhaustive list of example insurance specific key risk indicators for operational risk, broken down by people, processes, systems & external events; Tool 7.6 Generic operational risk KRIs: a non-exhaustive list of example generic key risk indicators for operational risk, again broken down by people, processes, systems & external events; Tool 7.7 A KRI tool: a tool that can be used to create heatmaps, trend analysis and management information for key risk indicators, with illustrated insurance specific KRIs including supporting dummy data tables (by operational risk category) and working KRI reporting graphics, formats and templates.

Page 7 of 16

Tool 7.1 Frequently asked Questions

Frequently asked questions & helpful pointers How should KRIs link with risk assessments and event reporting? Focus on indicators which track changes in the risk profile or the effectiveness of the control environment. What is the most practical way of arriving at a set of KRIs? Concentrate on the significant risks and their causes and consider forward looking and historical indicators. How should KRIs be set in terms of measurement? Consider absolute values and numbers, ratios, percentages, ageing, etc. With what frequency should KRIs be captured? Data on KRIs should be collated on a systematic and consistent basis in order to be meaningful, e.g. on a monthly basis. How should acceptable ranges and escalation thresholds be determined for KRIs? Consider in detail the risk appetite set by the board and cascaded down to business level when setting thresholds. Will KRIs change over time? Key risk indicators will evolve over time. Analysis of actual losses and near misses will assist in identifying which KRIs are the best at giving early warning and allowing timely action. Periodic review needs to take place of the indicators themselves and their associated thresholds to ensure they remain aligned with the dynamic of the business environment and the significant risks faced by the franchisee at any point in time. How can KRIs be linked to operational risk appetite and tolerance? By defining targets, tolerance ranges and escalation thresholds for risk appetite. How should roles and responsibilities be assigned for the collection, collation, monitoring and challenge of KRIs? Assign KRI owners who are responsible for collection and collation. The risk management function should provide challenge and internal audit, independent validation.

Page 8 of 16

TOOL 7.2 Illustration of a KRI FOR STAFF TURNOVER

A key risk indicator for monitoring and responding to risk around the effectiveness and continuity of essential business functions relates to staff turnover levels. This is important for all positions and particularly for positions where there is a critical level of dependency. Key risk indicators of this type require; tolerance thresholds in order to give a meaningful representation of the risk; and the resultant ratings which could be used to create heat map reporting on indicators.

For example, when given thresholds are breached there will be a requirement to escalate to an appropriate level of management: Below 24% No risk. The organisation is comfortable with the level of staff turnover. No escalation or treatment required. Above 24% Potential risk. The risk is a concern and HR would be expected to monitor actively and establish causes and actions. Escalation required raising awareness but explanatory report not required. Above 28% Significant risk. Action and escalation with explanatory report required. Staff turnover KRI % 0 24 24 28 > 28 Thresholds can be used alongside targets set by management. These could be flexed over time as objectives / strategy and risk appetite develop. These targets will help drive the desired behaviour and outcomes and improve the organisations operational risk profile over time. Risk level

Page 9 of 16

TOOL 7.3 EXAMPLE ROLES AND RESPONSIBILITIES FOR KRIS

The following diagram illustrates principal roles and responsibilities in respect of KRIs and the associated reporting structure2.

Business function Identification of indicators

Risk management Provide guidance and challenge the selection of KRIs and thresholds Monthly reporting on KRI breaches Ad-hoc escalation reporting to Board Identify trends across the business

Internal audit Provide validation / independent assurance around the KRI process Incorporate outputs into audit plan

Setting of thresholds Monitor position against targets and limits Escalate breaches to operational risk management

Key risk indicator workflow diagram

The diagram below illustrates the steps from development to reporting of key risk indicators and could be used to facilitate a franchisee workshop or group discussion on KRIs.

Set up KRI definitions

Define / assign thresholds

Set up submissions

Submit to KRI owner

KRI owner KRI reporter

KRI owner review Submit data to KRI controller KRI owner review & approval Capture data

2 This structure follows the Three lines of defence approach used in the governance section of the toolkit with respect to roles and responsibilities within a risk management function.

Page 10 of 16

TOOL 7.4 MAJOR STEPS necessary to generate KRIS

This example illustrates 6 major steps for establishing key risk indicators.

1 Inventory exists

Are there existing metrics which could meaningfully be used as KRIs?

Potential Key Risk Indicators CSM cancellation rate by product ($ per AOF) Partner credit report (number by $ amount) 1 1 9 9 5.70
20.0% 25.0%

2 Assess KRI

How well do these existing metrics cover the risk drivers?

Root Causes File not received by BP Complete file not received by BP BP credits to wrong cardholder account BP credits incorrect amount Overall Rating

20% 10% 30% 30%

9 3 0 0 2.10

1 1 1 3 1.50

1 1 0 1 0.60

3 Design

What new KRIs do I need to develop to address any gaps?

Chart Title 14 12

10

4 Validate

How well do these KRIs correlate to a potentially significant risk events?

0 0.0%

5.0%

10.0%

15.0%

Days to credit metric (contact to credit)

30.0%

File receipt indicator (% received/sent)

Weighting

35.0%

40.0%

F10 Flowdown

5 Develop KRI

Flowdown

What type of graphical or other reporting should I use to monitor these KRIs?

03/09/01-11/02/01: Mean = 1.85% Std Dev = 0.57%

4.00%

*A s of the w eek ending 11/30/2001, Fraud is r esponsible for F10 f low dow n.

3.50%

+3 Std Dev

3.00%

+2 Std Dev

2.50%

+1 Std Dev

2.00%
M ean

1.50%
-1 Std Dev

1.00%
-2 Std Dev

0.50%
-3 Std Dev

0.00%

6 Establish KRI

What actions do I need to take to implement these KRIs? e.g. source data collection

Week Ending

The following diagram illustrates the need for a balanced range of KRI perspectives in order to monitor a significant risk, for example, LMP slip compliance. Risk event LMP slip standards are not adhered to.

Error rate metric

Slip error rate (%)

Checking procedures accuracy rate (%)

Control effectiveness driver

Volume driver

Slip load (# slips)

Period of risk not completed correctly (%)

Causal driver
(individuals not following procedure)

Page 11 of 16

TOOL 7.5 Insurance specific operational risk kris

Risk category People Example key risk indicator staff turnover rates sickness rates per division per month / year number of contract staff percentage referral rates number of complaints referred to ombudsman per month / year

Processes

Basic underwriting process percentage slips recorded within 24 hours percentage slip entry error rate percentage endorsements recorded percentage aggregates (or proxy for max aggregate) recorded within 24 hours percentage of slips underwritten within authority percentage referral where appropriate percentage underwriting guidelines to be followed, subject to referral Underwriting review percentage of slips with greater than 1m premium or 10m exposure to be peer reviewed within 10 days Monitoring of underwriting number of premium and reinsurance debt greater than 90 days outstanding number of claims greater than 10m to be reviewed number of non-moving claims to be reviewed every 6 months number of contracts greater than 110% loss ratio to be investigated, (subject to de-minims 1m premium) number of exposures not recorded within 1 month of underwriting number of less than 100 binders not written, with minimum income 100k number of outstanding wordings greater than 1 month Agency level underwriting controls percentage of major contracts to be independently reviewed within 1 month of underwriting Placement of reinsurance programme percentage of reinsurance order forms and cover notes reviewed against requirements per month percentage accuracy on reinsurance orders and cover notes per month

Page 12 of 16

Tool 7.5 Insurance specific operational risk KRIs

Risk category

Example key risk indicator Agency level reinsurance purchase controls percentage set limits on exposures to reinsurers (e.g. less than 10% premium to be placed with 1 reinsurer) elapsed days since security ratings for reinsurers last measured number of material exposures / recoveries from reinsurers / erosion of reinsurance per month / year Claims processes percentage material claims / disputes / complaints reported to senior management / Board per month number of second adjustor review for material claims greater than 10m per month / year number of monthly reconciliation breaks of claims reserves percentage of claims advices reviewed within 2 days percentage of collection notes issued within 30 days percentage of outstanding debts chased after 90 days number of ongoing disputes with syndicate actuaries or accountants per month / year Liquidity processes percentage contingency plans for potential or expected cash shortfalls Generic process failures percentage manual input errors per month number underwriting staff with access to accounting system percentage of management information and exception reporting produced and reviewed within 2 days of month end percentage of IT system queries responded to within 24 hours percentage of complaints resolved within 1 month of receipt number of adverse press comment(s) per month / year number of outstanding external and internal audit / compliance / regulatory report points

Systems

number of IT system outages per month number of IT security breaches per month / year number of IT virus caused outage per month / year number of IT supplier failure incidents per month / year number of 1 day server failures per month / year number of tested IT disaster recovery procedures and systems per year

External events

number of incidents of third party provider failure, from outsource providers per month / year number of successful external fraud incidents per month / year number of IT system security breaches per month / year number of regulator action / concerns per month / year number of outstanding Lloyds operational risk review points raised

Page 13 of 16

TOOL 7.6 Generic operational risk kris

Risk category People Example key risk indicator diversity stats by ethnicity / sex / age / disability number of staff disciplinaries / dismissals percentage of staff appraisals below satisfactory number of staff grievances results of staff surveys staff turnover rates percentage of joiners leaving within the first 6 months actual versus budgeted FTE proportion of permanent versus temporary staff average length of service per member of staff average time to fill vacant positions staff absenteeism / sickness rates overtime actual versus budgeted training costs average number of days training per member of staff percentage of staff who have attended health and safety training percentage of staff who have attended financial crime / anti money laundering training percentage of staff who have not had two weeks consecutive leave proportion of permanent versus temporary staff average length of service per member of staff

Processes

number / percentage of accounts with outstanding / incomplete customer documentation number / percentage of unauthorised customer accounts opened number / percentage of customer accounts with significant change in volume / value of transactions number of incidents reported to the Money Laundering Reporting Officer number / percentage of customer accounts with unusual transactions number and nature of limit breaches number of new products market share by product customer intake / retention / churn by product versus budget significant revenue variance by product number of new products / new products awaiting approval / unapproved products projected transaction processing volumes versus capacity percentage change in transaction volumes percentage of total transactions handled number / value / age of processing exceptions processing exceptions as a percentage of transaction volumes number of customer complaints number of compliance / regulatory breaches

Page 14 of 16

Tool 7.6 Generic operational risk KRIs

Risk category

Example key risk indicator budgeted versus actual FTE within customer & account servicing / moving money / collections supplier performance versus SLA number of unreconciled accounts number / value / age of unreconciled items

Systems

number and type of security violations number of virus incidents systems usage versus capacity systems downtime number, type and severity of system incidents / SLA breaches number of system upgrades / version releases number of open system change requests number of help desk calls virus incidents number of outstanding business continuity plans utility performance statistics

External events

number of overdue tests / maintenance of detection & suppression mechanisms number of outstanding disaster recovery plans number of overdue disaster recovery plan tests number and nature of physical security incidents

Page 15 of 16

TOOL 7.7 A kri tool

This is an Excel tool that can be used to create heatmaps, trend analysis and management information for key risk indicators. This tool can be found in key risk indicator section of the toolkit. The data in this template is illustrative and for example purposes only. The pie charts and graphs are plotted using data contained within the tool. As a result, an organisation could use this tool as a template replacing the example data with their own. It is important to remember that as it stands, this tool will not automatically update cell colour.

Page 16 of 16