Practical Guide To Producing Consumer Price Indices
Practical Guide To Producing Consumer Price Indices
Practical Guide To Producing Consumer Price Indices
ECE/CES/STAT/NONE/2009/2
Foreword
This Handbook is targeted at developing countries focusing on practical solutions to the problems facing
compilers of the consumer price index (CPI) in the developing world. It provides practical guidance on all
aspects of compiling a CPI, and offers advice on a range of operational issues in data collection, data
processing and publication. Although targeted at CPI compilers in developing countries it will also be of
interest to compilers more generally.
The Handbook follows and supplements the international Consumer Price Index Manual. Theory and Practice,
published in 2004. It is accompanied by a CD‐ROM with training aids and references, including interactive
examples of the methods and calculations given in the Handbook, where users can input their own data.
The publication of the Handbook is the result of a collaborative effort involving experts from a number of
countries and organisations, and it has benefited from the input from CPI compilers in developing countries.
By referring to the CPI Manual and drawing on experience of experts from both developed and developing
countries the Handbook provides internationally agreed guidance and suggests good practices with the
objective to help national statistical offices to further improve their CPI statistics.
iii
Preface
Introduction
The production of this Handbook has been funded by the UK Department for International Development
under the direction of the UK Office for National Statistics and developed with support from the Inter‐
Secretariat Working Group on Price Statistics. It is designed to give practical guidance on the compilation of
consumer price indices (CPIs), focusing on practical solutions to issues facing the compilers of CPIs in the
developing world. It offers advice on a whole range of operational issues confronted by index compilers from
the sampling and collection of prices through to index construction and final publication. Although targeted
at compilers of CPIs in developing countries it will also be of practical use to compilers of CPIs in other
countries.
This Handbook supplements the ILO Manual on Consumer Price Indices. The latter provides a comprehensive
review of the economic and statistical theory which underlies CPIs, as well as an in‐depth guide to
compilation, and has been written for the benefit of users as well as producers. In contrast, the Handbook
takes the underlying theory as given. Cross‐references to the ILO Manual are provided for those who wish to
investigate the more theoretical aspects. Users of this Handbook are encouraged to take advantage of this as
it is important to understanding the conceptual and theoretical context underlying the practical advice being
given.
The Handbook is also accompanied by a set of PowerPoint slides for local training purposes and a CD‐ROM
with interactive examples of the illustrative calculations given in the Handbook, where users can input their
own data.
Background
In 2004 a comprehensive Consumer Price Index Manual: Theory and practice was published by the
International Labour Office (ILO), with the support and co‐operation of a number of other international
organisations. It is an expanded revision of the earlier ILO publication Consumer price indices: an ILO manual,
published in 1989. The new Manual is widely regarded as the main international reference source for CPIs. It
covers virtually every aspect of a consumer price index. Yet some CPI practitioners, especially in developing
countries, have found that its comprehensiveness has made it less suitable as a day‐to‐day source of
guidance, particularly on detailed practical issues. This Handbook, which is a companion volume to the
Manual, focuses primarily on practical matters, and deals less with theory than does the Manual. It should be
seen not as a replacement for the Manual but as a supplement.
In December 2003 the International Conference of Labour Statisticians, convened by the ILO, adopted a
Resolution concerning consumer price indices. This Resolution replaced the previous Resolution on CPIs
adopted in 1987. The ILO Resolution and the CPI Manual are compatible documents, and indeed the
preparation of the draft Resolution was done in parallel with the preparation of the Manual. The text of the
ILO Resolution is included as Appendix 3 to the CPI Manual itself.
In planning the present Handbook, it was decided that although designed for use by CPI practitioners in every
country, special attention should be paid to the needs of developing and transition countries. The authors of
each chapter were made aware of this policy from the start. The ILO, in collaboration with UNECE, carried out
a user survey of the CPI Manual in order to elicit useful ideas for inclusion in the Handbook, and to discover
more about the perceived limitations of the Manual. The survey was addressed to all national statistical
institutes and also to other users such as ministries of labour and workers’ and employers’ organisations.
Over 100 responses from a similar number of countries were received. More than half of respondents felt
that more space should be devoted to practice related issues and the most useful chapter in the Manual was
thought to be Chapter 9 on the practical aspects of CPI calculation.
v
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Given the special focus on developing countries, it was decided that a panel of reviewers from such countries
should be involved in the preparation of the Handbook. More than ten CPI experts from a representative
group of countries were accordingly asked to comment on specific chapters. Their views were regarded as
extremely valuable and the content of the Handbook has benefited as a result. For example, there are
frequent references to the problems met in collecting prices in markets, which to the reader in a developed
country may seem disproportionate, but to CPI practitioners in developing countries are crucial.
To assist readers of the Handbook, the chapter headings have been made as compatible as possible with that
of the Manual. In terms of general ordering this has been achieved, though exact numerical compatibility is
limited to just a few of the 18 chapters.
With its emphasis on practical matters, the Handbook has been written with a view to use by CPI statisticians
working in national statistical offices. The various topic‐specific chapters should be useful for practitioners
faced with specific problems in certain areas of CPI compilation (e.g. how to deal with seasonal products or
how to aggregate prices at the lowest level where there are no weights available). It is also hoped that the
Handbook can be used for training purposes. The language used, as well as the style, has been deliberately
aimed at a level which should be easily understood not only by CPI experts but also by new professional staff
joining a CPI department in a national statistical institute. As with the Manual itself, an electronic version of
the Handbook is available on the internet at http://www.ilo.org/cpi‐manuals and is regarded as a “living
document” that will be amended and updated as necessary over the coming years. As mentioned in the
Introduction, there is an accompanying CD‐ROM which includes not only worked examples which can be
adapted using national datasets, but also the text of the CPI Manual, and some other useful background
papers.
A guide to readers
The Handbook broadly follows the same sequence of chapters as in the Manual so that it covers in logical
sequence the successive steps in index compilation. But it is not designed necessarily to be read from cover
to cover. It is expected that compilers will be able to dip into it as required. Each chapter is self‐contained,
giving a brief summary of the corresponding chapter in the main ILO Manual before providing practical
guidance. Illustrative examples of calculations and model documents are also presented to assist the user.
The Handbook uses the same terminology as the Manual. Further details on the contents of the Handbook
are given in Chapter 1.
The Handbook is not prescriptive where it cannot or does not need to be, and in these cases makes
recommendations or gives examples of what might be considered best practice. The reader should bear two
things in mind. Firstly, it is not always possible to give definitive practical guidance as this would wrongly
assume that the solutions to conceptual problems are, themselves, always clear‐cut and that there are no
choices to be made about precisely how a practical solution is implemented. Secondly, what is applicable
locally and what can be achieved on the ground will depend on the resources available to the individual
National Statistical Institute.
Acknowledgements
The production of this Handbook was a collaborative effort involving many people. Authors of the individual
chapters and their co‐authors were drawn from CPI experts around the world and contributions were sought
from experts with extensive experience in working on CPI issues in developing countries. The Handbook also
drew on documentation from the UK Office for National Statistics and from work undertaken by the
Statistical Office of the European Communities (Eurostat) in connection with the development of the
European Union’s Harmonised Index of Consumer Prices. The latter acted as a focal point for improving the
compilation of CPIs in the European Union and significantly increased levels of understanding and expertise in
CPI methodology.
David Fenwick was the Editor in Chief of the handbook and accompanying CD‐ROM. John Astin the co‐editor
of the Handbook and Terry Offner the co‐editor of the CD‐ROM. The authors and, in brackets, co‐authors of
the individual chapters are as follows:
vi
PREFACE
1. Introduction David Fenwick and John Astin
2. The Scope of the CPI John Astin (Paul Armknecht)
3. Product Classification Marc Prud’Homme (John Astin)
4. Expenditure Weights in the CPI Marc Prud’Homme (Paul Armknecht)
5. Sampling Procedures David Fenwick and Matthew Powell (Terry Offner)
6. Price Collection Terry Offner (John Astin)
7. Substitution and Quality Change Paul Armknecht (Walter Lane)
8. New Products Paul Armknecht (Matthew Powell)
9.1. Special Cases ‐ Housing David Fenwick (Yoel Finkel, John Astin)
9.2. Special Cases ‐ Own Account Production Keith Blackburn (John Astin)
9.3. Special Cases ‐ Services Walter Lane (Marc Prud’Homme )
9.4. Special Cases ‐ Tariffs Marc Prud’Homme and David Fenwick (Walter Lane)
9.5. Special Cases ‐ Seasonal Products Yoel Finkel and David Fenwick (Marc Prud’Homme)
9.6. Special Cases ‐ Second‐Hand Goods David Fenwick (Paul Armknecht)
10. Index Calculation John Astin and Marc Prud’Homme (David Fenwick)
11. Special Indices John Astin (Marc Prud’Homme)
12. Data Validation Terry Offner (Matthew Powell)
13. Sources of Errors and Bias in a CPI Yoel Finkel (Paul Armknecht)
14. Publication, Presentation, Analysis and Interpretation David Fenwick (John Astin)
15. Organisation and Management David Fenwick and Yoel Finkel
16. User Consultation David Fenwick (Yoel Finkel)
17. CPI/ICP Integration and Harmonisation John Astin (Keith Blackburn)
18. Quality Reporting and improving the CPI: Frameworks, checklists and work programmes. David Fenwick
(George Beelen)
19. Glossary John Astin
The quality of the Handbook was also increased by the valuable contributions of colleagues, including user
input from CPI compilers in a number of national statistical institutes particularly in developing countries.
Special thanks are due to staff at the African Development Bank and the Asian Development Bank, who
provided useful comments, to the UNECE, ILO, the IMF, COMESA, and CEPAL who facilitated consultation
with CPI compilers on earlier drafts, and to the following individuals who provided some helpful comments
which were incorporated in the final text: Nelson Taruvinga; Vincent Musoke Nsubuga; Patrick Kelly; Saad
Bashir; Phillip Miti; Sanjev Bhonoo; Nicholas Nsowah‐Nsowah; Borbala Minary; Marietta Morada and Zakayo
Msokwa.
I would like to thank my co‐editor of the handbook, John Astin, who also drafted or co‐authored a number of
chapters. I am also grateful to Erwin Diewert, who took on the task of reviewing the more technical aspects of
the handbook, including the scrupulous checking of formulae for errors and inconsistency in annotation.
The idea of a CD‐ROM evolved during the early stages of drafting of the handbook and became a project in its
own right. I am indebted to Terry Offner for applying his technical skills in undertaking much of the detailed
work and for acting as my co‐editor on the CD‐ROM, and to Marc Prud’Homme for drafting the PowerPoint
presentations, which are an integral and important part of the materials included in the CD‐ROM.
The production of an international handbook, which relies on contributions from around the world, requires
a great deal of co‐ordination. Ben Whitestone (UK Office for National Statistics) took on the role of project
manager and his efforts were a significant factor in the success of this project.
Finally, thanks go to the UK Department for International Development who funded the drafting of this
Handbook, the UK Office for National Statistics who provided a great deal of day to day support and the
United Nations Economic Commission for Europe (UNECE) for undertaking the printing and distribution.
David Fenwick
Editor in Chief
vii
Contents
Foreword ......................................................................................................................... iii
Preface ............................................................................................................................. v
Introduction.......................................................................................................................... v
Background .......................................................................................................................... v
A guide to readers............................................................................................................... vi
Acknowledgements............................................................................................................. vi
1 Introduction................................................................................................................... 1
Systems of Price Indices and supporting frameworks......................................................... 1
Lower‐level Frameworks...................................................................................................... 2
Choice of index number formula ......................................................................................... 3
Index Formula at Lower (Elementary Aggregate) Level ...................................................... 3
Acquisition, use, or payment approach............................................................................... 3
Contents of the Handbook................................................................................................... 4
Terminology in the Handbook ............................................................................................. 6
2 The Scope of the CPI ...................................................................................................... 8
Introduction ......................................................................................................................... 8
The different uses of a CPI ................................................................................................... 8
Geographical coverage ........................................................................................................ 9
Reference population for the CPI ...................................................................................... 10
Democratic versus plutocratic weights.............................................................................. 11
Product exclusions ............................................................................................................. 11
Imputed transactions and imputed prices......................................................................... 12
3 Product Classification .................................................................................................. 13
Introduction ....................................................................................................................... 13
Classification systems: the general case............................................................................ 13
The CPI classification system ............................................................................................. 14
Specific requirements of a CPI classification ..................................................................... 14
The COICOP classification .................................................................................................. 15
Benefits of adopting the COICOP classification ................................................................. 16
National versions of COICOP at the sub‐class level ........................................................... 16
Incorporating new products in the CPI classification ........................................................ 17
Changing from a national classification to COICOP ........................................................... 17
4 Expenditure Weights in the CPI.................................................................................... 18
Introduction ....................................................................................................................... 18
Background ........................................................................................................................ 18
Conceptual basis of the weights ........................................................................................ 19
Weighting structure ........................................................................................................... 23
Data sources....................................................................................................................... 25
National Accounts .............................................................................................................. 28
Weights reference period .................................................................................................. 29
5 Sampling Procedures ................................................................................................... 38
Introduction ....................................................................................................................... 38
Background ........................................................................................................................ 38
ix
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Overview ............................................................................................................................ 39
Illustrative example of selecting a sample for price collection ......................................... 41
6 Price Collection ............................................................................................................ 48
Introduction ....................................................................................................................... 48
Background ........................................................................................................................ 48
The principles of price collection ....................................................................................... 49
Item specifications ............................................................................................................. 49
The prices to be collected .................................................................................................. 51
Problems sometimes encountered.................................................................................... 56
Frequency and timing of price collection .......................................................................... 56
Practical collection procedures: planning and organisation ............................................. 58
Practical collection procedures: collecting prices in the field ........................................... 59
Training and work instructions .......................................................................................... 62
7 Substitution and Quality Change.................................................................................. 71
Introduction ....................................................................................................................... 71
Background ........................................................................................................................ 71
Substitution procedures .................................................................................................... 72
Temporarily (non‐seasonal) missing products................................................................... 73
Permanently missing products .......................................................................................... 73
Quality adjusting the price for differences in quality ........................................................ 75
Direct comparison.............................................................................................................. 75
Direct or explicit quality adjustment methods .................................................................. 75
Implicit quality adjustment................................................................................................ 77
Matched models ................................................................................................................ 81
Summary ............................................................................................................................ 81
8 New Products .............................................................................................................. 88
Introduction ....................................................................................................................... 88
Definition of New Products................................................................................................ 88
Planning for the introduction of new products ................................................................. 89
Timing of the introduction of new products ..................................................................... 90
Methods of incorporating new products into the index ................................................... 91
Geographical dimension .................................................................................................... 95
Services Previously Provided Free ..................................................................................... 95
9.1 Special Cases ‐ Housing ............................................................................................101
Introduction ..................................................................................................................... 101
Owner‐occupier housing costs ........................................................................................ 101
Rented accommodation................................................................................................... 105
9.2 Special Cases ‐ Own Account Production..................................................................108
Introduction ..................................................................................................................... 108
Background ...................................................................................................................... 108
The own account production dilemma............................................................................ 109
Goods produced for own consumption........................................................................... 110
Construction of a “general consumption” index ............................................................. 111
Construction of a “monetary” index................................................................................ 112
Construction of a “compensation” index ........................................................................ 112
Own account household services (excluding owner‐occupier housing services) ........... 112
9.3 Special Cases ‐ Services ............................................................................................114
x
CONTENTS
xi
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Other special indices........................................................................................................ 169
Illustrative examples ........................................................................................................ 170
12 Data Validation .........................................................................................................176
Introduction ..................................................................................................................... 176
Background ...................................................................................................................... 176
Data validation and editing procedures in the field ........................................................ 177
Data validation at Regional or National Headquarters.................................................... 178
Data editing...................................................................................................................... 178
Non‐statistical checking ................................................................................................... 178
Statistical checking........................................................................................................... 179
Dealing with outliers ........................................................................................................ 181
Missing prices................................................................................................................... 182
Credibility checking.......................................................................................................... 182
Output editing of indices ................................................................................................. 183
Analytical tools................................................................................................................. 183
Prioritisation..................................................................................................................... 184
13 Sources of Errors and Bias in a CPI ............................................................................188
Introduction ..................................................................................................................... 188
Background ...................................................................................................................... 188
Summary of General Measurement Problems ................................................................ 188
Bias in a CPI ...................................................................................................................... 190
Sampling and Non‐sampling Errors ................................................................................. 193
14 Publication, Presentation, Analysis and Interpretation .............................................196
Introduction ..................................................................................................................... 196
Timing of publication ....................................................................................................... 196
Pre‐release access............................................................................................................ 197
Political and policy statements by ministers and their officials ...................................... 198
Procedures on publication day ........................................................................................ 199
Format and content of statistical press notice ................................................................ 199
Standard calculations and special presentations and analysis........................................ 201
Internal purchasing power of national currency ............................................................. 204
Rounding .......................................................................................................................... 205
15 Organisation and Management ................................................................................211
Introduction ..................................................................................................................... 211
Overview of Quality Management System...................................................................... 211
Documentation ................................................................................................................ 212
Internal Audits.................................................................................................................. 214
Review systems................................................................................................................ 216
Business continuity .......................................................................................................... 219
16 User Consultation .....................................................................................................226
Introduction ..................................................................................................................... 226
The general principles of consultation ............................................................................ 226
Special arrangements for consultation on the CPI .......................................................... 227
Who to consult and on what ........................................................................................... 227
Practical options for consultation‐ advisory committees................................................ 228
Advisory committees ‐ operational arrangements and terms of reference ................... 229
Advisory committees ‐ role of the NSI and the National Statistician .............................. 229
xii
CONTENTS
xiii
CHAPTER 1
Introduction
1.1 Consumer price indices measure changes 1.4 In short, consumer price indices are now
over time in the general level of prices of goods and considered as one of the most important economic
services that households acquire, (use or pay for) for and social indicators produced by National Statistical
the purpose of consumption. In many countries they Institutes (NSIs) throughout the world. Against this
were originally introduced to provide a measure of background, the challenge of national statistics
the changes in the living costs faced by workers, so institutes is fourfold: to identify user needs; to
that wage increases could be related to the changing conceptualise user needs in terms of economic
levels of prices. However, over the years, CPIs have theory; to translate the underlying concept into
widened their scope, and nowadays are widely used statistical measurement terms following the
as a macroeconomic indicator of inflation, as a tool fundamental principles of price index measurement;
by governments and central banks for inflation to construct the indices so defined and evaluate
targeting and for monitoring price stability, and as them against purpose.
deflators in the national accounts. With the
globalisation of trade and production and the Systems of Price Indices and
liberalisation of the markets, national governments,
central banks and international organisations place supporting frameworks
great importance on the quality and accuracy of
national CPIs, and in their international Higher‐level frameworks
comparability.
1.5 The System of National Accounts 2008
1.2 Different conceptual frameworks can be used provides the basic guidelines for building
to address fundamental issues relating to nature of appropriate frameworks for CPIs and other price
the index. For example, whether the CPI should indices in the “family” of price indices, covering all
measure the change in cost of a fixed‐weight basket major economic activity including, most particularly,
of goods and services or whether it should measure production and consumption and intermediate
the change in the cost of living, i.e. the cost of outputs. In theory the System of National Accounts
maintaining a given standard of living, taking into also provides a methodology for developing and
account the fact that when prices change consumers extending the core system of price indices to meet
change their expenditure patterns. The use and specialised needs whilst maintaining consistency of
conceptual basis of the index will determine the approach, both in concepts and practical
method of construction, including the formulae used measurement, and coherence in terms of the
for the aggregation of prices. definition, classification and measurement of flows
and stocks of goods and services. The use of such
1.3 The method of construction also allows (or
frameworks also facilitates the examination of the
should allow) CPIs to be adapted for a wide range of
relationships between different price inflation
specific uses. For example, they can be adapted to
measures and can help analysts in their
calculate specific inflation rates for social groups
understanding and interpretation of inflationary
such as pensioner or low‐income households. Their
signals. With their ability to identify statistical gaps,
product coverage can be adapted so as to show
these frameworks can also facilitate future
what the rate of inflation is in particular sectors such
developments in the field of price statistics. One
as energy or food, or excluding particular sectors
such framework is the Stage of Processing
such as alcohol and tobacco. They can shed light on
Framework (see Appendix 1.1).
the effect of tax changes or government‐regulated
price changes on the rate of inflation. They can be Stage of Processing Framework
compiled on a regional basis, showing different
inflation rates within different parts of a country or 1.6 A “Stage of Processing” framework has a
between urban and rural areas. useful role to play in informing policy formation. It
can also provide the basis for systematically
analysing the build‐up of inflationary pressures in
1
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
2
INTRODUCTION
at a macro‐level and one at a micro‐level, and both relate and will span, say, a year rather than a point in
complement one another. Thus: time. This is because the main source of weights
data is a Household Budget Survey (HBS) which
• A COGI measures the inflationary pressures in
typically produces usable results a year or more after
the economy from price developments in the
the end of a survey period which is not a point in
retail sector. It represents one of many sectors,
time. In these circumstances, either the HBS period
albeit an important one, in the Stage of
weights are used without adjustment, or they are
Processing Framework.
price‐updated to the price reference period. Either
• A COLI measures the expenditure required by a of these methods results in a Lowe index, but is
household to maintain their standard of living or often referred to as a “Laspeyres‐type” index. The
utility. Putting aside the definition of utility and CPI Manual (Chapter 1) goes into considerable detail
whether the COLI is unconstrained or on these and other index number types.
constrained, it is an index constructed from the
1.19 The Handbook in general is based on the
viewpoint of the individual consumer and has its
assumption that the index being compiled is a Lowe
foundation in micro‐economics and the theory
(or Laspeyres‐type) index.
of individual consumer behaviour. Unlike a
COGI, it takes into account the substitutions
consumers make when faced by relative Index Formula at Lower (Elementary
changes in prices either between different Aggregate) Level
goods and services or between different outlets
and suppliers. 1.20 The first stage in the calculation of CPIs is the
1.14 The different uses of a CPI set within these calculation of elementary price indices, which are
frameworks determine a number of measurement then aggregated to obtain higher‐level price indices.
issues including the choice of index number formula. Expenditure weights are not usually available below
the elementary aggregate level. The three most
widely known elementary index formulae are the
Choice of index number formula Carli, the Dutot and the Jevons and each is
associated with a number of assumptions which will
1.15 Most countries state that they use a impact on measured inflation. The Carli (a simple
Laspeyres index or a “Laspeyres‐type” index for their arithmetic average of price relatives) and Dutot (the
national CPI which, in practice, is somewhere along ratio of simple arithmetic averages of prices)
the continuum between a cost‐of‐goods index formulae have a number of problems associated
(COGI) and a cost‐of‐living index (COLI). with their use – particularly the Carli, which is
1.16 Experts generally agree that the ideal type of positively discouraged as it is particularly associated
index for a CPI would be a “superlative” index such with some bad characteristics. Because of this the
as the Fisher index. Superlative indices make equal Jevons formula (the ratio of simple geometric
use of the prices and quantities in both of the averages) is increasingly used. It should be noted
periods being compared (such as the base period that an arithmetic average is always greater than or
and the latest month). In reality, quantities for equal to a geometric average and that the difference
recent periods are almost never known, so that in will be greater the greater the variance in the price
practice nearly all NSIs produce CPIs which rely on relatives. The choice of formula becomes more
quantities relating to a base period some time important the greater the diversity of price
earlier. movements which is one argument for ensuring that
elementary aggregates are as homogeneous as
1.17 Some countries aim to produce a “cost of possible.
living” index (COLI). But such an index is in fact a
type of superlative index and suffers from the same
practical defect as mentioned above. Acquisition, use, or payment
1.18 It is important, nevertheless, for national
approach
statistical offices to be able to state publicly what
type of index is being calculated in their CPI. A true 1.21 A CPI is based on the measurement of the
Laspeyres index uses quantity data which relate to change in prices of the goods and services included
exactly the same period as the price reference in the basket. The vast majority of goods (but not
period. This is rarely the case. Most statistical offices necessarily of total values) are priced in the retail
have a price reference period which is later than the outlets selling them. It should be noted that in
period to which the quantity data (i.e. the weights) normal circumstances the prices recorded are the
labelled prices, which are assumed to be the prices
3
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
4
INTRODUCTION
1.28 The following summary of the contents of • Chapter 8 covers the treatment of newly
each chapter draws attention to some of the more available products. It deals with planning for the
crucial links between chapters. introduction of new products; the timing of their
introduction in the CPI; sample rotation; how to
• Chapter 2 on the scope of the CPI, covers such
add a new elementary aggregate; and the
issues as: the different uses of CPIs;
consequential effects on the calculation of the
geographical coverage (including regional
overall index.
coverage); the reference population; the choice
between democratic and plutocratic weights; • Chapter 9 deals with a variety of specific goods
the exclusions of certain products such as and services which need special treatment. They
narcotics; and the treatment of imputed are:
transactions and imputed prices.
o Owner‐occupied housing – the various
• Chapter 3 deals with the subject of product methods of attributing values to the shelter
classification, and focuses almost entirely on the costs of owner‐occupiers.
standard international classification COICOP.
o Own‐account production, with particular
• Chapter 4 is devoted to the important issue of emphasis on home‐produced food.
expenditure weights. There are links to issues
o Certain services, in particular health and
dealt with in other chapters, including:
educational services, where there are
plutocratic and democratic weights; the
usually issues concerned with “free” or
acquisition, use or payments approach to the
highly‐subsidised provision.
recording of prices; the treatment of taxes and
subsidies; the population and geographic o Tariffs – such as those used in utilities like
coverage; the treatment of own‐account public transport, gas, electricity and water
production; the treatment of in‐kind supply, etc.
remuneration; and the coverage of second‐hand
o Seasonal products – methods of dealing
goods. All of these topics are dealt with in
with products (especially fresh food
greater detail in other chapters. Chapter 4
products) which are available in certain
covers in depth the sources of weights data,
months but not in others.
including Household Budget Surveys and
adjustments which may be needed; the weights o Second‐hand goods, such as clothing and
reference period and price‐updating; and the motor cars. The chapter covers situations
use of weights in calculating the overall index. where second‐hand items are traded via
dealers or directly between households, and
• Chapter 5 deals with the complex issue of distinguishes between imported and
sampling. CPIs necessarily rely on the use of domestic products.
samples covering time, location, outlet‐type,
product type, and detailed product • Chapter 10 is devoted to the detailed methods
specifications. All of these aspects of sampling of calculation of CPIs, focussing in turn on the
are dealt with in Chapter 5. compilation of elementary aggregate indices
and then the staged aggregation to the overall
• Chapter 6 on price collection, follows logically CPI. It covers such issues as the stratification of
from Chapter 5. It deals with most of the elementary aggregates; the use of implicit
practical aspects of price collection, including: weights within elementary aggregates; the pros
frequency and timing of collection; period or and cons of various elementary aggregate
point‐in‐time pricing; outlet types; seasonal formulae; the choice of formula for calculating
availability (also dealt with in Chapter 9); price the aggregate indices; and the chain‐linking and
volatility; the management of price collection splicing of indices.
including quality control and documentation;
staff management and training; methods of • Chapter 11 deals with the calculation of variants
recording prices; treatment of special offers and of the standard CPI, including: indices relating to
discounts; bargained prices; central collection. particular socio‐economic groups; regional CPIs;
indices excluding certain product groups such as
• Chapter 7 deals with the problems which occur alcohol and tobacco; constant‐tax indices; and
when products or outlets become unavailable, seasonal adjustment of the overall CPI.
either temporarily or permanently. The chapter
covers the important questions of substitution • Chapter 12 deals with data validation at all
(replacement) and quality adjustment. stages of compilation of a CPI. This includes
checking at the initial input stage; credibility
5
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
6
INTRODUCTION
Appendix 1.1 Stage of Processing Framework
/a from wholesalers/dealers/import agents
/b capital eqpt purchased through wholesalers/agents are not covered by the IIP ‐ which reflects changing levels of output prices (ie
manufacturers list or order prices)
/c there may be direct purchases, but such transactions are not reflected in any published indices
/d The only export prices collected by PPS are from the producers (EPI1s). The EPI2s published by Trade Stats cover all exported goods;
but only the EPIs determined by PPS are based on direct price collection
/e this block is equivalent to the sale of labour by private households to the productive sectors of the economy
/f most capital goods will, by definition, feed into intermediate demand ‐ but capital goods purchased for re‐sale by dealers (eg cars) will
be classified as intermediate demand
/g not yet based on directly‐collected prices ‐ but an estimate of imported capital goods prices is made for the FEPI (and for the PINCCA)
/h expenditure on services incurred as an integral part of the acquisition of capital goods is classified as part of investment expenditure
/i output from the construction sector feeds into Govt final consumption (why?)
/j from wholesale to retail
/k eg domestic help
/l capital goods that feed into Govt Final consumption (eg PCs costing < £1000)
7
CHAPTER 2
The Scope of the CPI
Introduction several CPI variants for specific purposes. Each index
should be properly defined and named to avoid
2.1 Chapter 1 of the CPI Manual provides a confusion and a “headline” CPI measure should be
summary of the background to CPI methodology. explicitly identified. Where only one CPI is published,
Paragraphs 1.8 to 1.12 provide some background on it is the main use that should determine its type and
the origins and use of CPIs, including their evolution scope. If there are several major uses, compromises
from compensation indices to macro‐economic may have to be made with regard to how it is
indices used by government to set inflation targets, constructed. The purpose of a CPI should influence
and paragraphs 1.13 to 1.146 discuss the all aspects of its construction. CPI producers need to
characteristics of the different index formulations know how their index is being used if they are to
available and their different properties, including the ensure that it is fit for purpose. In this connection
tests that can be applied in deciding which user consultation is important. Chapter 16 provides
formulation to adopt in different circumstances. detailed advice on consultation procedures.
Much of the remaining parts of Chapter 1 cover 2.5 This chapter reviews the issues confronted by
issues relating to concepts, the scope and definition the index compiler relating to the scope of the index
of a CPI and a wide range of other subjects and the practical measurement and compilation
confronting the index compiler including the decisions which have to be made. But before doing
acquisition, payments and user‐cost approaches, so it reviews the various uses of a CPI to put the
cost‐of‐living indices, the construction of discussion in context.
expenditure weights and the collection of prices
data. Chapter 2 of the CPI Manual discusses in The different uses of a CPI
greater detail the different uses of a CPI and the
practical consequences for the price statistician.
2.6 CPIs have three main uses:
2.2 The ILO Resolution on CPIs specifically
addresses the scope of the CPI in paragraphs 8 to 14 • Indexation
and makes the point that “the scope of the index A CPI used for wage or contract indexation of
depends on the main use for which it is intended” any specific group, whether of population or of
and “should be defined in terms of the types of products, should represent the coverage of the
households, geographical areas, and the types of group concerned. For instance, it can be argued
consumer goods and services acquired, used or paid that the weights of a CPI used for indexation of
for by the reference population”. pensions should cover only the expenditure of
2.3 A CPI can be used for a variety of purposes, the pensioner population. The product and
the more common ones being: the indexation of outlet list could also be more appropriately
wages, rents, contracts and social security payments; targeted, if the data exist. This means, for
the deflation of household consumption in the example, that a CPI used for indexing pensions
national accounts; as a general macroeconomic may use weights relating to pensioner
indicator, especially for inflation targeting and for households and may exclude products which
setting interest rates. Elements of a CPI are also may be thought largely irrelevant to, say, poorer
often used in the calculation of purchasing power households, such as luxury items or brands.
parities (PPPs) required in the International Similarly, for domestic indexation, the CPI
Comparison Program (ICP) .
1 should cover only the expenditure of the
resident population (see section below on
2.4 Given the many uses of CPIs, it is unlikely that “Geographical coverage”). More generally, it has
one index can perform equally satisfactorily in all to be decided whether the CPI should be, in
applications. Some countries therefore construct principle, a cost‐of‐living index or a pure price
index – these two very different concepts are
1
See Chapter 17.
discussed in Chapter 1.
8
THE SCOPE OF THE CPI
9
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
10
THE SCOPE OF THE CPI
charges just once and the standard convention is to likely to be bought only by the excluded group, nor
record it against the household which incurs the should outlets specialising in such products be
costs. In the above example, the accommodation included in the sample. Conversely, if the wealthy
costs should be included under the expenditure of are included, some “luxury” products and outlets
the family, and not the individual. If individuals should also be included in the sample. For the
spend their own money on clothes and other analysis of national inflation, it is considered that the
incidental expenditure then the Household Budget more comprehensive the CPI the better.
Survey should record this expenditure as being
incurred by the individuals. In reality, many Democratic versus plutocratic
Household Budget Surveys do not cover institutional
households and, where this expenditure is weights
considered to be significant, estimates will need to
be made from, for example, special surveys of 2.20 A “democratic” CPI uses weights which
people living in institutions or by reference to the average unweighted expenditure proportions over
expenditure patterns of similar people, say, the the whole population, rather than dividing total
same sex, age and socio‐economic group, living in aggregated expenditure by the population. The
non‐institutional households. latter method gives more weight to the high‐
spending households, and is referred to as a
2.18 In reality, in considering the practical issues “plutocratic” index. It is argued that a democratic
relating to the inclusion of institutional households index is more suitable for showing the impact of
in a CPI, two questions need to be asked. First, is the inflation on the average household but in reality is
expenditure pattern of institutional residents likely very rarely computed by NSIs. There is a general
to be significantly different from household consensus that a plutocratic index is the appropriate
residents? Secondly, even if the answer is yes, would index to use for national accounts deflation or for a
their exclusion from the CPI be likely to significantly general measure of inflation.
affect the national (or regional) CPI? To answer these
questions, some research should be carried out on a 2.21 Many countries publish a range of CPIs
sample basis. relating to sub‐sectors of the population such as: all
households, low‐income households, pensioner
2.19 Some countries exclude certain household households etc but a CPI based on “democratic”
types from the CPI, such as the very wealthy or the weights is very rare.
very poor. Such exclusions may be on theoretical
grounds (for example, using the argument that the
expenditure of the wealthy, who are relatively few in
Product exclusions
number, should not be allowed to affect a CPI which (Refs: CPI Manual 3.39‐3.73; 3.121‐3.127; ILO
may be used for indexation of wages of ordinary Resolution 13‐14)
workers) or on practical grounds (for example, using
the argument that wealthy households tend to have 2.22 In its role as an indicator of total consumer
low response rates to Household Budget Surveys, inflation, the CPI should in principle cover all types of
and their inclusion can lower the quality of goods and services which are consumed in the
expenditure weights). For a CPI which is used for national retailing market. In practice, some types of
indexation of wages, the exclusion of pensioner and product may be excluded for policy reasons while
wealthy households may be justified on conceptual other exclusions are unavoidable in practice. These
grounds. For example, it may be considered that may include: goods sold illegally, such as narcotics;
such households are likely to spend their money on black market sales; gambling; and prostitution. In
atypical things and including them would distort the most of these cases, there will be no expenditure
relevant overall average. It is also argued by some data from the Household Budget Survey or national
that the inclusion of pensioner households should be accounts and prices are difficult if not impossible to
excluded in principle from an index used for the up‐ collect. Whilst many would consider it inappropriate
rating of state pensions because of the circularity to have wages indexed to a CPI which includes, say,
involved (the level of state pension influences illegal narcotics, it should be noted that the
expenditure patterns which are then used in the up‐ valuation of the totality of the narcotics market, and
rating calculation) whilst others would argue that it indeed, the non‐observed economy in general, is
is logical that indexation should be based on an now a requirement for the SNA as far as GDP
index reflecting the expenditure of pensioner estimates are concerned. Thus, in principle,
households and their specific inflationary estimates of weights and prices will need to be made
experience. Note that if wealthy households are for the purpose of producing deflators, even if the
excluded, the CPI basket should not include products expenditure is not covered in the CPI. Solutions will
11
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
need to be found to the practical measurement a prescribed medicine provided free as part of a
issues. For instance, if a CPI covers gambling, it is not national health service. There is a “transaction” in
the gross stakes which should be included in the the sense that a product changes hands but at
weight, but the net stakes, which is broadly “zero” price so that it doesn’t constitute a monetary
equivalent to the margin taken by the gambling transaction. The conventions for a CPI constructed
operator. As this is not likely to be measurable, one for the purposes of indexation or the measurement
solution may be to distribute the weight for of inflation as a macro‐economic indicator exclude
gambling across other classes in COICOP group 09.4 from coverage this category of non‐monetary
(Recreational and sporting services). transaction, so no price should be imputed.
2.23 The treatment of second‐hand goods is often 2.27 A CPI should measure the prices of final
found to be problematic. As far as transactions consumption by a household. In principle, the first
within the household sector are concerned, sales will category, imputed transactions where households
balance purchases (apart from any dealer charges), do not incur a financial liability but bear the costs of
so the effect on the CPI is close to zero, and they acquiring the good or service in another way, should
may be excluded. But in many developing countries be included in a CPI where used for GDP deflation
there are significant sales of imported second‐hand and, in principle, can be included in a CPI compiled
goods, such as cars and clothing. Where sales of for other purposes. Perhaps the most important
imported second hand goods are significant relative example is the consumption of own‐produced
to sales of new goods of the same product, such products such as food. Here, there is no actual
sales should be included, both in weights and prices. transaction at all, and thus no price. If the
“transaction” is to be valued a price has to be
2.24 CPI compilers sometimes face proposals from
imputed. This would usually be done by reference to
governments or pressure groups to exclude certain
actual purchases of the same product in, say, nearby
categories of product for non‐statistical reasons. A
markets. But even when this is done for the purpose
common example is alcohol where in some countries
of estimating GDP (which values consumption
its consumption is associated with social stigma or it
regardless of its market mechanism) it is not
can only be purchased illegally. While it is acceptable
necessarily appropriate to include it in the general
to produce a variant of the general CPI excluding
CPI or in a CPI used for indexation where the
such products, the main CPI should include them,
narrowest concept of consumption that can be used
where practical, to ensure that the index presents a
is one based on monetary expenditures only. Both
true and accurate picture of national inflation.
the CPI Manual and the ILO Resolution leave this
2.25 The CPI Manual and the appropriate sections latter question open, but, from the point of view of
of this Handbook cover in some depth the treatment measuring inflation and also for the purposes of
of other excluded or partly‐excluded products, income indexation, the most common view is that it
including: taxes and licenses, subscriptions, is best to omit it on pragmatic grounds although
insurance, gambling, financial transactions, hire goods and services purchased by households which
purchase and interest payments. are then used as inputs into own account production
are normally treated as if they themselves were
Imputed transactions and imputed consumption goods and services, and should
therefore be included in the CPI. Some countries
prices may find it useful to produce two versions of the CPI:
(Refs: CPI Manual 3.74‐3.89; 3.130‐3.132; ILO one including and the other excluding own‐account
Resolution 17‐18) consumption.
2.26 A distinction can usefully be made between 2.28 The treatment of owner‐occupied housing,
imputed transactions and actual transactions where which involves the own consumption of housing
a price is imputed. In the second category would be services, is dealt with separately in Chapter 9.
12
CHAPTER 3
Product Classification
3.1 Paragraphs 3.144 to 3.168 of the CPI Manual, those that have not adopted COICOP. First, the
deal with the subject of classification systems. The chapter may help in understanding more fully the
paragraphs list the criteria for establishing a relevant advantages of COICOP and provide some guidance
classification system for a CPI. These are that: the for moving to a COICOP structure if this is a future
classification must reflect economic reality: meet the objective. Second, some of the issues that are raised
needs of users of CPI sub‐indices; be unambiguously in this chapter will be relevant for many aspects of
mutually exclusive and exhaustive (that is, providing CPI classification regardless of the one actually used.
full coverage). The paragraphs then go on to review
3.7 COICOP was first developed for the System of
the different classification schemes available to the
National Accounts (SNA 1993) to provide the
CPI compiler and describe in some detail the
structure for classifying household consumption
international standard classification of individual
expenditure. Expenditures on the various
consumption expenditures – COICOP (Classification
components of household consumption are often
of Individual Consumption according to Purpose).
used as the basis for the weights in the CPI (see
Paragraphs 20 to 22 of the ILO Resolution reiterate
Chapter 10). The 2003 ILO Resolution on CPIs
the point about addressing the needs of users of
requires that national CPI classifications should be
sub‐indices and go on to state that for international
reconcilable with COICOP at least at its higher
comparisons the classification used for a CPI should
aggregation levels. Many countries have adopted
be compatible with COICOP at least at its division
COICOP in their economic statistics (for example, in
level.
the CPI, national accounts, International Comparison
Programme (ICP), and household budget surveys),
Introduction with a clear advantage for integration of data‐sets
and enhanced analytical capabilities.
3.2 The importance of the product classification
system cannot be overstated. Choosing a
Classification systems: the general
classification system is the first step in compiling the
CPI because its sub‐aggregates must be defined in case
such a way that the expenditure weights and prices
will relate precisely to the coverage of the sub‐ 3.8 In its broadest sense a classification is a
aggregates. procedure in which individual items are organised
into categories based on information on one or more
3.3 The classification is important also because it characteristics inherent to the items. A classification
establishes a framework from whose boundaries the scheme will usually have these same items (or
representative items for inclusion in the index (and elements) arranged in a hierarchical ordered system
sometimes the outlets) will be defined and drawn. based on category‐subcategory relationships where
3.4 Finally, the classification system helps in the subcategory has the same description as the
defining which level of the hierarchy will be suitable associated class in addition to one or more
for publication. descriptions. For example, an apple is a subclass of
fruit. So any apple is a fruit, but not every fruit is an
3.5 Classification is a central theme in the
apple. A product needs to have a more detailed
compilation of the CPI. In years past, countries used
description to be an “apple” than to be a “fruit”.
their own distinct systems for classifying the range of
products covered by their CPI. Most countries have 3.9 In principle, a classification system can be
now, however, moved to the international standard based on any attribute of the objects being
classification COICOP – see above. This chapter will classified. Normally, organising a population of items
therefore draw upon COICOP unless otherwise into categories must leave no two categories with
noted. any item in common, in other words the categories
must be mutually exclusive. Also, the categories
3.6 It should be emphasised that the chapter
must collectively include all of the items which are in
should benefit CPI compilers in two ways ‐ even for
13
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
14
PRODUCT CLASSIFICATION
15
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
categorised as a service because a significant portion National versions of COICOP at the
of its value added comprises the preparation of the
meal and other services. Similarly there are classes sub‐class level
which contain either both non‐durable and semi‐
durable goods or both semi‐durable and durable 3.27 The most detailed level of COICOP is the Class
goods. Again, such classes are assigned an (ND), (SD) level (4 digits). This is almost always too broad for
or (D) according to which type of good is considered CPI sampling purposes, which require not only well‐
to be the most important. specified items for price collection but also the
grouping of products for which weights are not
3.22 COICOP fulfils the essential requirements of a normally available into elementary aggregates. The
CPI classification as described in paragraphs 3.162 to published list of products covered by each COICOP
3.168 of the CPI Manual. The basic structure reflects Class level should provide sufficient information to
current patterns of consumption and, given that it enable the CPI compiler to create sub‐classes and
allows countries to add further detail at sub‐class “micro‐classes (groupings below the sub‐class level),
level, it fulfils the requirement for reflecting market which are relevant to the products available in the
realities and for providing the facility to incorporate national market. In defining the sub‐classes and
new or local products. The hierarchic structure can micro‐classes, CPI compilers should refer to other
be used to define the levels of sub‐indices which are 2 3
international classifications such as CPA , CPC and
to be published. Moreover, the use of the S, ND, and 4
ISIC , which may suggest useful groupings at these
D classifications permits the presentation of results detailed levels – useful in the sense that analysts
in ways useful for analysts. may be able to use the data in conjunction with
other economic data based on similar classifications.
Benefits of adopting the COICOP But these are not designed for the purpose of a CPI
classification and as a result the classifications do not fit neatly
into patterns of consumer expenditure. More
detailed groupings may also be useful for sampling
3.23 COICOP has become the de facto
purposes in the compilation of the CPI.
international standard for CPI classifications, in line
with the requirement of SNA 2008 to use COICOP in 3.28 The following principles should be used for
the national accounts. For purposes of international the purpose of defining a sub‐class independent of
comparisons of inflation at any level below the all‐ the classification system.
items CPI, a standard classification is necessary, and
• The sub‐class should have a clear and
it is for this reason that the ILO resolution
economically meaningful content.
recommends the use of COICOP, particularly for
international comparability. At the national level, • It should account for a significant share of the
countries need to make comparisons of their own CPI basket.
inflation performance at different levels of detail,
and to do this accurately their own classification • Taken individually, the calculated price index at
needs to be comparable with those in the the sub‐class level should generate analytical
comparison countries. value and interest to the various types of user.
16
PRODUCT CLASSIFICATION
17
CHAPTER 4
Expenditure Weights in the CPI
Introduction Background
4.1 A CPI measures changes in the cost of a 4.4 Expenditure weights are used to combine
representative basket of goods and services. This elementary price indices to derive higher level
involves weighting together aggregated prices for aggregations up to and including the All Items CPI.
different categories of goods and services so that
4.5 The three most common ways of expressing
each takes an appropriate share to reflect the
the weights are: as shares (or fractions) that sum to
budgets of the household covered by the index. For
unity; as percentages summing to 100; as per mille
instance, if most people spend far more on fresh
numbers summing to 1000.
vegetables than on electricity then a price rise for
fresh vegetables must have more effect on overall 4.6 The use of expenditure weights is consistent
price rises than a similar‐sized increase for in concept with a CPI based on the acquisition,
electricity. At the lowest level therefore, each payment and user cost approaches although the
elementary aggregate (see Chapter 10) should treatment of major durable goods and housing can
receive a weight equal to the ratio of expenditure by present a problem, particularly the housing costs of
index households on goods and services represented owner‐occupiers. The use of expenditure weights in
by that aggregate to all expenditure by index a CPI is often referred to as a CPI based on the
households on items within the scope of the CPI. concept of plutocratic weights and being based on
total expenditure this concept gives more weight
4.2 Chapter 4 of the CPI Manual discusses the
implicitly to the expenditure patterns of high‐
derivation and sources of the expenditure weights.
spending households (which will also tend to be
More detailed guidance on specific issues is also
those with higher incomes).
given, including:
4.7 The goods and services consumed by the
• Data sources (paragraphs 1.191 to 1.199)
households can in principle be acquired in six ways:
• Geographical coverage (paragraphs 3.97 to
3.104) • Purchase in monetary transactions.
• Price updating (paragraphs 1.17 to 1.20, 1.28 to • From own production
1.29, and 9.95 to 9.104)
• As payment in kind
• Seasonal products (paragraphs 4.63 to 4.66).
• Social transfers in kind
4.3 Paragraph 19 of the 2003 ILO Resolution on
CPIs makes the obvious but important point that the • Barter
weights follow directly from the scope of the index
as well as from the choice between the • As transfers or gifts from other economic units
“acquisition”, “use” or “payment” approach and,
4.8 The weights themselves are determined by
paragraph 23, that there are two basic sources of
the scope of the CPI and should be derived on basis
information: Household Budget Surveys and National
of the relevant coverage and types of consumption
Accounts. It also mentions, in paragraph 25, the
and with reference to SNA concepts. The broadest
possible use of price‐updating where the weight
possible scope for goods and services would cover all
reference period falls significantly behind the price
six of the above categories. It would include all social
reference period and, in paragraph 26, that weights
transfers in kind in the form of education, health and
should be reviewed at least every five years.
housing and other goods or services provided free of
charge or at nominal prices. The total acquisition of
goods and services thus described is equivalent to
total actual consumption of households in the SNA.
18
EXPENDITURE WEIGHTS IN THE CPI
For the CPI as a general measure of inflation the 4.13 The conceptual issues relating to the
more relevant would be to include only goods and construction of weights are discussed in more detail
services purchased in monetary transactions by the below.
households. The latter excludes the second, third
and forth categories. Only monetary expenditure Conceptual basis of the weights
generates prices that can be observed for the CPI but
this then leaves outstanding the issue of owner‐
occupier housing which is considered in detail in
Plutocratic or democratic weights?
Chapter 9.1. 4.14 This use of aggregated expenditure by index
households to derive weights reflects the principle
4.9 It is against this background, that a CPI often
that each index household contributes to the
follows the concept of “household final monetary
weights an amount proportional to its expenditure.
consumption expenditure” (HFMCE), the approach
This is sometimes referred to as plutocratic
often recommended for a CPI being used as a macro‐
weighting and means that the expenditure patterns
economic indicator, restricted to the appropriate
of high‐spending households (which of course tend
reference population, or “Index Households”, where
to be those with higher incomes) have more
the CPI is being used as a compensation index. The
influence on the index. The use of plutocratic
latter might, for example, exclude the very rich5.
weights is generally considered more appropriate
4.10 But it should be noted that the SNA operates particularly for consumer price indices which have
with many delineations of consumption. One of been constructed to be a general macroeconomic
them is “household final consumption expenditure” indicator. In principle, it is possible to derive
(HFCE) – which includes non‐monetary consumption democratic weights, where each household is given
(such as for owner‐occupier housing and equal weight, but these are not considered
consumption of own production (food)). HFMCE is appropriate for an index used as a general measure
certainly a very useful concept but many countries of inflation. They are better suited to CPIs which are
prefer to also include some non‐monetary intended to reflect the inflation experience of the
expenditure (in particular owner‐occupier housing) typical household. Even so they are very rarely
in their CPI, sometimes using imputed costs thus computed.
moving the coverage of the index closer to HFCE.
Acquisition, use or payment approach?
4.11 Following the HFMCE approach, the coverage
will be consumption expenditure that is incurred in 4.15 The acquisition approach relates to when the
the reference period: good or service is acquired irrespective of when it is
actually used or consumed. The total value of all
• by households irrespective of nationality or goods and services delivered during a given period,
residence status, and whether or not they were wholly paid for or used
during the period, is taken into account in the
• in monetary transactions6, and
weights. A CPI based on this approach measures the
• in the economic territory of the country, and change in the cost of acquiring a product. The use
cost (or consumption) approach relates to the period
• on goods and services for personal consumption over which the product is consumed or used; a CPI
by individuals or households. based on this approach measures the change in the
4.12 The third bullet point is a reference to the cost of using the product over time; in other words
domestic concept. The alternative ‐ the National the cost of the good is distributed over its useful life
concept ‐ where a CPI covers the non‐business The payment approach relates to the period of time
expenditure of the resident population of the when the actual period‐to‐period payments for the
country, regardless of where the expenditure takes product are made. This can differ from the period
place, is appropriate when a CPI is being used for when it is acquired and when it is used. A CPI based
indexation of incomes. on this approach will measure the change in the cost
of purchasing a commodity over time. The use of
5
Note that expenditure weights derived from unadjusted
expenditure weights is consistent in concept with all
Household Budget Survey results, unless re‐weighted for variable three approaches, although it raises a number of
response rates, will tend to under‐record the expenditure of the conceptual and practical issues for the use approach
wealthier households whose response rates are usually low. particularly in connection with the time dimension.
6
As already mentioned, it can be argued that according to
National Accounts principles own‐account production should be
See Chapter 2 for further detail.
included. Although in practice this will depend on the purpose of
the index and the ability to overcome practical measurement
4.16 In practice, the choice between the
issues. acquisition, use or payment approaches is an issue
19
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
relating to durable goods and its practical impact is nature of the pricing strategies applied to these
likely to be limited to the weight given to owner‐ products.
occupier housing costs. The latter is discussed in
4.20 Other countries may exclude the expenditure
Section 9.1 of the Handbook. In countries where
of the very poor. It is argued that by including the
food expenditures and other expenditures on non‐
high‐income socio‐economic group the relevance of
durables, semi‐durables and even services account
the CPI as an indicator of the inflation experience of
for a significant share of the CPI basket and where
the majority of the population could be put in
credit financing is rarely if ever used, the acquisition,
question. It is further argued that as a consequence,
use and payment approaches will give very similar
if the main purpose of the index is that it should be
results and hence the CPI can satisfy many uses
used as a compensation tool, then the compiler
equally well. This is the principal reason why most
should aim for a CPI that excludes the households
countries use, either implicitly or explicitly, the
from the extremes of the income scale. This brings
acquisitions approach to define what constitutes
the CPI closer to the concept of democratic weights
consumption expenditure.
and is based on the idea that households should be
Taxes and subsidies compensated on the basis of the inflation experience
of a “typical” household. Of course, it is open to
4.17 Given that the prices collected for the CPI are countries to compile supplementary CPIs aimed at
final prices paid by the purchaser, i.e. inclusive of the measuring the inflation experience of different
impact of all taxes and subsidies, the expenditure segments of the population. Many would argue that
weights should also be inclusive of the impact of if a certain subgroup of the population is to be
taxes and subsidies. Thus the weights should be compensated for increases in the cost living, then in
based on the price paid after any general subsidies principle, expenditure weights should be
by government, for instance grants given to constructed for this subgroup accepting that this re‐
operators to subsidise public transport fares, are enforces the continuation of the current expenditure
deducted, and after any taxes such as VAT, are patterns of the sub‐group. Thus, it is sometimes the
added, i.e. the expenditure weights are based on practice to create a special CPI which relates only to
actual purchase price paid by the consumer. the poorest households. Such an index is sometimes
Payments made to particular households to help referred to as a “subsistence” CPI. A common
fund purchases, in the form of, say, additional social practice is also to compile a CPI which excludes both
security payments, but which are not explicitly extremes of the income distribution.
related to particular transactions or purchases
should not be reflected in the CPI. These transfers Geographical scope
are treated as a form of income for CPI purposes.
4.21 The geographical coverage may follow one of
4.18 This chapter makes the working assumption two alternatives:
that the CPI weights are calculated on the basis of
the acquisition approach using plutocratic weights • The “domestic” concept where the scope of the
based on expenditure inclusive of taxes and CPI (in terms of both prices and weights)
subsidies. extends to the whole economic territory and
includes the consumption of foreign residents
Population coverage and visitors. This is most appropriate where the
main interest is in the price changes within a
4.19 The target or reference populations will be country. In reality the expenditure of foreign
defined depending on the main purpose and use of nationals within the domestic territory may be
the index. For example, in some countries the difficult to measure since not all foreign
wealthiest households are excluded because their nationals will be covered by household budget
expenditure may be atypical or the Household surveys. Foreign resident nationals should be
Budget Survey information may be unreliable covered in the normal HBS, while special surveys
because of low response. Thus, some of the products of foreign (non‐business) visitors (such as
that are consumed exclusively by these households International Passenger Surveys at borders and
can be costly to collect given the small number of airports) are usually required to cover the
transactions involved. Air fares and packaged holiday expenditure of overseas tourists.
tours are a good example, where in many countries
only a small number of households actually purchase • The “national” concept where non‐residents are
these services and for which accurate measurement excluded from the CPI weights but the
of price change can be costly because of the complex consumption of nationals abroad ‐ for example
while on holiday ‐ is included. This is particularly
appropriate when the CPI is used for indexation
20
EXPENDITURE WEIGHTS IN THE CPI
of the incomes and benefits of the national such as heating and lighting are used partly to
population. Household Budget Surveys are operate the business and partly for household
generally capable of identifying nearly all consumption. For example, take the case of a
relevant expenditure although it is unlikely that subsistence farmer who receives a monthly
the corresponding prices for goods and services statement from the electric company; part of the
purchased abroad will be measurable. It may be invoiced amount was used to light the stable or
possible to use sub‐indices from the “partner” operate the machinery while the remainder of the
countries’ CPIs (adjusted for changes in invoice reflects the expenditures incurred for lighting
exchange rates). the house and operating home appliances. Only the
portion that is used for household consumption
4.22 A uniform approach to geographical scope is
should be included in the CPI weights.
particularly important where harmonized indices are
being compiled for like‐for‐like international 4.27 Where household activities overlap with
comparisons. In countries where the aim is to business activities, another complicating issue occurs
produce a harmonized price index among with the purchase of some durable goods such as
neighbouring countries it is important that they all cars. In some households the same car can be used
use the same concept to avoid the risk of as private final consumption or can be considered as
measurement gaps or double counting. gross fixed capital formation (i.e. part of business
expenditure). This would be the case when a car is
4.23 The same principles apply when regional
used to provide a taxi service and is also used as the
indices are being compiled within a country.
household’s main (or secondary) mode of transport.
Institutional Households In this case, and assuming that the acquisitions
approach to consumption is used for defining
4.24 Institutional households refer to people living consumption for the CPI, the expenditure for the
permanently in an institution or who may be automobile should be allocated between gross fixed
expected to reside in an institution for a very long capital formation and household consumption on a
time, such as: pro rata basis. In other words, if the household
estimates that the car is used 60 percent of the time
• Members of religious orders living in
for business purposes, then the weight used for the
monasteries or convents.
CPI should be based on the balance (40 percent) of
• Long‐term patients in hospitals. the purchase price of the automobile.
• Prisoners serving a long sentence. Own‐account production
• The elderly living in retirement homes. 4.28 Own‐account production is treated by some
countries as within the scope of the CPI, and in other
• Military personnel living in barracks. countries as out of scope. It can be argued that while
• Temporary construction workers living in special it is definitely part of GDP (which values
quarters. consumption regardless of its market mechanism)
and should therefore be included to achieve
4.25 Such people are treated as belonging to an consistency with national accounts and for the
institutional household when they have little or no purpose of producing deflators, it is not necessarily
autonomy of action or decision in economic matters. appropriate for a general CPI or a CPI used for
Neither the CPI Manual nor the ILO Resolution is indexation where the narrowest concept of
prescriptive about whether a CPI should cover the consumption that can be used is based on monetary
expenditure of people living in institutional expenditure. Own‐account production can account
households. Many countries exclude such for a significant portion of a country’s household
expenditure in their CPI because of the difficulty of consumption. This issue is discussed at length in the
obtaining reliable expenditure information, or CPI Manual (paragraphs 3.74 to 3.89). In some
because the expenditure associated with such developing countries over 90 percent of households
households is unlikely to be very significant in report some own‐account production, most
comparison with non‐institutional households. frequently with respect to food. Own account
production is covered in Chapter 9 of this Handbook.
Business expenditure
4.26 Business related expenditures are explicitly
In‐kind remuneration
excluded from the scope of a CPI. Households may 4.29 If the CPI follows national accounts principles,
engage in business activities from their home in‐kind remuneration from an employer (the regular
whereby part of the expenditures for some products
21
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Figure 4.1 Aggregation Procedure
Raw Data
Central Shop Weights
Elementary aggregate
(stratum) indices
Stratum Weights
Product or item indices
Product or item Weights
Section indices
Section Weights
All items index
provision of goods or services in exchange for labour a compensation index, i.e. an index purporting to be
services rendered, often operating as part of a a cost‐of living index, that the price they pay should,
contract of employment) is part of the compensation as a matter of principle, be reflected in the CPI as
of employees and should be viewed as out‐of‐scope this, to them, is the purchase price. A subsidy to
of the CPI. But there are different interpretations employees is just one of many ways in which the
and choices clearly depend on the use of the index. price of a commodity can vary. It is, in effect, a form
One interpretation, for example, is that all subsidised of discount offered to a special group of consumers.
travel should be excluded from the CPI on principle. And, just like other discounts which may vary both in
Another interpretation is that the value of free travel terms of the groups to which they apply and in terms
for, for example, railway workers should not be of level, the terms of subsidies to employees may
included in the CPI. But where fares are partially vary over time. However, as already indicated, it is
subsidised by the employer the CPI expenditure more debatable whether these changes in the terms
weights should reflect that element of the fare paid of the discount should be reflected in the CPI. Some
by the employee in the base period and the price argue that the changes in terms should not be
index should only reflect changes in the fare tariff reflected in the CPI and that where fares are partially
and not changes in the level of subsidy arising from subsidised by the employer the CPI expenditure
changes in the employee’s remuneration package, weights should reflect that element of the fare paid
this interpretation is contrary to SNA by the employee in the base period and the price
recommendations. index should only reflect changes in the fare tariff
and not changes in the level of subsidy arising from
4.30 As indicated, complications make operation
changes in the employees remuneration package.
of the above rule far from clear cut. For example,
Given the conceptual issues and the measurement
take again the case of subsidised travel for railway
problems, subsidised fares are often excluded
workers. It is argued that the value of free or
because of the measurement difficulties which arise.
subsidised travel should not be reflected in the CPI.
Exclusion is not necessarily a matter of principle.
But if persons within the CPI population group
receive subsidised travel for the satisfaction of their needs of the employer – e.g. a tradesman is expected to
own personal wants7, it can be argued, especially for travel directly to a building site or to the location of his first
service call ‐ and the cost is met by the employee, the tax
office will often allow the travel expense as a work
7
Two forms of 'free or subsidised travel' for transport employees deduction. By extension, if an airline chooses to deploy staff
should be distinguished: in a different city to the one in which they live, and the
• Operational travel ‐ when the primary reason for the travel airline provides that employee with free or subsidised daily
is the operational convenience of the employer, the cost of transport, such travel should fall outside the scope of the CPI
the travel is attributable to the operation of the enterprise because it s not household consumption but is, in effect, an
and is conceptually out of scope of the CPI. The tax intermediate consumption of the airline.
treatment of expenditures may help distinguish personal • Personal travel ‐ when the primary purpose of the travel is
and work expenses. For instance, in some countries for tax for the satisfaction of the wants of the employee (for
purposes a daily commute is in most cases regarded as a example, if the employee is involved in recreational travel or
personal expense of the employee and is within scope. a regular commute). Such travel is conceptually within the
However, if the travel can be related to the operational scope of the CPI, even if it is heavily subsidised (or free).
22
EXPENDITURE WEIGHTS IN THE CPI
4.31 A good or service given without reciprocal markedly by region or type of outlet and in
services being rendered, referred to in the national these cases stratification will improve the
accounts as a transfer, is out‐of‐scope of a CPI as accuracy of item indices. For example, each
there is no transaction or payment involved. locally collected item in the index could be
allocated to one of the different stratum types
Second‐hand goods to allow the best available information about
4.32 In principle second‐hand goods are in scope. purchasing patterns to be incorporated in the
A net approach (expenditures minus receipts from index calculation. Depending on the structure of
sales) is normally used to estimate weights. Most the retail market, the stratum types could be:
transactions are limited to the household sector and region and shop type; region only; shop type
typically cancel out (sales of second‐hand goods are only; and no stratification. The assignment of
negative expenditures) unless the transaction is stratum type will depend on the information
through a dealer (e.g. used car dealer) in which case available for constructing the weights for each
the difference would be the dealer’s margin and item and the number of prices collected per
represents the price charged for providing the item. In principle, all locally collected items
service of selling second‐hand goods. A significant would be stratified by both region and shop
number of countries import second‐hand goods and type, but if the weights data are unreliable or
the expenditure on these should be included in the non‐existent, then the item may be allocated to
CPI. Chapter 9 of this Handbook gives more detail. another stratum type. Allocation also partly
depends on which shop types are specified for
Weighting structure the collection of prices and the number of prices
collected. If the rules for the choice of outlets
4.33 The weighting structure should follow the did not specify that both a multiple and an
aggregation structure of the CPI. For instance, if the independent should be chosen for an item,
latter is based on The Classification of Individual there may be too few prices collected in one of
Consumption According to Purpose (COICOP) then these shop types to make stratification by shop
this is the structure which should be used for the type meaningful. In some instances, there may
weights. be no stratification because research has shown
that stratification has little effect.
4.34 Additional sub‐divisions can be introduced
where there is further stratification of the sample to The weight of an elementary aggregate, i.e. the
include geographical location, outlet type or a more stratum weight, should reflect the expenditure
detailed product level classification. Thus the on the entire elementary aggregate and not the
weighting structure will depend on the sample weights of the outlets and items that have been
design for price collection and compilation and in chosen to represent it. Therefore, if spaghetti is
particularly the need for more detailed weights chosen as the representative product under the
which may be generated by additional sample elementary aggregate with the heading of pasta
stratification. In general, national statistics institutes products, then the weight of this category
will collect some prices centrally and adopt up to should reflect expenditures for all pasta
four levels of sampling stratification for local price products and not solely the “lower” weight of
collection: locations; outlets within locations; items spaghetti, i.e. the weight of the pasta category
within different sections of expenditure; and product will be represented entirely by spaghetti.
varieties. The latter is normally conducted in the Similarly, if an expenditure category is divided
field by price collectors and does not normally into two elementary aggregates according to
involve explicit weights (see Chapter 6). Stratification outlet type, say, open markets and
is frequently used to increase sampling efficiency, supermarkets, with corresponding market
especially where the retail market is heterogeneous shares of food sales, 70 percent and 30 percent
(see chapter 5), and also operational efficiency. respectively, then the same rule as above should
Figure 4.1 illustrates the case where there are: apply. For instance, suppose a single store is
selected as the representative outlet for a
• Central shops weights to represent a small
particular food item sold in supermarkets in a
number of large super‐market chains or chain
country where two supermarket chains have
stores which have uniform prices across
equal sales, then the sales from the sampled
branches and prices are provided by the shops’
store will account for the total value of the
Head Offices.
expenditure weight of 30 percent; the weight of
• Stratum weights. For some types of the elementary aggregate for this food item sold
expenditure, purchasing patterns may differ in supermarkets should not be 15 percent (0.30
23
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
x 0.50) i.e. a weight based only on the sales of 4.36 The sample of price observations based on
the selected supermarket. implicit weights can be updated independently and
more frequently than the weights of the elementary
• Product or item weights (in the current context aggregate but the price statistician will need to
the terms can be interchangeable). Some ensure that the weights are both coherent and
products or items may be intended only to consistent within the elementary aggregate. It is best
represent themselves; others represent a to review them at the time of updating the basket.
subclass of expenditure within a section. For
instance, within electrical appliances, an electric Weights for products for which prices are
cooker may represent only itself and not any not collected
other kinds of electrical appliances. However,
other products or items will represent price 4.37 As it is not feasible to collect a full set of
changes for a set of products or items, which are prices from every outlet, including market stalls and
not all priced, so for these the weight reflects street vendors, and from every provider of a service,
total expenditure on all products or items in the most prices have to be sampled. This means that in
set. For example, a screwdriver may be one of practice there will be some products which
several items representing all spending on small consumers spend money on for which prices will not
tools within home improvement and be collected (see Chapter 5). But the expenditures
maintenance materials, and there are other for these products need to be included in the
items within the section representing all expenditure weights. There are two ways of doing
spending on paint, timber, fittings and so on. this:
Some items, such as fresh fruit and vegetables,
• Including the weight in a related elementary
have “seasonal” weights that vary over the year.
aggregate (this may involve the creation of a
These are covered in Chapter 9 of this
“miscellaneous” category). But note that it is
Handbook.
desirable to make the elementary aggregates as
• Section weights. It is common practice to give homogeneous as possible. The price movement
each section an integer weight in parts per of the other products within the elementary
thousand or per hundred so that the sum of the aggregate become the source for the imputation
section weights is 1000 or 100. It is likely that of prices for the non‐priced product. For
most of these weights will be based on the HBS example, assume an elementary aggregate
results. The main exceptions will be some called “milk and cheese” and that it has been
housing sections, including (where applicable) decided that because of the small share of
mortgage interest payments and depreciation, cheese consumption in the country no prices
where weights are estimated from other will be collected for this product. Then the
sources (again see Chapter 9), and for certain movement for the elementary aggregate will be
other sections (tobacco, confectionery, soft determined solely by any observed price
drinks and alcoholic drinks) where the HBS may changes for the products from which prices are
be thought to under‐record expenditure and actually collected, in this case milk.
better data are available elsewhere.
• By having the weight of the product for which
Implicit weights within elementary no representative prices exist equal to zero.
Using the previous example, the weight of the
aggregates
“milk and cheese” index will be reduced by the
4.35 An un‐weighted formula (e.g. Jevons or Dutot value of the weight of cheese. Consequently, its
– see Chapter 10) is usually used when aggregating weight will be implicitly reallocated to the
the individual price relatives of the sample of weights of the other elementary aggregates in
products at the elementary aggregate level. This the CPI. In effect cheese prices are being
practice is usually justified on the grounds that the imputed by the movement in the price of milk
required information such as market shares is simply and of the prices in the remaining elementary
not available to a sufficient level of precision. indices
However, if broad‐based estimates of market shares
4.38 In general, the prices for the product for
are available, then these can be used as implicit
which prices are not collected will be expected to
weights for determining the sample of price
exhibit a similar movement to the other products in
observations to enhance the accuracy of the
the elementary aggregate and the first of the above
elementary price index. Some possible sources for
two methods should be used. The second method
this information are trade publications, market
may be used where the elementary aggregate is
reports, and consultation with industry experts.
24
EXPENDITURE WEIGHTS IN THE CPI
heterogeneous or the associated price index is not acceptable statistical quality is based on the
considered very reliable. Because of the negligible coefficient of variation (the ratio of the standard
size of the weight value involved, the consequence deviation to the mean).
on the overall index will also be negligible whichever
4.43 For those expenditure weights that are
method is used.
unable to meet the minimum requirement of
reliability, three options should be considered:
Data sources
• Combine the elementary aggregate that has the
4.39 This section provides an overview of the main problem weight with another related one to
data sources for weights with a discussion about form a new broader elementary aggregate (e.g.
some of their advantages and limitations. It should “white bread” could be combined with “brown
be emphasised that various data sources can often bread” to form a new category called “bread)”.
complement each other in establishing more reliable This approach will often lead to a more reliable
and accurate overall weights. It is in fact often elementary aggregate but may require an
necessary to take advantage of other sources when adjustment to the existing structure of the CPI.
the data from the main source are incomplete, lack
• If annual HBS data are available, then
the required level of detail or are of questionable
expenditures could be averaged over more than
reliability.
one year thus improving the statistical reliability
4.40 Depending on the population coverage of the data, in terms of standard errors, but to
weights for a CPI are derived either from the detriment of timeliness. It should also be
expenditure data, based on estimates drawn from a noted that averaging may not improve the
sample covered in the Household Budget survey or statistical quality of the expenditure estimates if
from national accounts estimates of household a particular category of household expenditure
consumption expenditure. It should be emphasised, is rapidly growing or declining. Averaging is
however, that expenditure estimates in the national useful if the particular expenditure category
accounts are themselves partially based on HBS under consideration shows a lot of variability
information, although they may differ in terms of over several budget surveys but no clear trend.
coverage, so the two sources are not entirely This is an area where the statistician will have to
independent. Note also that national accounts data use his or her judgement. The “price‐updating”
may also be used when the HBS is conducted too of weights, described later on in this chapter, is
infrequently to ensure the reliability of the CPI or sometimes used to overcome the latter
when the expenditure weights need to be updated difficulty. Using multi‐year expenditure data can
more often than the periodicity of the HBS. also reduce the risk of having unrepresentative
Nevertheless, an HBS will still have to be eventually weights for the base period. Average
conducted because it is an important source for expenditures over a span of three years, for
benchmarking the components of personal instance, will usually yield representative
consumption expenditures of the national accounts. estimates of the basket shares for the base
Other sources for the weights are also available and period but again, the compiler has to consider
are usually complementary to these two main the dynamic parts of the market where there
sources. These include tax revenue returns and retail are rapid increases or declines in the use of the
production and sales data. These are discussed later expenditure category. If resource availability is
in this chapter. an issue, then a less ambitious household
budget survey which generates multi‐year
4.41 When various sources of information are used weights only for one category of commodities,
for generating the weights in a CPI, the compiler such as for the food category, may be
should take the time to check the data to ensure considered. The basket reference period should
that the results are consistent and plausible with not be arbitrarily chosen and periods of less
what is expected a priori. than a year should be avoided because of
seasonal influences on consumption patterns.
Household Budget Surveys
Furthermore, some countries exclude, from
4.42 As a general rule of thumb, the sample size multi‐year averages, years which are
(number of households) for the Household Budget exceptional, e.g. as a result of particularly poor
Survey when used for the purpose of the CPI should harvests leading to high prices.
be such that the expenditure data yielded are able to
• Leave the CPI structure unchanged and simply
ensure statistically reliable weights at the
accept that the weight for the particular
elementary aggregate level. In some countries the
25
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
elementary aggregate concerned is of doubtful less likely to accidentally pick up business
quality. Whether this is an acceptable position expenditures.
to take will depend on the weight of the
4.48 The HBS can theoretically provide information
elementary aggregate and on its importance,
on regional breakdowns of expenditures and hence
particularly to analysts For example it would be
the weight data by geographical stratification.
a difficult position to sustain if the elementary
aggregate has a large weight and is presented as 4.49 The HBS provides data about household
a published sub‐index. characteristics such as income and number of
members. This is useful for ensuring that the
4.44 It should be noted that in normal
expenditures correspond to those of the CPI
circumstances weights can tolerate a certain degree
reference population and can also be used for
of imprecision before having a significant effect on
producing CPIs for different population sub‐groups.
the overall CPI, particularly at the higher levels of
aggregation, or main divisions of the CPI. But this is 4.50 Depending on its design, the HBS may be able
less so at lower levels. For instance, take an index to provide information on the types of outlet where
described as “fruits and vegetables” where with a purchases are made and the brands purchased. This
biased estimate of the expenditure weights, fruits information can be used to construct elementary
account for 60% of the index and vegetables the aggregates at a finer level of detail and to improve
remaining 40%. This biased fruit category will not the sample design for shops and items for price
only influence this product’s weight but will collection. But it should be borne in mind that the
compound itself by affecting also the relative HBS places a heavy burden on respondents and as a
importance of vegetables in the basket, i.e. if the result can suffer from non‐response. Extending the
true weight for fruit is 40%, then residually the HBS with further questions may make this problem
weight of vegetables will be 60%, not 40%. worse.
Consequently, the price index for “fruits and
vegetables” will be also biased. To minimize the 4.51 Given that most HBS’s are probabilistic
potential for such occurrences, it is recommended samples, statistical quality indicators (e.g. standard
that the compiler always strives to get the best error and coefficient of variation) for the weights can
possible estimates for the expenditure weights. be calculated. Such information can provide the
index compiler with some direction on the structure
4.45 An annual HBS is optimal for a CPI because as of the elementary aggregates. If an analysis of the
well as avoiding one‐off set‐up costs, it permits the HBS shows, for example, that the expenditure data
annual updating of the weights hence reducing the for lemons is of poor statistical quality, then a more
substitution bias associated with out‐of‐date weights broadly defined elementary aggregate such as citrus
in a fixed‐basket index like the CPI. Furthermore, it fruit could be considered instead.
provides the opportunity for using multi‐year
weights to reduce the sampling error and, where Disadvantages of the Household Budget
considered appropriate, minimise the sampling Survey
variance associated with unusual expenditure
patterns in a particular year (for instance, abnormal 4.52 HBS’s are resource intensive and costly. They
circumstances affecting consumer behaviour such as also impose a significant burden on respondents
political events, natural disasters, or oil shocks). But which may have an impact on response rates and on
obtaining reliable consumption estimates is the quality of information received. Estimates from
challenging and there is a persistent trade‐off an HBS are prone to measurement error, potentially
between data quality and survey cost. leading to biased results and inaccurate weights
data. This is especially true of recall surveys, which
4.46 The following outlines some of the are widely used in developing countries. Of
advantages and disadvantages associated with using relevance to the construction of CPI weights are
the HBS as the main source of weight data for the errors associated with:
CPI.
• Prestige (under‐ or over‐reporting associated
Advantages of the Household Budget Survey with the prestige of an item).
4.47 The HBS can be tailored to the particular • Telescoping (incorrect placing of an expenditure
needs of the CPI so that the information obtained in time by respondent). Telescoping usually
maps directly into the CPI classification system and affects non‐routine and infrequent purchases
targets the precise reference population. The HBS is such as books or furniture. This error can bias
designed to cover only consumer expenditures and is the results either upward or downward.
26
EXPENDITURE WEIGHTS IN THE CPI
27
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
years if they are known to be stable. The approach is
Table 4.2 Expenditure according to HBS illustrated in Appendix 4.1.
HBS all‐ Number of Implied HBS all‐ National Accounts
household households household total
(weekly (million) 2002 annual
4.65 The use of National Accounts weights ensures
average, $) Year expenditure
to June 2003 ($ million)
consistency and comparability between the CPI and
national accounts definitions and classification
4.816 25 6278 systems for household consumption, which is an
advantage when compiling a CPI as a macro‐
economic indicator and for use as a deflator.
4.61 To obtain the correction coefficient used to 4.66 National Accounts have two inherent
calculate the section weight the ratio of National advantages:
Accounts to HBS all‐household data is calculated. • The Household Consumption element of
National Accounts is derived mainly from the
Table 4.3 Correction factor (year to 30 June HBS but national accountants will often use
2003) other sources of information before finalising
their results, especially in cases where the
accuracy of the HBS is in doubt such as where
National Implied HBS National Accounts
under‐reporting is present. The fact that
Accounts all household expenditure/HBS National Accounts go through an additional
expenditure total annual all household data quality assurance process and re‐estimation
($ million, expenditure = correction factor should increase the reliability of the weights.
current prices) ($ million)
• Even if the HBS is updated infrequently, CPI
Annual total
weights can still be updated at regular intervals
13675 6278 2.178 from national accounts data for higher level
aggregates at the division or group level.
4.62 The HBS average weekly expenditure on 4.67 But there are three inherent disadvantages
cigarettes or tobacco index households is therefore with National Accounts data:
multiplied by 2.178. • It is generally only available at the national level
4.63 Adjusting for under‐reporting will impact on so deconstruction of the national accounts data
all weights, not just those where the under‐reporting to provide a finer‐level of detail or to produce
has been detected. All weights will need to be re‐ regional expenditure weights may be necessary
scaled to sum to 100 or 1000. using other available sources of information.
Other data sources include Household Budget
The commodity flow approach to adjust Surveys, retail enquiries and administrative data
household expenditure data from an HBS such as statistics on excise duty. National
Accounts data can be used to derive weights at
4.64 This generalised approach uses domestic the more aggregate level and HBS data can then
production and import/export figures obtained from be applied to the higher aggregates to derive
the national accounts to adjust consumption weights at the lower levels of the aggregation
expenditures which have been under or over scheme. If the expenditure data from the HBS
reported in the Household Budget Survey. Domestic are not viewed as sufficiently detailed to ensure
consumption is estimated from domestic production a minimum of acceptable accuracy, or if a
plus imports less exports, the latter including re‐ demand exists for indices of a finer breakdown
exports. Estimates of the proportion of domestic (e.g. there is a need for a price index for apples
consumption which relates to non‐household but only expenditures for all fruit can be derived
consumption are then used to estimate household from the HBS), then other potential data
consumption and the latter are compared with the sources can be used for disaggregating the
corresponding estimates from the household budget expenditures, including surveys on retail sales,
survey to provide conversion factors which can then point‐of‐sales surveys, surveys of production,
be applied to HBS expenditure data to adjust the export and import data, and administrative
latter for under or over‐reporting. The same data. Note that some of these sources may also
conversion factors can be used over a number of
28
EXPENDITURE WEIGHTS IN THE CPI
29
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
affected items now account for a smaller share of data at the higher levels of aggregation from the
total expenditures, all else being equal and on the most recent year available, say 2006, but the more
other hand, it increases as a manifestation of detailed information needed for deriving the weights
consumers’ reaction of shifting their purchases in at the lower level is only available from an HBS
favour of the items for which the relative price has conducted before 2006 then the latter can still be
declined – the “substitution effect”. Clearly, the used as a basis for disaggregating the national
compounded effects of these two influences can accounts data. The weights from the HBS may
potentially lead to serious distortions in the weights, benefit from price updating to ensure a common
which is not limited to the products that were price reference period for both sources. The practice
affected by the adverse event. How significantly the of price updating the weights is not without
basket shares are distorted will depend on the items’ controversy and is discussed in the following section.
importance in the basket and the magnitude of the
4.75 When weights are updated, the index using
distorting effects.
the updated weights needs to be calculated for an
4.71 For some goods or services an average of, overlapping period with the index using the previous
say, three years data may be used. This may be the weights for the purpose of chain linking. This is
case for items that are purchased infrequently or usually done in conjunction with a systematic review
which are subject to local variations in the weather of the CPI basket of goods and services. A more
so that reported expenditure can fluctuate detailed description of chain linking is given in
significantly from year to year. Fresh food is an Chapter 10.
example of a product category for which abnormal
circumstances are most likely to occur ‐ a category Weight reference periods and price
which carries a large weight in the CPI basket of updating
most countries. The purchase of new cars may be an
4.76 This section deals with the issue of how to
example of an infrequent but high value purchase.
align the weight reference period of expenditure
Multi‐year weights for fresh food would smooth out
data from national accounts or household budget
the effect of fluctuating expenditure, if this is
survey with the price reference period. This is
considered desirable in the CPI. Multi‐year weights,
normally done by price‐updating the expenditure
at least for cars, are likely to improve the precision
values, as explained in the following paragraphs, and
of the share estimate for personal transport costs in
afterwards using these price‐updated values to
the basket.
subdivide the more aggregate weights from national
4.72 Whilst the ILO Resolution states that accounts (assuming constant quantities) or by using
consumer’ expenditure patterns in the weighting the expenditure shares from the Household Budget
reference period should be no more than five years Survey as they stand, without any updating, and
old, in reality the weights of the CPI need to be using these to subdivide the aggregate national
updated with a frequency sufficient to ensure that accounts weights (assuming constant expenditure
there is no significant bias from product substitution. shares). The following explanation in principle
relates to the price‐updating of weights irrespective
4.73 Clearly a systematic review of weights of whether these are derived from national accounts
becomes more pressing when there are rapid or household budget surveys.
changes in consumption patterns (for example,
resulting from high inflation, significant changes in 4.77 But because of the time lag associated with
disposable income or the introduction of new the collection and compilation of expenditure data,
products or features) or the time from the weight the weights will refer to a period prior to the price
reference period lengthens. In principle, it is reference period of the CPI. Under these
recommended that weights should be updated more circumstances the expenditure weights may be
frequently than every five years and, in particular, aligned to the price reference period by price‐
that an annual review is carried out of the weights updating the expenditure weights from their
attached to sub‐indices and other major reference period to the price reference period. Price‐
components of the CPI. Such a review should updating is usually applied at the lowest level of
examine whether there is evidence of significant aggregation, i.e. at the elementary aggregate level,
changes in consumption patterns. where each expenditure weight is multiplied by its
corresponding elementary index for the period
4.74 If different sources of weights are used, then stretching from the weight reference to the price
their reference periods do not necessarily have to be reference periods. The resulting price‐updated
of the same vintage although this is desirable. For weights are then rescaled to sum to unity.
example, if the compiler is using national accounts
30
EXPENDITURE WEIGHTS IN THE CPI
31
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
elementary aggregate is an outlier which influences then the weights can be re‐referenced by chaining.
the published headline CPI. Chain linking constructs a continuous price series by
multiplying together price indices that have been
4.87 In addition, the compiler will need to make a
constructed using different weight reference
judgement on the likelihood of a weight being out‐of
periods. Chain linking is covered in detail in Chapter
date. A view on this will need to take into account
10.
such factors as the extent to which the dynamics of
the retail market suggest, a priori, that the weights Publication
could go out‐of‐date relatively quickly (for instance,
where a new technology, such as mobile telephones, 4.89 For transparency reasons, it is recommended
has generated new sales), the size of any divergence that some weighting data be made available to
in indicative trends in expenditure compared with users. However, the elementary aggregate indices
the rest of the CPI (for example, by extrapolating and weights do not necessarily need to be published
previous trends and taking into account any recent or released to users. (This is not a criterion for
market intelligence including trends in industrial defining an elementary aggregate, although it is
production or retail sales) and whether prices have often thought so.) There can be issues of
changed relative to other goods and services. The confidentiality that will prevent indices at the lower
compiler will also wish to take a view on whether the levels of aggregation from being released to users.
price elasticity of demand for a product is likely to be This would be the case, for example, when an
zero or one. elementary index associated with a product is based
on data from only one outlet. Moreover, the
Updating of weights including new volume compilers may feel that they are able to obtain
information information about market share which is reliable
enough to create, say, strata by outlet‐type, but not
4.88 When new expenditure information becomes sufficiently reliable for the corresponding indices to
available, for example from a recently completed be published.
HBS or a more up‐to‐date set of National Accounts,
32
Appendix 4.1 Example of how the commodity flow approach can be used to adjust the household expenditure data
during the computation of CPI weights
Soap 148,187,444,625 12,808,341 20,118 12,828,460 135,358,984,399 13,535,898,440 121,823,085,959 10,017,156 120,205,874,424 1.01
Sugars 1,647,793,730,000 49,055,645 11,643,590 9,747,532 21,391,123 1,675,458,252,044 1,172,820,776,431 502,637,475,613 25,681,735 308,180,825,088 1.63
Tobacco 92,860,273,000 6,812,955 56,058,304 56,058,304 43,614,924 4,361,492,405 39,253,431,647 3,607,513 43,290,160,308 0.91
Beverages 734,682,341,075 11,809,525 12,868,875 133,531 13,002,407 733,489,459,675 73,348,945,968 660,140,513,708 23,611,336 283,336,039,140 2.33
Petroleum 580,418,789 54,150,138 44,296,048 98,446,187 481,972,602,045 337,380,821,432 144,591,780,614 9,825,886 117,910,636,080 1.23
ASSUMPTIONS (these assumptions are for illustrative purposes)
1. Assume 10% of the domestic consumption to be non‐household consumption in cases of Soap, Tobacco and Beverages
2. Assume 70% of the domestic consumption to be non‐household consumption in cases of Sugar and petroleum products
N.B
1. Situations whereby the conversion factors are close to unity as in case of Soap and Tobacco, no adjustments are made on the Household Expenditure data from the Household
Budget Survey (HBS).
2. Commodity Flow approach is not restricted to only known items that are underestimated during the HES such as Tobacco, Beverages and the like
3. Commodity Flow approach is applied to items: (i) Whose weights look to be outliers; (ii) Whose coefficients of variation are substantially high according to the HBS results.
4. Under Beverages, there are four elementary aggregates; Sodas, Mineral Water, Bottled beers and locally brewed beers.
33
EXPENDITURE WEIGHTS IN THE CPI
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 4.2 Price reference 4.2.4 The regular calculation of a CPI will use an
approach that differs from the one suggested by the
periods, price updating and the Laspeyres price index formula. This is because the
index formulation weight reference period, which spans a year, will
precede the price reference period, which usually
4.2.1 In the basic Laspeyres index the price relates to a month (or in some cases a quarter or a
reference period and the weight reference period year).10 For example, a monthly CPI may be compiled
are identical, namely period b. from January 2009 onward with December 2008 as
the price reference month, but the latest available
weights may refer to, say, 2006. Consequently, the
∑p q t
i
b
i
statistical office will have to exercise the choice of
either price‐updating the weights from the weight
t /b
PLA = i
( )× p pi0
that data on the individual quantities, q, are typically
unavailable, while expenditure information is wi = p q b b
i i b
= pi0 qib
available to the compiler. i
34
EXPENDITURE WEIGHTS IN THE CPI
Figure 4.2.1 Timeline: price‐updating weights
Weight- Price- End-of
reference reference Current index link
period period period period
Time
b 0 t T
35
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 4.2.1 Price‐updating of expenditure weights
36
EXPENDITURE WEIGHTS IN THE CPI
longer the period of time between the weight is to review and update them as frequently as
reference period and the price reference period possible to reduce the bias that would (or could)
when the weights are introduced. Thus, whether the otherwise distort the index.
weights are price‐updated or not, the ideal solution
37
CHAPTER 5
Sampling Procedures
Introduction where sampling would not make sense or would be
unreliable because the number of prices is very
5.1 Chapter 5 of the CPI Manual gives advice both small. For instance, no sampling would be involved if
on sample selection, that is how to construct a an electricity tariff consisted of a standard standing
sample (the focus of this chapter), and on estimation charge for service provision and a standard charge
procedures, that is how to estimate the CPI from the per kilowatt of electricity used, which was the same
sample of prices collected (the focus of chapters 7 for all customers regardless of location and only
and 8). The CPI Manual is not prescriptive on the varied with total usage (with heavy users getting a
selection of samples, recognising that in practice discount after a certain threshold). In this case the
non‐probability sampling often needs to be adopted. tariff prices would be collected and applied to a
Similarly, although paragraph 35 of the 2003 ILO typical cross‐section of users and varying quantities
Resolution states that probability sampling of electricity. Sampling would be used to choose a
techniques are to be preferred, paragraph 36 goes cross‐section of users.
on to say that “where appropriate sampling frames 5.4 This chapter focuses on sampling procedures
are lacking and it is too costly to obtain them, for local price collection in outlets, including options
samples of outlets and products have to be obtained relating to probability and non‐probability sampling.
from non‐probability methods” and that There is a section which specifically addresses the
“statisticians should use available information and special challenges of sampling prices in markets and
apply their best judgement to ensure that prices charged by street traders. The associated
representative samples are selected”. issue of price bargaining is addressed in Chapter 6.
Chapter 6 also discusses the issue of volatile prices.
Background 5.5 Advice on the sampling of tariffs and other
centrally collected prices is given in Chapter 9 on
5.2 In order to construct a perfectly accurate CPI, Special Cases. Specific problems arise in a CPI,
the price statistician would need to record the price associated with the drawing of static samples and
of every variety of every good and service purchased the maintenance of a fixed basket in a dynamic
by the consumer. This would mean collecting a full universe, where retail outlets and goods disappear
set of prices from every outlet, including market to be replaced by new ones. Chapter 7 deals with
stalls and street vendors, and from every provider of item substitution and quality change and Chapter 8
a service, including public utilities such as water and with how to introduce new items.
electricity, private transport including shared
minibuses and the hire of rickshaws, modern forms 5.6 As only a sample of prices will be recorded in
of communication, such as mobile telephones, and the course of local price collection, there is inevitably
the provision of domestic service. As this is not some sampling error in measuring the CPI. The
feasible in practice, most prices have to be sampled sampling procedures should aim to minimise this
and this involves local price collection in a selection sampling error, maximise sampling efficiency (i.e.
of outlets in a sample of locations chosen to be obtain the maximum sampling precision for
representative of the country as a whole and at minimum fieldwork and processing costs) and
selected times on selected days. ensure no bias. The sample design itself should allow
publication of sub‐indices at all levels which have
5.3 The exceptions to the above are prices which been decided upon, such as regional indices or
can be collected from a central source, such as a separate sub‐indices for urban and rural areas. As
public utility provider or a government department. well as cost, a limiting factor in sample design is the
For many of these items, all prices will be taken and time taken in collecting prices. Practical
no sampling will be involved. For example, the considerations that will need to be taken into
service provider may give the national statistics account include the availability of price collectors
institute a full price list or tariff from which all the and transportation issues.
prices can easily be extracted. This may be the case
38
SAMPLING PROCEDURES
5.7 In general, national statistics institutes adopt openings, and even fewer will have lists that cover
four levels of sampling for local price collection: all market stalls in all types of markets, or mobile
locations; outlets within locations; items within street vendors. The relative advantages and
different sections of the expenditure classification; disadvantages of random and purposive sampling
and product varieties. Stratification is also frequently should be examined at each stage of the sample
used to increase sampling efficiency, especially selection. It is recommended that the National
where the retail market is heterogeneous. Often a Statistics Institute should first decide on the ideal
mixture of probability sampling and non‐probability sampling solution and then modify this to take into
(purposive) sampling is used. account practical constraints.
5.8 When using probability sampling, the units in 5.10 The ultimate goal should be:
the sample are selected so that each has a known
non‐zero probability of selection. For instance, • An overall sample which is representative of the
locations could be randomly selected from local total population of goods and services being
administrative areas with probability according to offered for sale and purchased. The sample
total population12, the latter representing a proxy for chosen should be representative of price levels
the retail turnover in the area. Within a location, and, most particularly, price movements. All
outlets could be randomly selected from a business variations of items and outlet types should be
register, with probability according to their considered for each product and chosen to
individual turnover of sales or by floor area reflect typical consumer purchasing habits.
measured by an enumerator listing and visiting each • A variance or mean square error which is as low
shop. Sample selection based on probability as possible. Samples should be reasonably
according to size increases sampling efficiency. Also optimised. At the very least, a basic analysis of
as the aim is to have a sample which is sampling variance should be conducted, even if
representative of retail turnover, the prices an overall estimate of the precision of the CPI
subsequently collected on the above basis would cannot be made.
then not need to be re‐balanced by re‐weighting if • Optimisation. The entire set of sample prices
the assumption holds that those population and should be optimized to meet the publication
floor areas are good proxies for turnover. needs of the CPI, taking into account user
Alternatively, each location and outlet could be requirements, practical data collection
given an equal chance of selection in the sample, considerations and cost.
regardless of the total proportion of the retail 5.11 An overview of some of the main sampling‐
market that they account for, but then re‐weighting related issues for a CPI is given first.
would be necessary.
5.12 The following paragraphs provide an example
5.9 In practice, sample selection is never of how a sample might be put together, including a
straightforward and compromises have to be made discussion of some of the choices to be made.
for good practical reasons even when a sampling Precisely how the CPI sample is selected in practice
frame exists. Administrative boundaries can be will, of course, depend on the individual
unusable. For instance, choosing administrative circumstance of each country. The detailed example
areas using probability according to population will does not address the time dimension, which is
ignore the inconvenience of administrative covered in Chapter 6 on price collection.
boundaries that straddle the border between a
commercial district and a residential area so that, Overview
contrary to the intention, the commercial district has
no chance of being selected as it contains no houses.
5.13 This section provides an overview of the main
Also a visit to the location may indicate that it is
sampling issues arising in connection with price
impractical for the collection of prices, for example,
collection for a CPI.
because of a physical barrier such as a railway or
river bisecting the area and causing difficulties of 5.14 Sampling for a CPI has three dimensions:
access. Similarly, very rarely do national statistics
• The product dimension, that is, all goods and
institutes have readily available sampling frames
services available for purchase. Available
which reliably list all retail outlets, particularly recent
sampling frames may include:
12
o Product lists of major wholesalers and the
A more appropriate alternative might be number of workers
sales values in a previous period.
employed in the location if this is known. This may be the case
where most shopping is done in town centres but with most
people living in residential areas in the suburbs.
39
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
o Outlet‐specific lists of products (shelf space sampling frame, that is, a list of all the outlets or
could be used as a volume of sales measure). products eligible to be sampled. There are three
main methods of probability sampling:
o Recent Household Budget Surveys.
• Simple random sampling, when outlets or
• The geographical and outlet dimension, that is
products are sampled with equal probability.
all places, outlets13 where a product is sold.
This can be used when all the outlets or
Available sampling frames may include:
products are homogeneous. But this is rarely the
o Business registers. case in practice. For example, a stall in a rural
market is unlikely to sell the same brands and
o Telephone directories.
quality of clothing as a store in an urban area
o Recent Household Budget Surveys. selling designer clothes which is also likely to
have a much higher volume of sales.
• The time dimension, that is, the sub‐periods of
the index. • Systematic sampling, when outlets or products
are chosen with equal probability but sampling
o Less attention is given to this dimension
units are selected at equal “distances” from
since price variation is usually small over a
each other in the frame, with only the first unit
short time period. Nonetheless it is
being randomly selected. Again the outlets or
important. For example, the price of fresh
products should be homogeneous.
fish or vegetables may vary depending on
the time of day, with price reductions • Stratified random sampling with probability
towards the end of the day when the proportional to size. This method divides the
product is less fresh, there is unsold stock heterogeneous population of outlets or
and buyers are scarce. products into various homogeneous sub‐
populations or strata. An independent sample of
5.15 Sampling techniques are used because to
appropriate size is then selected for each
include all goods and services would be both
stratum.
cumbersome and costly in term of data collection
and processing and could not be carried out in a 5.19 The sampling frames required for probability
short enough timescale to allow the timely sampling are rarely available, so the price statistician
publication of the CPI. either has to create one or rely on non‐probability
sampling. For this reason non‐probability sampling
5.16 But it is also worth bearing in mind that
has been heavily used to draw samples for price
sample data may be of better quality because of, for
collection in the CPI. Unlike probability sampling,
example:
non‐probability sampling more easily caters for the
• The more effective use of a smaller number of need to ensure that:
more specialised and better trained data
• Samples can be monitored over a long period.
collectors, editors and processors.
• Price collection takes place in accessible areas
• More time available to handle the data collected
where there are price collectors.
– the time available for data collection,
processing and analysis is always short. The • Costs are kept to a minimum.
more data collected, the less time is available to
thoroughly check and edit each price quotation. 5.20 Non‐probability sampling techniques include:
40
SAMPLING PROCEDURES
41
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Figure 5.1 Location selection
Range
Location Name No. Outlets No. Employees Cumulative Total Lower Upper
Location A 607 5377 5377 1 5377
Location B 306 2486 7863 5378 7863
Location C 264 2265 10128 7864 10128 Section 1
Location D 449 4006 14134 10129 14134
Location E 322 2589 16723 14135 16723
Location F 319 2097 18820 16724 18820
Location G 283 2127 20947 18821 20947
Location H 457 5252 26199 20948 26199
Location I 539 4945 31144 26200 31144 Section 2
Location J 371 4101 35246 31145 35246
Location K 518 4875 40121 35247 40121
Location L 928 10923 51044 40122 51044
Location M 407 3366 54410 51045 54410 Section 3
Location N 374 2449 56859 54411 56859
Location O 539 3625 60484 56860 60484
Location P 326 3357 63841 60485 63841
Location Q 291 4273 66314 63842 66314
Location R 277 2052 68366 66315 68366
Location S 1815 16499 84865 68367 84865 Section 4
Location T 443 3930 88795 84866 88795
Location U 329 2387 91182 88796 91182
Location V 258 2122 93304 91183 93304
Location W 420 3513 96817 93305 96817 Section 5
Location X 1714 20335 117152 96818 117152
Location Y 305 2819 119971 117153 119971 Section 6
Location Z 458 3429 123400 119972 123400
Location AA 380 3777 127177 123401 127177
Location BB 264 2375 129552 127178 129552
Location CC 452 6218 135770 129553 135770
Location DD 271 1839 137609 135771 137609
Location EE 250 1792 139401 137610 139401 Section 7
Location FF 870 8100 147501 139402 147501
Location GG 1315 16303 163804 147502 163804 Section 8
Location HH 321 2139 165943 163805 165943
Location II 283 2227 168170 165944 168170
Location JJg 2365 21887 190057 168171 190057 Section 9
Location KK 312 3097 193154 190058 193154
Location LL 314 2724 195878 193155 195878
Location MM 332 2649 198527 195879 198527
Location NN 309 1723 200250 198528 200250
Location OO 892 7864 208114 200251 208114 Section 10
Location PP 499 5921 214035 208115 214035
Location QQ 408 3299 217334 214036 217334
No. of collections of basket* 10 *For example, ten price collectors collecting prices in ten locations
Employment total 217334
Interval value 21733.4 = Employment total/No. of collections of basket
Random number 0.39904
Random starting point 8672.5 = Interval value x Random Number
Random numbers for selection: 8672.5 = Random Starting Point
30405.9 = Random Starting Point + Interval Value
52139.3 = Random Starting Point + 2 x Interval Value
73872.7 = Random Starting Point + 3 x Interval Value
95606.1 = Random Starting Point + 4 x Interval Value
117339.5 = Random Starting Point + 5 x Interval Value
139072.9 = Random Starting Point + 6 x Interval Value
160806.3 = Random Starting Point + 7 x Interval Value
182539.7 = Random Starting Point + 8 x Interval Value
204273.1 = Random Starting Point + 9 x Interval Value
g
JJ will be selected with certainty.
42
SAMPLING PROCEDURES
statistician may decide that in order to ensure that a useful for simple random sampling or systematic
full shopping basket (all the items in the CPI) can be sampling but not for PPS sampling unless
collected in each selected location, then locations additional information is sought, for example by
with less than a certain number of outlets should be visiting the outlets.
excluded. Similarly, a decision might be taken to pair
• Records of local administrations. These might be
an out‐of‐town shopping area with a small adjacent
records kept by local government, associations
location in the near vicinity to form a new location.
of businesses or market place managers etc.
This may be done where the out‐of‐town shopping
These records could be used to create frames
area does not contain food outlets, despite
and might be particularly useful for sampling
attracting significant expenditure on non‐food items,
local markets. Depending on the presence of
such as electrical goods, home furnishing and
information on size, these sources might be able
decorating and other home improvement goods. In
to provide a frame for PPS sampling.
order to obtain a full basket it may be necessary to
pair such out‐of‐town shopping areas with locations • Recent Household Income and Expenditure
in which food is available. Surveys (HIES) or Household Budget Surveys
(HBS). Some of these surveys are designed to
Sampling of outlets indicate, on the survey form, the actual outlets
Judgmental sampling from where the households purchased the
goods and services recorded (“point‐of‐
5.30 For this method, price collectors are purchase” surveys). Surveys that do include this
instructed to choose outlets which are reasonably information could be used as a source of
popular with the typical shopper and which will relevant outlets for survey frames for various
represent the typical shopping pattern in the area regions. In addition, the number of survey
for the types of products being priced. This can be respondents making purchases could be used as
done by providing a check list which provides further a measure of retail turnover for PPS sampling.
detail of the characteristics to look for. It is unlikely But a survey is normally a sample and so is
all these characteristics will be available from a unlikely to constitute a complete list of all the
central register. This method of sampling often outlets available to consumers.
involves the price collector visiting the outlet to
gather together the necessary information on the 5.32 In the absence of existing sources as
outlet’s characteristics prior to decisions being made suggested above, it is necessary for the outlets in a
on its inclusion in the sample. For instance, if prices location to be enumerated to provide
for children’s clothes are to be collected, a price supplementary information for the sampling frame
collector may visit a clothes shop initially selected or to construct a sampling frame from scratch. This
from a central list to check whether it sells children’s enumeration could be carried out by price collectors
clothes and not just adult’s clothes. or their supervisors, visiting each location and noting
details of all retail outlets found. It can be a very
PPS sampling costly activity and the price statistician will need to
weigh up the costs against the benefits in terms of a
5.31 Sampling frames for the outlet dimension can
more representative sample. To reduce costs the
be derived from several sources including:
price statistician may decide to limit the number of
• Business registers. These are lists of businesses, outlets enumerated per location, for instance by
normally maintained by the National Statistics enumerating only a sub‐district of a location. The
Institute or Ministry of Commerce. They may details noted for each outlet during enumeration
contain information such as location, type of might include: address; the range of items sold; and
business activity, turnover, and employment. whether it is independent or a branch of a multiple.
Business registers should be updated regularly Shops of centrally collected chains of retail outlets,
both for new businesses and just as importantly markets stalls and street vendors would normally be
for closures. Provided that some size measure is excluded from the enumeration as they will be
included in the information, business registers subject to different sampling procedures or at the
can provide a useful basis for preparing a PPS very least be part of a separate exercise. Prices in a
sample. CPI are combined using expenditure weights. In
order to use PPS sampling the ideal size measure of
• Telephone directories of businesses (e.g.
an outlet would be turnover. But a proxy may be
“yellow pages”). These normally contain less
used where an estimate of this is not readily
information, for instance just business name,
available. For instance, the approximate net retail
address and business activity. They do not
floor space, as estimated by the outlet enumerators,
include any size information. Therefore they are
43
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
may provide an adequate alternative. For shops, should be collected in each location for that
such as department stores, selling a wide variety of commodity group (two for meat). For meat, there
goods, the floor space devoted to each commodity should be one price from a butcher and one from
group should be measured. either a supermarket or a department store that
sells meat. The fifth column shows what type of
5.33 The reason for recording the range of items
sampling is used to select the sampled outlets from
sold in a shop is to provide a link between outlets
all those of that type in that location. The butcher is
and items to enable some form of sample
selected by a simple random sample of all butchers
stratification, so that outlets selected for inclusion in
in that location; the supermarket or department
the sample provide an adequate range of goods to
store is sampled with probability proportional to
generate sufficient prices for the CPI basket. Where
size, since store size in this outlet group is likely to
COICOP isn’t used to code what categories of goods
vary widely.
each shop sells, an appropriate mapping should be
created to map across to the corresponding COICOP 5.36 A shop holding a closing down sale should be
category to allow outlets to be classified by treated as already closed and hence should be
commodity group. Whilst this may not be a true excluded from the sampling frame. This is because
stratification ‐ an outlet may be in more than one its prices will not be comparable with previous ones,
stratum if it sells items from more than one and will not be available in the future.
commodity group – it should be adequate for the
5.37 Examining Table 5.1 in more detail, the
purpose of drawing a sample of outlets for the CPI.
decision making process works as follows:
5.34 For each commodity group, the required
• In the context of outlet selection, the
number of outlets, plus some reserves (used if an
commodity group column relates to the way in
outlet closes down) is then drawn from the sampling
which similar types of items are grouped
frame by either simple random sampling (SRS) or PPS
according to the type of shop in which they are
sampling. The latter is used where there is known to
likely to be found. Collectors are required to
be a wide range of store sizes and therefore a wide
collect prices for all items within a particular
range of turnover, such as for furniture or hardware
commodity group ‐ so in this particular example
stores which may be superstores or local shops.
all three meat commodity groups need to be
5.35 The framework in Table 5.1 shows how this collected from butchers. Although it has no
approach works for meat. Items are grouped into direct bearing on sampling other than to provide
commodity groups, so fresh beef and lamb are a framework to identify the different types of
grouped together, as are all cooked meats. The shops in which we might want to collect prices,
second column lists the types of outlet where meat it can be seen that this categorisation approach
is sold. These meat items are sold in butchers, is useful from a field management perspective.
supermarkets, and, although less frequently, some
• Outlet types relates to a form of stratification
department stores. The third column shows whether
intended to ensure that prices are collected
a multiple or an independent shop should be
from all types of outlet. In the case of cooked
selected: for meat, or whether either may be
meats, or any other meat product listed, this
selected. The fourth column shows how many prices
Table 5.1 Outlet selection procedures
44
SAMPLING PROCEDURES
might in theory cover small one‐man shops, Specific brands or varieties conforming to the item
small supermarkets, large or out of town description must be easy to find by the price
supermarkets. The decision on the number of collectors, ensuring that estimates of price changes
types of outlet from which to collect prices for are based on an adequate number of price
each meat product will depend on a number of quotations. As far as possible, they should also be
factors including the extent to which price available for purchase during the life of the CPI
movements for a particular commodity group basket. For instance, if the CPI is based on the cost of
vary by outlet type; the overall weight of the a fixed annual basket of goods and services, the
items collected in those shops; the aggregation expectation should be that the specific brands or
formula (greater stratification increases varieties selected will be available throughout the
homogeneity, which is particularly year. The exception is certain products, such as some
advantageous if the ratio of averages is used in foods, which are seasonal, and so require a slightly
the elementary aggregate); and whether different treatment (see Chapter 9).
average prices are required either for
5.40 In selecting the sample of items to represent
publication or for imputing the data into the
distinct categories of household spending, the items
computation of purchasing power parities
must be well defined so that changes in the recorded
(PPPs) (see Chapter 17). In the case of the latter
prices from month to month reflect only the change
a representative full cross‐section of shops is
in price and not changes in the products observed.
required and, probably, a larger sample.
However, sometimes a relatively wide definition is
• Sample methodology column. The sampling used to accommodate rapidly changing consumer
methodology will depend in large part on the tastes, for instance clothing where fashions can
homogeneity of shop sizes, the assumption change very rapidly, particularly designer clothes and
being that larger shops have more sales. such things as football T‐shirts with club logos and
Butchers (or, for example, small independent players’ names. If the definitions were too specific in
grocers) tend to be of similar size so SRS which is these cases it would be very difficult for the price
easy to apply is used. For larger stores, this may collectors to find the examples of the items in the
not be the case and PPS should be used using a shops. The diversity of products and therefore the
proxy measure for turnover. Estimating floor range of possible price quotations that conform to a
area of that part of a store devoted to a particular item description have implications for the
particular type of product (e.g. meat items) and choice of elementary aggregation method (see
using this as a proxy for turnover, just one of a Chapter 10). Examples of typical item descriptions
number of possible approaches, can be difficult. are given below:
For example, the stock may be on the counter in
• large loaf, white, unsliced (800g);
a butchery department, in fridges or in freezers
which are generally not co‐located. So overall • locally killed fresh beef stored in refrigerated
floor area of the store may be used as a proxy conditions, rump steak (per kg);
and PPS is applied using this proxy. Of course,
• butter, home produced (250g);
the assumption that the floor area of a store is
directly proportional to sales of a particular • heap of fresh vegetables: greens (per kg);
product may be questioned. For example, larger
stores are likely to stock a wider range of • bundle of firewood (per kilo);
products (e.g. some large supermarkets stock • local beer (half a litre);
electrical goods and clothing); but it may be as
good an assumption as any depending on local • single bed (width approx. 3ft/90cm);
circumstances. • electric cooker, 4 rings, grill and oven;
Sampling of representative items • domestic help (hourly rate);and
5.38 The sampling of representative items for a CPI • men’s cotton suit (ready made).
is usually purposive or judgmental. The significant
difficulties involved in defining an adequate sampling 5.41 The number of items chosen to represent
frame (that is, a list of all the individual goods and price changes within each CPI commodity category
services bought by households), often preclude the will depend on the weight of the section and the
use of standard random sampling methods. variability of price changes between the various
items that could be chosen to represent the category
5.39 A number of factors need to be taken into (reflecting, for example, the diversity of products
account when choosing representative items. available). Intuitively, it makes sense to select more
45
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
items in product groups for which total expenditure there is a valid base price. When this is part of a
is high; this helps to minimise volatility in estimates general and systematic updating of the CPI basket,
of price changes for high‐weighted categories and prices should be collected for both the old and new
therefore in the CPI overall. However, if price products simultaneously in one month, where
movements for all possible items in a given category possible, to permit chain‐linking. As mentioned
are very similar, it is sufficient to collect prices for earlier, chapters 6 and 7 provide more practical
only a few. By contrast, if price movements within a guidance on price collection, substitution and quality
category are very different, a much larger selection change.
of representative items will be needed to get a
reliable estimate of price change for the category as Replacement of disappearing outlets and
a whole. For instance, a relatively large number of products
items may be selected in areas such as food and 5.46 Sometimes price collectors will find that an
clothing, whereas price changes for more outlet at which they have been collecting prices
homogeneous product categories such as petrol, closes or no longer stocks one of the products being
alcohol and tobacco may be based on fewer items. priced. Disappearing outlets and products within a
5.42 In practice, relative expenditures on the specific outlet are handled by re‐sampling on a one‐
different types of goods and services will play an to‐one basis when the outlet closes or the product
important role in determining the selection of ceases to be sold by the selected outlet. The criteria
representative items used to compile a CPI, using for selecting replacements will vary:
Household Budget Survey data on household
• Replacing an outlet on a like‐for‐like basis by
spending patterns, the same source of information
another market stall, independent shop or retail
which underpins the calculation of weights (see
chain in the same or similar location selling the
chapter 4). This may be supplemented by
same or similar range of items and products.
information on retailing trends from trade journals
and press reports. A request can also be made to the • In replacing a product one of two strategies is
price collectors and supervisors to report usually adopted. If the initial selection rule was
developments in the retail environment. “most sold” or “probability proportional to
(sales) size” then the replacement could follow
5.43 The usual practice is for representative items
the same rule with the advantage of maintaining
to be selected by Headquarters to ensure that they
the representativity of the sample. Alternatively,
are representative of consumer behaviour as a
the replacement may be made based on the
whole rather than one particular shopping location.
product most similar to the one which
Consistent with the principle of a fixed basket they
disappeared, thereby reducing the need for
are normally reviewed periodically at the same time
quality adjustment.
as the weights and chain‐linking is carried out (see
Chapter 10). 5.47 A more proactive approach is recommended
where there is a high turnover in outlets and
Selection of products and varieties products so that the number of forced replacements,
5.44 For most, if not all, products, the selection of referred to above, is kept to a minimum. This can be
products and varieties within outlets will be done by sample rotation, which for products is
purposive. In each outlet collectors choose one usually associated with the overlap method of
variety representative of what people buy in the adjusting for quality change (see Chapter 7) and with
area or which people typically purchase in the outlet chain linking (see Chapter 10). Sample rotation
from all products matching the specification of each involves full or partial re‐sampling. The method of
item to be priced in that outlet. To facilitate this they re‐sampling can follow any of the methods used for
may ask the retailer what are the most popular initial sample selection. Re‐sampling involves an
brands and which are those stocked regularly. As it is overlap period where the first period of the new
vital that the same product is priced each month, sample overlaps with the last period of the old
collectors must record enough detail of the product, sample. Sample rotation can also be applied to
such as make and model, to ensure that it is uniquely items.
identified.
Sampling procedures for markets and for
5.45 The chosen products should be reviewed prices charged by street traders
periodically to ensure that what is being priced still
reflects the above criteria. If the product being 5.48 In many countries there are distinct outlet
priced becomes permanently unavailable, one that is types for the selling of goods and services: shops,
available must be chosen as the replacement so that traditional markets and mobile vendors. Each outlet
46
SAMPLING PROCEDURES
type tends to be a distinct part of retailing, often tightly specified product types is drawn up to
with very little customer overlap. Each is likely to provide effective control of the item sample and
follow a different pricing behaviour and so prices price collectors are asked to collect
must be obtained from each. “representative” prices for these products by
visiting a “typical” market stall or street vendor.
5.49 Markets, particularly unlicensed or
Prices are collected only for the products
unregulated ones, and street traders are particularly
matching this tight item specification
challenging for the compiler of a CPI both from a
sampling perspective and from the point of view of 5.50 Mobile vendors are particularly difficult to
obtaining the appropriate “representative” price include in a sample frame as they move around and
(the bargaining issue, which is addressed in Chapter may not trade every day or even regularly. Mobile
6). As far as sample selection is concerned, the CPI vendors may be a significant part of retail activity
statistician is often confronted with the challenge of: and so should not be ignored just because they are
informal. One option, adopted by many countries, is
• Constructing a tailor‐made sampling frame. For
to treat the locations where mobile vendors
instance, there might be information available
regularly sell their wares as “pseudo‐outlets”.
from city or other local administrations if
stallholders have to register in order to operate. 5.51 For instance, if it is known that second‐hand
Alternatively, the price collectors themselves clothes can always be purchased at a particular
may have to construct a list of traders in a street corner (but the individual vendors change),
locality. This would be feasible if most of them then consideration can be given to treating the
occupy the same space every day. corner as the outlet. This collection method assumes
that buyers have no preference between the
• Using a mixture of quota sampling and the
vendors and consider them as providing goods of
selection of “representative items or products”
similar quality.
to get a sample of individual prices. For instance,
judgemental procedures with respect to known 5.52 Price collectors should be instructed to visit
and relevant characteristics, such as the product different market stalls or street vendors until a
group, could be used when choosing which representative price is obtained. The latter will be a
items or products to price in a particular type of price which the price collector, from their experience
outlet (for example, the choice of fresh of price collection, considers reasonable for the
vegetables to price in a market). Then for the specified item of pre‐determined quality. It is
actual price collection the “representative advisable to use tight item specifications for markets
product” method could be adopted. A list of and street traders.
47
CHAPTER 6
Price Collection
Introduction at a different “unofficial” price. Central price
collection can be further broken down into:
6.1 Chapter 6 of the CPI Manual provides an o Prices which are combined with prices
overview of the most appropriate survey methods collected locally. For instance this may
for the collection of prices. In large part, the occur when a supermarket chain
considerations are the same as for sample design provides a centrally determined price
and will depend on local circumstance. For instance: list, eliminating the need for the price
the purchasing habits of consumers and the extent collector to visit a shop in person;
to which they use licensed and unlicensed markets
(regulated and unregulated or informal markets) and o Prices which are used on their own to
use catalogues for postal shopping; the structure of compute centrally constructed indices.
the retail market including the balance between Most tariffs fall into this category.
markets, small independent shops and large retail 6.3 Goods and services, such as public transport
chains; the extent of public ownership and price fares, electricity, mains water supply, physicians,
control; the diversity of goods and services being hospital services and telecommunications, which are
sold; the pricing structures used, including tariffs; paid according to a tariff normally, for the utilities,
and whether bargaining is common. The availability consisting of a standard rate per unit of
of central records of prices charged also has an consumption, sometimes in combination with a fixed
important bearing. Paragraph 48 of the 2003 ILO charge, can pose problems when their structures are
Resolution emphasises the importance of well‐ modified over time, damaging the principle of
trained price collectors who adhere to the standard unchanged consumption. A solution to this problem
procedures. The latter are elaborated in subsequent is to define representative services or bundles of
paragraphs which cover the rules relating to services (for example, categories of consumers and
transaction prices. specific services consumed). For these it is important
that the prices experienced by a representative
Background range of customers and tariffs are observed and that
customer profiles are kept constant over time. More
6.2 There are two basic price collection methods: detailed advice is given in Chapter 9 on Special
Cases.
• Local price collection where prices are obtained
from outlets located around the country. This 6.4 The focus of this chapter is on locally
will include licensed and unlicensed markets and collected prices. It begins by reviewing the principles
street vendors as well as shops. Normally the behind collecting prices for a CPI and then considers
price collector will need to visit the outlet the practical collection issues and how these should
although the prices for some items may be be managed. A working assumption has been made
collected by other means, including telephone that the index being compiled is an acquisition index
and price lists. (see Chapter 4).
• Central price collection. This is often used where 6.5 For the purpose of this chapter, it is also
prices can be collected by Headquarters without assumed that prices are being collected for a
the need for field work. This may also include monthly price index with prices therefore being
centrally regulated or centrally fixed prices collected, in general, every month. Some countries
which can be obtained from the regulatory produce only a quarterly CPI, while others produce a
authorities, although in these cases checks will weekly index, especially for fresh food. The concepts
need to be made to ensure that the goods and and procedures discussed here will apply to price
services in question are actually available and collection practices, whatever the frequency of index
actually sold at the stated price. It is not unusual publication.
to find goods subject to price control being sold
48
PRICE COLLECTION
The principles of price collection The principle of a fixed basket
6.9 Underlying all of what follows is an important
6.6 Except in a small number of cases, such as the principle: the necessity to compare prices on a like‐
treatment of owner‐occupier housing costs (see to‐like basis from one period to the next. This has
Chapter 9), a CPI normally is designed to measure two consequences:
the change in the actual transaction prices of
products and services bought by consumers. • Where the price collector has some
However, collectors cannot normally observe discrimination in what variety of a product to
individual transactions as they occur, so they must price in a particular shop, a consideration should
usually observe the price marked on or nearby the be whether that variety will be available to price
product and assume that these are the transaction over a reasonably long period (tight
prices. There are exceptions to this, such as specifications are of no use if the described item
bargaining, where a price might not even be cannot be found in the outlets). This is in
displayed and these cases are discussed in the addition to being typical of what is sold to
section entitled “Practical collection procedures”. customers.
Special procedures apply when outlets in which • The price collector should record additional
items are being priced close down, or the items information needed to ensure the unique
which the price collector was pricing at a particular identification of the variety priced so that:
outlet are no longer sold by that outlet. These
procedures are covered in Chapter 5 with additional o The same variety continues to be priced
references in chapters 7 and 10. in the case of price collection being
carried out by a different person.
Catalogue and list prices (not mail order)14
o The identification of quality change can
6.7 It can be argued that the use of catalogue or be made when the variety disappears
list prices, which supposedly give the prices at which and is replaced by a different one
goods and services are sold in outlets, is contrary to allowing an adjustment for quality
the principle of recording transaction prices, as the change to be made.
catalogue or list price may only be the
recommended price. Even where it is supposed to be
the actual price charged shopkeepers may not
Item specifications
always comply. However, in many cases the list price
6.10 There are no firm rules, especially regarding
is identical to the transaction price. In practice list
the use of loose or tight item specifications: each
prices can be a cost‐effective method of obtaining
country may choose its preferred methods – and
prices. They can be used ‐ but their reliability should
stick to them. However, there are a number of
be checked periodically.
considerations in deciding on item specifications:
Collection by telephone • Tight specifications leave less discretion to the
6.8 The prices for certain items, particularly price collector, so the reliability and training of
services such as electricians’ and plumbers’ charges collectors are factors to consider when deciding
and the cost of home security, may be obtained by whether to use loose or tight specifications.
telephoning the business or organisation concerned.
• Particular care should be taken to ensure that
This is the case where it is relatively easy to avoid
the specification is very detailed for
ambiguities in price as the outlets provide standard
heterogeneous items where there is scope for
items or services. However, even if prices are
significant difference between one variety and
obtained by telephone, the retailer should be visited
another, and for items which by their very
occasionally. This helps to maintain cooperation
nature are highly‐specified. Cars and hi‐tech
through personal contact and to ensure that there
goods fall into the latter category.
are no misunderstandings over the prices. This will
be more important for some retailers than others. • Tight specifications also allow for the
For example, the price of hiring a van may be less calculation of meaningful average prices:
certain than the cost of an eye test.
o Average prices are useful in identifying
outliers and assessing the accuracy of the
reported prices.
14
The pricing of goods and services ordered by mail can be
conducted by reference to mail order catalogues and the index
will need to take into account posting and packing costs.
49
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 6.1 Structured Product Descriptions: example of a photo bank
Product Product
Indicator product Picture Comments
code group
1. FOOD
1.1.Dry Food
Rice ‐
Brand name: Tastic
01111001 Rice Quantity: 1 bag
Size: 2
Unit of size: kilogram
Loaf of white bread ‐
Note: If no Brand name is indicated,
use the Outlet name as the Brand
name, e.g. Woolworts.
01112001 Bread
Product name: White sliced bread
Quantity: 1 loaf
Size: 700
Unit of size: gram
Loaf of brown bread ‐
Brand name: Albany Bakeries
Product name: Superior brown
01112002 Bread bread
Quantity: 1 loaf
Size: 700
Unit of size: gram
Sweet biscuits Marie
Brand name: Bakers biscuits
Product name: Blue Label Marie
Quantity: 1 pack
01112003 Biscuits Size: 200
Unit of size: gram
Brand name: Bakers
Product name: Eet‐Sum‐Mor
Quantity: 1 box
50
PRICE COLLECTION
51
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
52
PRICE COLLECTION
53
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
54
PRICE COLLECTION
Table 6.1 Example of a survey form showing the number of price quotations by shop or stall
Items Targeted number of Actual Number of Quotations
quotations
Shop/Stall 1 Shop/Stall 2 Shop/Stall n
(set by head office)
Item 1 5 0 3 5
Item 2 4 4 5 4
Item 3 8 5 8 8
....
Item k 5 7 2 6
Collecting prices in a market can be an
unstructured process which is difficult to fully
Informal “bonuses”
plan for in advance because of, for instance, 6.48 An additional issue often faced by price
missing goods and differences in quality collectors in markets is the informal “bonus”,
between different stalls. For this reason price whereby either the stallholder refuses to bargain on
collectors should keep track of how many price for an individual purchase but will provide a
prices have been obtained from stall holders little bit extra (for example, by adding a few more
or customers from each stall and each item. A greens to the “heap”) or in return for buying a
sample form designed for this purpose for variety of items will offer the shopper some extra
recording the number of prices by stall and quantities or an extra item. This has some similarity
taken from the CPI Manual is presented in to the “2 for the price of 1” offer in supermarkets
Table 6.1. (see earlier section) except in one very important
iv. Ask the price from a supplier to the market. respect ‐ in developing countries the shopper is likely
This involves a regular survey of wholesalers. to have actively haggled for the bonus amount or
This approach could be considered to items, because they wanted the extra quantity, so
represent a departure from standard CPI that there is a genuine discount which should be
price collection procedures. The experience of covered in the CPI. The argument is particularly
countries which have tried this method compelling where people are living at subsistence
suggests that it should not be relied on as a level. Paragraphs 6.99‐102 of the CPI Manual provide
substitute but as a supplement to normal a detailed example of how such cases can be
retail price collection because of the difficulty treated, based on market values. Another example is
of converting wholesale prices into retail given below.
prices. Ideally, prices should be obtained from 6.49 Where a stall holder gives the purchaser
the wholesalers who actually supply the extra quantities as a bonus, the method of
relevant retailers. Provided certain conditions determining the price is as follows. A purchaser, in
are met (such as unchanged taxes, this case the price collector, sets out to buy 5kg of
transportation costs, licence fees or rental for yams, is offered 5kg for $20 and after a lot of
the market stall) the trend of wholesale prices bargaining is offered 5kg of yams at $15 and then
may be used as a proxy for the movement in after further bargaining is offered a free “bonus”
the retail prices of the products concerned. made up of 500g of yams and 1kg of “greens”,
Wholesale prices cannot replace retail prices making a total purchase of 5.5kg of yams and 0.5kg
but assist in determining a realistic price of “greens” at a total cost of $15. Since the price
movement when checking the validity of collector cannot bargain for the purchase of 500g of
observed price movements or to provide yams and 0.5kg of “greens”, the estimated value of
supplementary observations when too many the bonus is worked out from what the goods would
retail prices are missing. have cost if they had been bought separately under
6.47 It may be necessary to use a mixture of some the same bargaining conditions and with the same
or all of the above methods in order to obtain a percentage reduction from opening offer to final
reliable sample of prices. Paragraphs 6.92‐6.102 of price after bargaining. The opening offer price for
the CPI Manual provide a good deal of detailed yams is $4 per kg and the final “price” $3kg, a
advice. reduction of 25%. Suppose that the price collector
ascertains that the opening offer price for “greens”
55
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
from the same stall holder would be $2 per kg. Then received and report the incident so that staff at Head
it is assumed that the final offer price will be 25% Quarters can decide on forward action.
less, i.e. $1.5 per kg. From this the market value of
the additional quantities given free can be Frequency and timing of price
calculated. For greens this is $0.75 for 0.5kg. This
represents the value of the bonus “greens” to which collection
should be added the value of the bonus quantity of
yams, i.e. $1.5 (0.5kg at $3 per kg). Thus, the total Frequency
value of the free “bonus” is $2.25 and the total
6.53 Decisions about the frequency of price
value of the vegetables obtained by the purchaser is
collection are governed by several factors. The most
$17.25 at a total price of $15. This represents an
important are the volatility of prices, the types of
average discount of 14.3% ($2.25/$17.25). So the
market from which the prices are collected, the
discounted price of 5kg yams is $15 less 14.3% i.e.
known regularity of price changes, and the
$12.86. If the price collector is unable to get from
frequency and method of calculation of the CPI.
the stall holder the opening offer price for “greens”,
These, and other factors, are discussed in detail in
this can be obtained by surveying the opening prices
the CPI Manual, beginning at paragraph 6.5.
of other stall holders.
6.54 To accommodate the broad range of users,
Problems sometimes encountered the CPI is usually calculated at least once a month
but calculating the index monthly does not
necessarily mean collecting all prices every month.
Regular prices advertised as sale prices Each item should be priced as often as is necessary
6.50 In some outlets prices are indicated as sale to ensure that a reliable index can be calculated. For
prices but in fact are regular prices if compared with instance, the prices of some products (such as fees
previous prices. In such cases price collectors should for government services, utilities, tickets to seasonal
indicate the type of price as indicated by the retailer, sporting events) might change only once or twice a
i.e., sale or regular and report the price as is. year at a regular point in time. When it is known
with reasonable certainty when prices will be
No price on item or shelf reviewed by retailers or service providers, prices
6.51 A difficulty for price collectors is to collect need be collected only when they are actually due to
prices in outlets that are poorly organised. In these change. During index periods when the price will not
outlets clothes may be stacked closely together, change, the existing price can be used for index
bundled in a bin, or appear on shelves with no calculation. However, it is advisable for the price
prices. In such cases price collectors should enquire collector to occasionally check that the assumption
from shop assistants what the correct prices are. of regular and predictable reviews still holds. For
However, price collectors should take heed of shop products which exhibit a relatively low degree of
assistants that do not always know what the correct volatility, such as housing rent and medical services,
prices are or are unwilling to assist. Price collectors prices may be collected on a quarterly basis. In
may rather request information on uncertain prices contrast, the prices of products with volatile prices
from shop owners or scan the item at the pay point might have to be collected more frequently than the
if possible. This also applies to determining the frequency of index calculation and publication (see
availability of an item, including seasonal items. Chapter 9).
Shop assistant insists on giving price with no Point‐in‐time or period price collection
checking by collector 6.55 Before a timetable for collecting prices can be
drawn up, a decision has to be made by the National
6.52 Price collectors face many challenges to
Statistics Institute as to whether the index will be
obtain accurate prices. There are respondents that
calculated from prices covering the whole period
insist on providing the prices and will not allow price
(e.g. a month) or from prices relating to a particular
collectors further access. Price collectors receive
time within the period (e.g. three days in the middle
prices from these respondents but often sense that
of the month). In practice, this decision will be based
the prices are not correct. An example would be
on a number of potentially conflicting factors such as
that the respondent insists that no change in prices
the main uses of the CPI, available resources and the
have occurred over an extended period of time. In
pattern of price movements. For example, it might
all cases where price collectors are uncertain of the
be argued that, in principle, if an index is to be used
accuracy of prices they should record the prices as
with other economic statistics then the period of
coverage for price collection should match the
56
PRICE COLLECTION
57
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
take place on, say, the second Tuesday of the month Practical collection procedures:
for a point‐in‐time collection, or during the first
three weeks of the month for a period collection. planning and organisation
6.59 It is strongly advised that price collection 6.63 This section looks at the planning and
dates are not published in advance. This is to avoid organisation required for successful price collection.
the risk that some service providers or retailers Actual processes will vary between countries but the
change their prices or time price changes in order to same general principles apply. The next section gives
influence the index. advice on practical collection procedures in the field.
The importance of adhering to the same 6.64 For the sake of simplicity, the discussion is
price collection schedule each month based on period pricing rather than point‐in‐time
pricing (see earlier references). This does not mean it
6.60 A CPI is constructed from price relatives. The is the preferred method. It is merely that one
collection of a particular set of prices at a particular method has been chosen for the sake of brevity and
outlet should occur at the same time of the day, consistency. The concepts discussed will generally
same day of the week and the same week of the apply to both collection methods. Any significant
month in each collection period. Doing this will variation necessary to manage point‐in‐time pricing
ensure that any observed price movements will is mentioned.
reflect true movements in the price level rather than
price changes resulting from temporary changes in 6.65 The procedures governing the collection of
selling or buying behaviour. For instance, buyers prices have not only to cover the requirements of
might have a tendency to buy certain products only obtaining usable prices from the outlets but also the
at the end of the week after they receive their practical problems faced in managing travel to and
wages. Sellers can take advantage of this and keep from the various localities, and validation of
prices higher than average during the busy days of information back in the office. The overall operation
the week while reducing prices during the least busy can only be achieved by cooperation between the
days. Collecting prices on the same day of the week price collectors and their supervisors.
for each collection period and in each outlet is
essential in such situations. Examples of this
Planning the collection schedule
behaviour might be the purchasing of leisure 6.66 The collection schedule should include
services (e.g. restaurants, cinemas). In some sufficient time for a collector to travel around all
countries, mid‐week prices are cheaper than required localities within a reasonable number of
weekend prices for these types of service. working hours in a day. Times of travel might be
limited in order to arrive at a remote rural market
6.61 A particular issue arises with fresh produce
with sufficient time left to obtain all the required
sold from market stalls. Sellers often reduce prices
prices before the market closes. Allowance will also
towards the end of the day to ensure that all stock is
need to be made where collectors have to share
sold, especially if there is no practical means of
common resources such as the use of an office car.
storing the produce overnight and the freshness of
In addition, a price collector needs time to perform
the products has declined, indicating a reduction in
all the necessary checking of prices and to answer
quality. If consecutive prices are not collected in the
queries from the supervisor or Headquarters, where
same circumstances each month ‐ for example,
necessary re‐visiting the outlet concerned. The
before the prices are reduced ‐ the movements in
collection schedule also has to allow for the
observed prices cannot be guaranteed to measure
movement of information and forms between the
the ‘true’ price movement between the two periods
collection centres and Headquarters. If paper forms
based on a like‐for‐like comparison. Also it can be
are being posted or hand‐delivered, sufficient time
argued that late‐in‐the‐day reductions are clearance
must be allowed to ensure that all information
prices relating to spoiled produce which should not
reaches its destination by the required deadline. If
be priced (see earlier reference to Special Offers).
electronic transmission of data and forms is used,
6.62 Item and outlet replacement are common then only a little time has to be scheduled for this
occurrences during price collection and impact on activity
the ability to compare like‐with‐like. These issues are
6.67 Public holidays can occur on days that would
discussed in detail in Chapter 7 on Substitution and
otherwise be price collection days. In general, prices
Quality Change.
due to be collected on the holiday should be
collected as close as practicable to that day. This
usually means adjusting the regular price collection
58
PRICE COLLECTION
schedule for the immediately preceding or following likely to be about the accuracy of prices and other
few days. Any adjustments to the preferred regular information obtained, and are likely to be received
schedule should be made in such a way that prices shortly after collection. They are also likely to
reflect the normal pattern of buyer‐seller behaviour. require an answer quickly, before price data are
forwarded to Headquarters. Procedures have to be
6.68 The supervisor must discuss all these
in place that will make the existence of these queries
problems with all price collectors for the region to
quickly known to the collector. Similarly, reporting of
enable collection schedules to be drawn up that are
a response has to be quickly made known to the
practical and meet the requirements of everybody
supervisor.
involved, as well as complying with the price
collection protocols for the CPI. 6.75 Queries from regional offices are likely to
involve matters such as explanations of extreme
6.69 It should not be assumed that it is best for the
movements and inconsistent price behaviour within
collection itinerary to begin from the locality of a
the region. Again, rapid response will be required to
collector’s home.
enable the data to be forwarded to Headquarters
Dealing with queries: enquiry management within the required deadline.
system 6.76 Queries originating from Headquarters are
6.70 Price collection queries need to be dealt with more likely to be concerned with the
in a timely and efficient manner both because of the representativity of the information rather than the
tight schedules associated with the compilation of a simple accuracy of the information. They will also be
CPI and because of the fact that it is very difficult to received some time after collection has occurred.
correct errors in pricing retrospectively. The progress Because they are received late in the overall
in dealing with queries needs to be monitored and processing cycle for index production, the allowable
the system for monitoring must be simple, if it is to response time is likely to be short.
be effective and flexible to the needs of the CPI
production cycle to the problems which arise. An Practical collection procedures:
enquiry management system should be able to collecting prices in the field
monitor progress and provide an effective audit trail.
Information recorded is likely to include: date
received and who from; date due; received from;
Price collection and price collectors
collection date/trip/period (if applicable); current 6.77 There are two practical means of recording
progress (with date); date completed; date response price information obtained from personal visits:
sent. It is particularly important that decisions are paper form or hand‐held computer. Both methods
recorded and signed‐off. need to provide the price collector with all the
information required to successfully collect prices.
6.71 The information gathered should also feed
The information collected should include:
into a Quality Management System (see Chapter 15).
6.72 A simple system might use emails to send the • Collection date and collector’s name.
query. As these will be seen and read they satisfy the • Name or location of the particular outlet.
notification requirement. A template form requiring
simply ticks and dates to be entered could be • Product name and specification of the actual
created in a spreadsheet and a new copy printed for item to be priced.
every collector each collection period. • Price entry.
6.73 The enquiry management system can be • Price movement reason code.
either paper or electronic. The two methods should
be matched to the resources and infrastructure • Price collector comments about the product or
available. For instance, messages to and from price price movement.
collectors to their supervisor might be on paper but 6.78 The optional inclusion in the form of the price
messages between a regional office and collected in the previous period raises concerns that
Headquarters could be by email. the price collector may be inclined to automatically
6.74 The raising of queries about collected prices is record the previous price or be too influenced by it
a two‐way process. Queries can originate from the when identifying the item in the outlet to be priced,
price collector, the price collection supervisor or but it is sometimes included to assist the collector in
from compilers and analysts in a regional office or at ensuring that the correct item and price are being
Headquarters. Queries from the supervisor are more recorded. It will help if the price collector adds
additional information to the standard description.
59
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
This extra information could include descriptive tracking of activities and formal sign‐off confirming
categories such as: brand, make, size, model name that processes have been completed and necessary
and number, reference number, distinctive features actions undertaken on data checking and the
and location in outlet (for example, “bottom shelf at transfer of information to Headquarters. Audit trails
rear of shop”). are essential for work and quality management.
6.79 For fruit and vegetables, weight and quantity 6.82 Completion of the price collection forms,
information should be part of the item name and full whether by paper or electronically, should comply
description but it is also essential to record the with the following guidelines:
amount actually priced so that a unit price can be
• All prices should be entered into the collection
computed.
sheet in full even if there is no price change.
6.80 Items should appear on the form in such a This is good practice as it helps to ensure that
way that the collector is quickly able to identify and the correct item is being priced. A price should
read all necessary information about each item. The never be recorded as, for instance, ‘No change’.
order in which the items appear on the form is very All other entry fields on the price collection
important for efficient price collection. As far as sheet should have something entered if only to
possible, they should appear in the order in which indicate that they were not accidentally missed,
the collector expects to find them in the shop. for instance, “NA (not available)”.
Especially with paper forms a correctly ordered list
• If a price is not available a reason must always
will reduce time‐consuming swapping and searching
be supplied. The information given will be
between pages. Even with a hand‐held computer
useful to the collector’s supervisor and the
there will be less time‐consuming searching and
index compilers as well as to the price collector
pressing of navigation buttons if the item list is
during the next collection period. The price
ordered so as to match the shop layout. An example
collector may need to ask the shopkeeper for
of a typical price collection form is given in Appendix
an explanation.
6.1.
6.81 No matter which form of collection is used it • All information should be entered on the
is essential to provide some procedures to allow the collection sheet as soon as it is obtained and
Table 6.2 Examples of price collection codes
Collection code Description
C Comparable ‐ A replacement item that is comparable to the old item in
all major aspects.
DC Drop confirmed ‐ A price drop that has been checked and confirmed by
the price collector or shopkeeper
M Missing ‐ The item is no longer available and is unlikely to return. There is
no suitable replacement item.
N New item – A replacement item that is not comparable to the old item
but is still considered representative.
R Recovery – A price rise back to a normal level, following a reduced sale or
previous special offer price.
RC Rise confirmed – A price rise that has been checked and confirmed by
the price collector.
T Temporarily out of stock – The item is unavailable this time but is
expected to be available again in the near future.
W Weight – A permanent change in the weight or quantity of the item has
occurred.
60
PRICE COLLECTION
61
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
some explanation, or a follow‐up price collection, • Understand and recognise occasions when
will be required to check the accuracy of the prices prices provided are unacceptable.
collected. Preferably this should be done once data
• Recording relevant information to describe
have been entered into the computer system and
quality change in a product.
have been checked for transcription errors. To
enable the supervisor to conduct these checks • Recognising unusual price movements when
efficiently the computer should provide various checking their collected prices.
tabulations of price changes grouped by product or
elementary aggregate. This will enable the Induction training
supervisor to quickly identify extreme or inconsistent 6.94 Induction training should be compulsory for
movements. Extreme or inconsistent movements all price collectors so that they gain the necessary
may indicate either errors in collection or skills before collecting prices for the CPI. It can also
unexpected behaviour in the market. These checks be a motivational tool. It should be delivered by the
should be conducted regularly during the price National Statistics Institute and ideally include
collection period. contributions not only from those supervising the
6.89 Data checking procedures at Headquarters fieldwork but also from Headquarters staff involved
are discussed in Chapter 12 on Data Validation. in the compilation of the CPI. The latter are best
placed to convey data requirements and to put price
collection in the broader context of compiling the
Training and work instructions CPI. Also, by helping to train price collectors,
compilers can gain useful practical knowledge about
6.90 The training of local price collectors is a vital
pricing.
element in ensuring the quality of the prices data
and of the CPI. Collectors need to be properly 6.95 A typical training schedule might consist of a
trained, require adequate instructions and easy one day training course at Headquarters (which may
access to guidance, particularly so given the facts include some refresher training for experienced
that: collectors). This should include:
• Price collection is not a trivial matter. • Background to the CPI and the National
Statistics Institute.
• Snap judgements often need to be made.
• Use of the CPI and importance of accurately
• Collectors often work remotely and on their recording prices.
own.
• The general principles of index compilation and
• Instant communication is not always possible. price collection.
• Collectors work in a dynamic environment. • How local price collection fits into the process.
• It is difficult to rectify errors after the event. • Retailer recruitment, getting permission to enter
6.91 The CPI Manual states, in paragraph 6.124, shops etc.
that “Documents are needed to explain what is to be
• Practical price collection issues. For example:
done, when it should be done, how it should be
product identification/descriptions; pricing (item
done and why it should be done”. Reviewing the
descriptions, definition of price/sale price, rules
documentation also provides an opportunity to
relating to seasonal items, quantity conversions,
review the procedures.
quality adjustment ‐ when is an item or product
6.92 The issue of good documentation as part of equivalent?).
an integrated quality management system is
• The timetable and administrative arrangements.
addressed in Chapter 15. This section of the
Handbook deals specifically with the documentation 6.96 Practical examples and practice collections
needs of price collectors. But first it considers should be an integral part of the learning process.
training. For example there should be an opportunity for:
6.93 The training of price collectors should enable • Discussions about “equivalent” replacements
them to successfully perform all essential activities using photographs and item descriptions.
and deal with all potential situations including not
• Practice collections in office.
just recording the price of an item but, for example:
• Supervised practice collection in the field.
• Persuading new outlets to become price
providers.
62
PRICE COLLECTION
63
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
the price collectors’ work instructions and should Centrally prepared documentation will help ensure
cover: consistent practices in the field including between
regions. It should be readily available to all collectors
• Checking the quality of the price collectors’
and supervisors. The documents could be available
work.
in paper or electronic form, whichever suits the
• Checking the accuracy and completeness of the organisation, but whichever medium is chosen it
prices collected. needs to be accessible to the relevant staff.
• Official recording of resource use (for example, 6.110 All documentation should be kept up‐to‐date.
cars and bicycles for transport and funds for With paper‐based documentation this could mean
buying goods in markets). keeping the instructions in a loose‐leaf folder and
issuing individual updates. The amendment pages
• Official procedures for maintenance of should include version number and date printed and
resources (for example, testing the accuracy of should be kept to a reasonable number for ease of
scales). reference. A judgement will need to be made on
• Creating complementary collection timetables when a re‐print of a chapter or the complete
for all collectors within the supervisor’s area of working instructions is justified.
responsibility. 6.111 Further advice is given in Chapter 15.
6.109 Most of the documentation should be
prepared by Headquarters, with input as appropriate
from regional offices and fieldwork supervisors.
64
PRICE COLLECTION
Appendix 6.1 Example of a price collection form
65
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 6.2 Structured Product Descriptions and SPD forms
Structure Product Descriptions are hierarchical. The following description follows the format in which the SPD
might appear on a price collection form. An SPD form, used to pre‐specify the items to be priced, is given at
Appendix 6.1.
The SPD heading section
The heading of the SPD is Consumer Price Index: Structured Product Description (SPD). The month concerned
is also indicated (September 2008). The Collection Period, e.g. WEEK 1 may be pre‐printed. The Price
Collector should indicate the day on which the SPD form was completed, e.g. 05 as the Collection Date. Prices
should be collected at approximately the same day of the month every month from a particular outlet, for
comparability reasons.
The SPD form in this example is applicable to Woolworths. The unique number is a unique outlet identifier
assigned by Head Office. Quote 1 of 502 indicates that this is the first survey form applicable to Woolworths.
The total number of survey forms applicable to Woolworths is 502.
The Contact information section
This section contains the Contact Person (Adriaan Joubert), the Contact Number (015‐516 2620), and the Alt.
(Alternative) Contact Person (Marisa Louw) and the Alt (Alternative) Contact Number (015 – 516 2620).
The Brand and Quantity section
The Product Group is BREAD and the corresponding Product Group Code is 1101. The Product Subgroup is
LOAF OF WHITE BREAD and the Product Subgroup Code is 1101001. The commodity sub code is (1). This
means that this form relates to the first price quote. There is another form with the product subgroup
1101001 (2), which relates to a second price quote. Thus a product subgroup might have more than one
quote number and each will have a commodity sub code. This means that for that specific product subgroup,
the price collector needs to collect different items that fit to the specific product subgroup. For example, if
the Price Collector receives the following forms:
Quote Product subgroup code Product subgroup description
011 01112001(1) Loaf of white bread
012 01112001(2) Loaf of white bread
013 01112001(3) Loaf of white bread
66
PRICE COLLECTION
The Price Collector needs to look for three different types of loaf of white bread in the outlet that are volume
sellers. The Price Collector can initiate three different brands (Albany, Blue Ribbon, and Sasko) from the
outlet. It is important for the Price Collector to ensure that the same item is not initiated more than once.
The Brand Name and the Product Name of the selected item should be completed. Also, information
regarding the Quantity, Size, and Unit of size, should be completed. Finally, the Model Number, Style
Number and Outlet Product Number should be completed, if it is applicable to the item. Appliances, furniture
and spare parts often have a Model Number. Clothing has a Style Number. The Outlet Product Number is any
unique number assigned by the outlet that will assist in identifying the item in future months.
The Item characteristics section
Item Characteristics:
The Origin section
Origin: Domestic
Imported Indicate the country if imported:
Not Stated
The purpose of this section is to provide an indication of whether the item is produced locally or imported
from abroad. If the item is imported, the country of origin should be specified. The country of origin must be
stated on the item to confirm this. For example, the item must be written, Made in ….., produced in …..,
product of …… etc. No assumptions must be made. For alcoholic beverages it must be written “Distilled and
bottled in …” and for non‐alcoholic beverages it must be written “Bottled in ….”
The Further Remarks/Observations section
Further Remarks
/Observations:
In this section any additional comments that may help to identify the item in future months are recorded. For
example, comment on the colour of the packaging, the colour of the initiated clothing and footwear, etc. This
section cannot be left blank except for items sold loose or on open display.
67
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 6.3 Auditing of prices: example of monthly audit report
1. Audit information
Applicable Month: Month of Audit]
Province audited: Province Audited]
Region: [Region Audited]
Supervisor/s: [Supervisors Name]
Date of audit: [Date]
Collection period audited: Collection period on which audit took place]
The scope of the audit: [Type of audit]
2. Audit sample: [Province Name]
Outlet Types Quotes completed by Sample selected for Percentage sample
price collectors at audit against price collectors
outlet sample
Totals
No. of quotes completed by fieldworkers in collection period [State
collection period] (includes zero weights).
No. of quotes sampled for audit.
Percentage of quotes sampled.
3. Audit sample against target:
Target for audit sample [State target value]
1.2
1
0.8
0.6
%
0.4
0.2
0
e
e
e
at
at
at
at
at
at
at
D
D
68
PRICE COLLECTION
4. Errors allocated:
Team A: (Region –Supervisor Name)
Outlet name Number of Potential Individual potential error
completed errors Error Type percentage per outlet*
quotes
High Impact Non High
Impact
AUDIT ERROR PERCENTAGE [Type error percentage]
Note: Audit Target value
>0≤5 Green >5≤8 Amber >8 Red
*The number of total errors are divided by the overall number of potential errors
5. Breakdown of findings:
Team A [Supervisor’s Name]
High impact errors Numbers of Quotes with Errors
Wrong Price
Wrong initiation
Wrong Item
Field message
Available but indicated as temporary unavailable
Non High impact errors
Brand and product name
Wrong type of price
6. Error type graphical presentation
Date
1.2
Number of errors
1
0.8
0.6
0.4
0.2
0
e
n
ge
on
e
e
ze
m
..
ic
tit
m
ic
io
io
a.
ite
si
sa
as
pr
pr
na
an
at
at
ed
of
es
se
nt
g
iti
of
u
ng
ct
on
at
se
in
m
it
m
du
pe
ro
Feb Mar
ic
un
r
re
ng
ite
d
Ty
ro
el
tp
Apr May
in
ro
Fi
/P
ue
an
uc
t
Jun Jul
bu
nd
iq
ze
od
un
Aug Sep
e
ra
Si
Pr
bl
Oct Nov
ila
va
Corrections required:
69
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Correction to be effected by Next collection period [Date]
Review quality plan Date
Submission date Date
7. Accuracy:
[Give the status of quality control checks on the audit quotes]
8. Status of previous audit findings:
Regions – [Date of Audit]
1.2
1
0.8
0.6
0.4
0.2
0
ity
ge
e
e
ze
n
am
ic
tio
si
nt
sa
..
pr
tn
ua
te
of
es
tit
ng
uc
Q
as
it
in
ro
un
od
d
ed
ng
w
el
d
r
at
/P
Fi
ro
an
ic
nd
W
nd
ze
ra
Si
ti
B
bu
e
bl
ila
9. Discussions with price collectors
[State the outcome of the price collector interviews]
10. General
[Give general comments about the audit]
70
CHAPTER 7
Substitution and Quality Change
Introduction collection periods exactly the same product in the
same shop in the same location so that the CPI
7.1 A CPI should reflect the change in the cost of compares “like‐to‐like”. Also the recording of
buying a fixed basket of goods and services of detailed characteristics, especially price‐determining
constant quality. In practice this represents a features, can help when needing to make
challenge to the price statistician as products can adjustments to the recorded price due to changes in
permanently disappear or be replaced with new specification and hence quality. In practice, the
versions of a different quality or specification, and particular product being priced in a specific shop
brand new products can also become available. may become unavailable ‐ for example, the product
is discontinued, may be in temporary short supply or
7.2 Chapter 7 of the ILO Manual discusses the may be a seasonal product which disappears when
nature of “quality” and methods for adjusting prices out of season. The latter situation is a special case
for quality change. It provides detailed guidance and is dealt with in Chapter 9. In all other cases, the
both on implicit methods of quality adjustment, such price statistician needs to estimate the price of any
as the bridge overlap method and class mean missing product that he or she believes will return to
imputation, and on explicit methods including expert the market within a reasonable time, or, if the price
judgement, and the hedonic approach. Chapter 8 statistician believes it will not return, find a suitable
delves more deeply into the issue of item replacement. When the latter situation arises, the
substitution, particularly on methods of replacement should be either (a) as similar as
incorporating new products into the index. possible to the previous one or (b) the most popular
“similar” product in the shop or (c) the “similar”
7.3 Paragraphs 61 to 67 of the 2003 ILO
product that most likely will be available for future
Resolution advises that when a product disappears
pricing. Unlike approach (a) which leaves the sample
“clear and precise rules should be developed for
“static” with the danger that it will be increasingly
selecting the replacement product”. It lists three
out‐of‐date and difficult to collect prices for,
selection strategies: the most similar; the most
approaches (b) and (c) have the advantage of
popular; the most likely to be available in the future.
introducing an element of sample replenishment.
On quality adjustment it states that “when a quality
This is where quality adjustment (QA) comes in. The
change is detected, an adjustment must be made to
price index should reflect only pure price changes ‐
the price, so that the index reflects as near as
the price index should NOT reflect any portion of the
possible the pure price change”. It guards against the
price difference that is due to increases or decreases
automatic assumption that all price change is a
in quality between the missing item and its
reflection of the change in quality”. It does not
replacement. A value needs to be placed on any
recommend particular explicit or implicit methods of
change in specification between the old and
quality adjustment but does state that “the methods
replacement item and a quality adjustment applied
used should as far as possible be based on objective
accordingly. This applies to any replacement
criteria”.
strategy, but is particularly relevant where sample
replenishment takes place.
Background
7.5 Chapter 5 of this Handbook deals with
7.4 To measure price change from one period to procedures for making the initial sample selection
the next, the price statistician tracks, for each and covers issues relating to the selection, for local
elementary aggregate, the prices of a fixed sample of price collection, of representative products or
items. The detailed characteristics of the products, representative items on sale in outlets in a sample of
that is the varieties (both goods and services) locations chosen to be representative of the country
selected in the outlet or centrally by Head Office for as a whole as well as the sampling of centrally
pricing, are recorded to assist the price collector to collected prices. Chapter 6 deals with the associated
fulfil the aim of pricing in consecutive price issue of price collection. The treatment of entirely
71
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
new goods and how they are introduced into an selection. For instance, assume that shopping
index is covered in Chapter 8. Methods for dealing locations were initially selected from local
with seasonal goods are covered in Chapter 9. administrative areas, stratified by an
urban/rural split and shop type, and then
7.6 This current chapter provides guidance on outlets randomly selected from those outlets
three topics: within the chosen local administrative areas.
Then the relocating outlet can be followed to
• Substitution procedures where an item, product its new location if it continues to fall in
or outlet disappears, including the issue of new precisely the same stratum. However, if the
items. outlet moves away from the original shopping
location, for example from an urban shopping
• The imputation of a price when a product is district to a rural area outside the city or to
temporarily out of stock (excluding seasonal another urban district within the city, or if it
goods). relocates within the same shopping location
but becomes part of a multiple chain, then it
• Quality adjustment where a change of product has moved to another stratum and a suitable
involves changes in its price determining replacement for it should be found from within
characteristics. the original stratum, in order to maintain the
sample balance.
Substitution procedures
• Operational issues. As mentioned in Chapter 5
7.7 In a dynamic retailing environment there is a there may sometimes be operational reasons
continuous turnover both in outlets and in products. for departing from the sample generated by the
standard selection procedures. For instance,
Outlets efficient scheduling of price collection and the
7.8 If an outlet goes out of business or refuses to availability of price collectors may make
participate it should be replaced with the same sort following the outlet to its new location
of outlet (for example, a market stall should be impractical even though it has stayed in the
replaced with a market stall, a single shop with a same stratum. Similarly, a visit to the new
single shop etc.) in the same location, and location may indicate that it is impractical for
conducting the same type of business (in other the collection of prices, for example, because of
words selling the same types of goods). For example, a physical barrier such as a railway or river
if the previous shop was a butcher selling bisecting the area and causing difficulties of
refrigerated meat, then another butcher selling access.
refrigerated meat should replace it. If probability Products
sampling was used to select the original outlet, the
sampling frame should be revisited and a 7.10 If a chosen product is temporarily out of
replacement outlet selected from the same stratum. stock and no price is recorded, a note to this effect
Regardless of how the replacement is found, the should be made by the price collector. If it is out of
original outlet’s sampled items should be assigned to stock for, say, three consecutive months, then the
the replacement outlet for price observation. collector should be instructed to choose a
replacement which matches as closely as possible
7.9 If an outlet changes location, a decision on the product description unless it is decided to take
whether the price collector should follow the outlet the opportunity of a disappearing good to update
to its new location needs to take into account both the sample. For a product temporarily out of stock a
sampling and operational issues: price has to be imputed. Where a product is
• Sampling issues. The principle of maintaining a permanently unavailable for pricing, procedures
like‐for‐like comparison holds. In practice the need to be in place for determining a replacement
balance of the sample needs to be maintained and then impute a new base price. Methods for
in order to ensure that it continues to be imputing a missing price will be discussed shortly.
representative. Stratification is frequently used 7.11 As the issues relating to temporarily and
to increase sampling efficiency and ensure that permanently missing products differ ‐ and their
the sample is representative. This means that treatment is different ‐ it is important for the price
when an outlet changes its location, reference collector to establish whether the unavailability of a
needs to be made to the stratification and product is likely to be temporary or permanent. A
selection procedures used in the initial sample price may be considered as temporarily missing if
72
SUBSTITUTION AND QUALITY CHANGE
the same product is likely to return to the market is appropriate only if there is reason to believe
within a reasonable time period. This includes that the price has not changed. Typically it will
seasonal items for which special procedures apply be difficult for the price statistician to validate
(see Chapter 9). Permanent unavailability, on the the belief that the price has not changed.
other hand, occurs when a variety is withdrawn from
the market with no prospect of returning in the • Imputation. Carrying forward a price may be
same form. Products may be temporarily missing better than simply adjusting the sample
due to supply shortages caused by factors such as composition to ensure comparability (matched
the seller underestimating demand, strikes by pairs) but this is not guaranteed, and can be
factory or transportation workers, or supply better than comparing the prices of products
problems with imported products. In these cases the which are dissimilar in terms of price‐
price collector, although not able to observe a price determining characteristics, although this
in the current period, may have obtained cannot be guaranteed. Nevertheless, the best
information (e.g. from the shopkeeper) to suggest solution by far is to impute a price. Imputation
that the same variety will become available again at makes use of the best available information to
some, perhaps unknown, time in the future. provide an unbiased estimate of price
movement. The principal methods for imputing
7.12 The above discussion does not cover seasonal prices are shown in more detail in Chapter 9
products i.e., where a product or item may Table 9.2(b) of the CPI Manual. There are
disappear because it is a seasonal product and it may essentially two choices:
be expected to reappear when it is next in season.
The latter is covered in detail in Chapter 9. The o Impute the missing price by reference to
problems of imputation are fairly similar for both the average price change for the prices that
temporarily missing and seasonal products. are available in the elementary aggregate
(overall mean imputation). This assumes
that the price change of the missing
Temporarily (non‐seasonal) missing
product, if it had been available in the
products shop, would have been equal to the
average change in prices in the elementary
7.13 If it is believed that a missing product will be
aggregate. This may be a reasonable
available again in the future then the price
assumption if the elementary aggregate is
statistician has four options:
fairly homogeneous. This method of
• Omit the product for which the price is missing imputation is equivalent to the “omit”
so that a like‐for‐like comparison is maintained method (see first bullet point), no matter
using matched pairs. The elementary index uses whether a Jevons, Carli or Dutot method of
only those observations for which the price aggregation is used at the elementary
collector obtained prices of exactly the same aggregate level.
variety in the current and previous periods. In o Impute the missing price by reference to
this approach the price change for the deleted the average price change for the price of a
product, which was recorded up to the point “comparable” item from a similar outlet
immediately before its disappearance will be (class mean imputation). This represents a
disregarded from that point on. This may cause more precise match between the missing
problems, for example, if it unbalances the product and the products supplying an
sample. imputed price. It is normally preferable to
• Carry forward the last observed price. Although imputation using the average price change
this provides a price continuity in the periods in the elementary aggregate unless the
when observations cannot be made, it is likely imputations are unreliable because of
that short‐term movements in the index are small sample sizes.
biased, since the sub‐indices in question will 7.14 A detailed discussion of the latter two
show no change when prices are not available. If methods is presented later in the chapter.
prices in general are rising, the bias will be
downwards, whereas if prices are falling, the
bias will be upwards. Carry‐forward is not
Permanently missing products
recommended, particularly when there is high
7.15 When the situation arises where a product
inflation or when period‐to‐period movements
permanently disappears or is replaced by a new
(as opposed to annual movements) in the price
index are important. The carry‐forward method
73
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
7.16 In practically every period for which a price 7.19 Replace with the most similar product. This
index is compiled, some varieties of a product reduces the role of quality adjustment, as the more
become permanently missing, not just in particular similar a product is the less is the required quality
outlets but also because they are no longer adjustment (see later paragraphs), but contributes
produced or imported. If no action is taken, the to the deterioration in the representiveness of the
sample of prices will diminish. The latter is sample where a product starts disappearing from
problematic not just because of the potential impact the shops because it is being replaced by something
on how representative the sample is but also new and sales are declining. Finding the replacement
because it will lead to estimation of price change that is most similar to the original variety requires
with samples that do not match from period to knowledge about characteristics of the previous
period, i.e. the composition of the matched pairs variety. Good practice in price collection involves
change. In addition, the index number for the latest maintaining up‐to‐date descriptions of the variety’s
month will be less reliable than that for the previous characteristics. The International Comparison
month because of the smaller sample size. Program (ICP) developed “structured product
descriptions” (SPDs) for most item categories: these
7.17 Note also the potential problems with the provide a framework to list the various
estimation of average prices and changes in these characteristics — prioritized in order of their
averages where these are published. For example, importance — of the varieties for each category.17.
using arithmetic averages, if prices for three varieties Such descriptions of characteristics can be used to
(A = $3, B = $3, C = $2) were available in the previous match the characteristics between the old variety
period giving an average price of 2.67, whereas in and various replacements so that critical
the current period prices are available for two characteristics are matched and less important
varieties only (B = $3, C = $2), the average price will characteristics can be noted. Critical characteristics
fall to $2.50 even though none of the variety prices are those that contribute to the price determination,
for which prices are available in the two periods has such as type of product (canned tuna fish), brand
actually fallen. The average price will have fallen (Star Kist), size (150 ml), and packaging (tin, in
purely due to a change in the sample composition, water). Less important characteristics are those that
the same also happening when a geometric mean is do not affect the price, such as colour of the label on
used. The arithmetic mean example is shown in the package or the location in the shop where the
Table 7.1. Further discussion on the publication of product is displayed. The salient considerations can
average prices is given in Chapter 14. be incorporated into a decision‐making framework
7.18 Because of the problems referred to above, for identifying a “similar” product, as follows:
the price statistician’s task is to maintain the sample • There is a basic match of the main
size by finding replacements for the specific varieties characteristics, including, and most particularly,
when they are no longer available and are not those which determine price.
expected to return within a reasonable time. One of
two alternative strategies can be adopted. Under • Consumers perceive them as similar even
both options it is important to identify any though some characteristics may be different.
differences in quality between the original and
replacement varieties as it is crucial to ascertain 17
See the ICP 2003–2006 Handbook on the World Bank ICP
whether there is a quality difference, and, if there is, website at http://go.worldbank.org/MW520NNFK0.
74
SUBSTITUTION AND QUALITY CHANGE
This may be the variety in the shop that buyers
are most likely to buy in place of the original. 7.23 In all cases, a nominal price in the base month
is needed for the new or replacement product ‐ this
• They are used for the same purpose and in
nominal base price is used until the next rebasing.
similar situations. For food this may include a
consideration of whether the brand is one for
7.24 The price collector should record the
everyday consumption or only for special
specifications (the price‐determining characteristics)
occasions.
of the new variety so that Head Office can determine
7.20 Under the “replace with a similar product” if the replacement is of similar quality to the original
strategy, an updated version of a product, i.e. the variety. The price collector should determine if the
one that the supplier lists as the replacement, is the shop is likely to continue selling the replacement. If
logical starting point for the replacement for its it is also expected to be discontinued in the near
predecessor. In most instances this would be the future, then a different variety should be selected ‐
one that is the most similar to the original variety, so either another that is similar to the first, or the most
the price statistician can compare “like‐to‐almost‐ popular variety within the product line. As will be
like”. In instances where the most similar variety is seen, there are several different methods for both
also one that is likely to soon disappear, the price the explicit and the implicit types of quality
collector should select the variety that is most adjustment, but there are some common themes in
popular within the outlet for the product class. the methods of each of these two main types.
Although this is less likely to yield a replacement that Explicit methods estimate the impact on price of
is sufficiently similar to permit direct price changes in characteristics or features of the product.
comparison, it will reduce the likelihood of the Implicit methods estimate the impact on price
replacement disappearing in the near future and will indirectly by reference to price differences between
keep quality adjustment to a minimum. different varieties.
7.21 Sample replenishment ‐ replace the missing
product or variety with the currently most Direct comparison
representative one by going through a process of re‐
sampling. The extent to which a sample remains 7.25 If the selected replacement product is
representative is highly dependent on the rules used regarded as comparable, then the observed price
for item replenishment when a particular variety or change is treated as a pure price change. But the
product disappears from the shelf of a particular price statistician should gather and examine all the
outlet. Compared with the option of replacement available evidence, if possible taking advice from
with the most similar product, sample replenishment market experts where necessary, before coming to
has the benefit of maintaining the up‐to‐dateness of such a conclusion. Even in cases where a
the sample. If disappearing products are always replacement product is believed to be of comparable
replaced with similar products the sample will quality, care should be taken, since experience
gradually become less relevant to market reality. suggests that most goods tend to undergo steady
Sample replenishment also increases the chances of improvements, especially hi‐tech and electrical
the replacement being available on the shop shelf goods.
for pricing. The problem of a deteriorating sample
increases with the rate of turnover in varieties and Direct or explicit quality adjustment
products and with the rate of product development.
methods
Quality adjusting the price for 7.26 In explicit methods, the monetary value of
differences in quality the quality difference is determined directly and
then applied to a previous period’s observed price
7.22 When a product disappears or is replaced for the “old” item. This will yield an estimate of what
the replacement item would have cost in the
with a new version of a different quality or specifica‐
previous period.
tion, then one of the following methods of
introducing the price of the replacement is adopted: 7.27 Assume that the statistical office was able to
determine that, based on the differences in
• Direct comparison. characteristics, the value of the quality difference
between Brand C and Replacement 1 was $25 in
• Direct (explicit) quality adjustment. period 1. The price statistician can add this amount
• Implicit quality adjustment (imputation). to the price of Brand C in period 1 to obtain an
75
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 7.2 Explicit quality adjustment
Variety Price Index Price in Price in Price Price Index
Period 1 Period 1 Period 2 Ratio 2/1 Period 2
Brand A 125.0 150 160 1.067 133.3
Brand B 150.0 225 250 1.111 166.7
Brand C 125.0 140 –
Replacement 1 (165) 180 1.091 136.4
Elementary Aggregate Index 132.83 1.0895 144.7
(geometric mean)
estimated price for replacement 1 of $165 ($140 + market information and is also liable to
$25) as shown in Table 7.2. subjectivity. The judgement needs to be
properly informed.
7.28 The price relative for Replacement 1 can be
derived ($180 / $165 = $1.091) and used to calculate • Reference to the producer cost. The production
the price relative for the aggregate index ([1.067 x cost from the producer can be used in the case
1.111 x 1.091]1/3 = 1.0895). The period 2 indices for of an improvement to an existing product,
the elementary aggregate and for Replacement 1 are although a judgement then needs to be made
144.7 (= 1.0895 x $132.83) and 136.4 (=$125.0 x on whether to apply an adjustment factor. For
1.091), respectively. example, the latter might consist of the normal
retail mark‐up to reflect wholesalers and
7.29 There are a number of direct or explicit
retailers costs and profits. In the context of a
methods for determining the price associated with a
cost‐of‐living index a downward adjustment
change in quality. But quality adjustment is not an
may be appropriate to account for the fact that
exact science, and different evaluations can yield
the new “standard” feature will not increase
different results. The point is that even if the
the utility to all purchasers, e.g. some may not
evaluation methods used are somewhat imprecise in
welcome air conditioning in a car because of
the measurement of the quality difference, it is
the extra running costs. Compilers of the
important to make an adjustment. If quality
Producer Price Index often attempt to gather
differences are not removed, the price index will
information on production costs from
reflect quality change in addition to pure price
manufacturers for quality adjustment purposes.
change and is likely to have an upward bias because
However, producers may be unwilling to
quality usually increases.
provide information on their marginal costs for
7.30 Figure 7.3 of the CPI Manual provides a reasons of confidentiality. There are a number
flowchart of the decisions needed for making quality of potential problems with using the producer
adjustments. This is reproduced as Appendix 7.1 cost method. In particular, it is not necessarily
here. the case that the cost of production, with an
adjustment factor along the lines describe
7.31 The most common ways of determining an
above, gives a good indication of the selling
explicit quality adjustment in practice are as follows:
price18.
• Package Size Adjustment where the value of a
• The former “option” price. In the case where an
change in package size is assumed to be
optional feature has become standard, the
proportional to the relative change in the
former price charged for the optional feature
package size.
can be used as the explicit quality adjustment
• Expert judgement. This is where the price
collector determines the value either through 18
In constructing a consumer price index, consumer valuations of
direct knowledge or in consultation with products should be decisive and the cost of production is
irrelevant in this context. In “equilibrium” situations, the cost of
personnel in the shop where the product is
production (using a “normal” cost of capital in the producer’s user
sold. Alternatively, statistical office staff, who cost formula) should give a close approximation to the selling
may be knowledgeable about the product, can price (and hence the consumer’s valuation of the product). But it
estimate a value. Reliance on an individual's is in exactly disequilibrium situations, involving new products
where R&D has to be amortised using monopolistic prices where
knowledge concerning the products depends,
new products and obsolescence of old products is likely to occur
however, on the individual obtaining sufficient that the assumptions of equilibrium breakdown.
76
SUBSTITUTION AND QUALITY CHANGE
value but again, consideration will need to be models simultaneously available in the market is
given to whether a scaling‐down factor should equal to the difference in price between the
be applied – in this case possibly a downward models20. Thus when a product disappears from the
adjustment to reflect the reduced production shelf, an underlying assumption is made when
costs of making a feature “standard” and also, imputing a price that a price differential continues to
in the cost‐of‐living context, the argument, exist to reflect a difference in quality. Examples of
presented earlier, that the utility gain is less the different methods of imputation available to the
than the increase in price. Some of the index compiler are given in the next section.
concerns relating to using producers’ costs (see
previous bullet point) apply, the main difficulty 7.33 Most countries construct some form of a
being that it is likely that the market valuation fixed‐base price index. If price movements are
of the options will have changed once they estimated using long‐term price relatives from the
become “standard”, indeed, it is often because base period, then the base price may be adjusted
of changing market circumstances that proportionately for the estimated quality difference.
producers make former options “standard”. If price movements are estimated using short‐term
relatives from the previous period, then an
• Hedonic regression. Another way to obtain a
imputation adjustment can be implicitly made by
value of the quality difference is to use hedonic
estimating the missing variety’s price in the current
regression to estimate the value of changes in a
month from the average price change in its
product’s characteristics. See Chapter 7 of the
elementary aggregate. The price change of the
CPI Manual and also Triplett’s Handbook on
omitted observation is equal to the change in its
Hedonic Price Indexes.19 Hedonic methods
elementary aggregate. Thus, imputation can be
require large databases with a wide range of
applied to either the long‐term or short‐term price
product characteristics. Such databases are
relative. Some examples of adjustments using short‐
seldom available in statistical offices (though
term price relatives are presented in more detail in
for some products scanner data from large
the next sections. Examples of adjustments using
stores may be a viable source in some
long‐term price relatives are given in paragraphs
countries) and can involve substantial
7.2.1 to 7.2.9.
development and maintenance costs. In
addition, hedonic models need to be re‐
estimated periodically. Hedonic methods Examples of implicit quality
should be applied only where they add adjustment methods
significantly to the statistical integrity of the
index. This is most likely to be the case with hi‐ 7.34 Consider the case in which the Jevons index
tech high turnover goods. formula (the un‐weighted geometric mean of the
short‐term price relatives) is used to estimate the
Implicit quality adjustment price index for an elementary aggregate which, as is
7.32 If the replacement product is of a different the usual case, has no explicit expenditure weights:
quality or specification and no information is 1n
It = ∏ (pi ,t pi ,t −1 ) × It −1
n
available to quantify the difference, then
assumptions have to be made about what
proportion of the price difference is accounted for i =1
by differences in quality. Implicit quality adjustment Where It is the current (Period t) elementary
creates an imputed “quality‐adjusted” price based aggregate index, pi,t is the current price of variety i,
on price changes from similar varieties of the
pi,t‐1 is the price of variety i in the previous period
product. The precise nature of the imputation
(Period t‐1), and n is the number of varieties for
depends on the index formula that is used. A
common basic assumption underlying the most
commonly used implicit quality adjustment methods
20
is that the difference in quality between varieties or For the argument to be valid it has to be assumed that the
consumers are rational in their thinking and well informed about
available alternative choices. It also has to be assumed that the
19
Jack Triplett, Handbook on Hedonic Indexes and Quality market is “in a state of equilibrium”, i.e. in a state that would
Adjustments in Price Indexes: Special Application to Information persist indefinitely if all given conditions remained unchanged. In
Technology Products, STI WORKING PAPER 2004/9, Statistical practice these assumptions may be adequate or inadequate to
Analysis of Science, Technology and Industry; OECD Publications, varying degrees in different product areas. Equilibrium conditions
2
rue André‐Pascal, 75775 Paris, Cedex 16, France. Available at may be disrupted by temporary imbalances between supply and
http://www.oecd.org/dataoecd/37/31/33789552.pdf demand, such as stock clearances, bad harvests or scarce, much‐
demanded new models
77
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
which prices are collected in both periods for the whether the old variety is available only at clearance
elementary aggregate. prices. This often occurs when only a few are
available and the shops in which they are sold have
7.35 This is the short‐term formula that computes
deep discount prices in an effort to sell quickly the
the current index by multiplying the previous index
remaining stocks of the old variety. A careful
by the average short‐term change.
evaluation must be made as to whether such
7.36 The other approach used frequently is the discount prices adequately reflect the quality
geometric mean of the long‐term price relatives, difference. Some commentators have also
which uses a short‐term price relative to impute the questioned the underlying assumption that all the
missing price for the quality comparison, and the price difference, whether relating to clearance or full
base price is adjusted to include an estimate of the price, is accounted by quality differences.
change in quality. This approach is discussed in detail
7.39 One effective use of the overlap method is in
in Appendix 7.2. The principles of implicit quality
the updating of samples that contain too many
adjustment are the same whether short‐term or
obsolete items, a topic which is discussed in Chapter
long‐term price relatives are used. The different
8.
approaches are discussed below using the short‐
term price relatives for illustrative purposes. 7.40 The example in Table 7.3 illustrates the use of
the overlap procedure.
Overlap pricing
7.41 The sample for this elementary aggregate
7.37 This method requires knowledge of the prices consists of three varieties. When the price collector
of the two varieties in the same time period ‐ the visits the shop in period 2 to collect the price for
overlap period. If the old and new varieties are Brand C, they find that it is permanently unavailable.
available at the same time and the price collector The shop manager and the price collector determine
either knows in advance that the old variety will that Replacement 1 with a price of $180 is the most
disappear soon and selects and prices a similar variety to Brand C. Review of the differences
replacement, or the outlet is able to accurately give between the varieties indicates that Replacement 1
the price of the replacement in a previous period has extra features and is not fully comparable in
when the old variety was priced, then the price quality to Brand C. The shop manager was able to
difference between the two is taken to be the value determine, however, that Replacement 1 was
of the quality difference. The rationale is that it available during the last price collection period and
cannot be price change ‐ price change occurs only its price was $160. The price change for
over time. The price index uses the old variety in the Replacement 1 ($180/$160 = 1.125) can now be
overlap period and the new variety in the next used to calculate the index this month. Using the
period and the price differential between the two geometric mean the price change for the elementary
varieties never affects the index. In this case, the aggregate can be derived to be (1.067 x 1.111 x
market has determined the value of the quality 1.125)1/3 = 1.101. Applying this to the previous index
adjustment. for the elementary aggregate yields its current
7.38 In reality, overlap pricing can only be applied period value of 146.2. The quality difference that has
in a minority of cases. Prior warning of a product been excluded from the index is the price differential
disappearing from the shelves is rarely given and between Replacement 1 and Brand C ($160 – $140 =
obtaining a price relating to an earlier period can be $20). If Replacement 1 had been compared directly
difficult. In addition, it is important to ascertain to Brand C ($180 / $140 = 1.151), then the item
index would have risen to 152.9, significantly
Table 7.3 Quality adjustment using the overlap method
Variety Price Price in Price in Price Price
Index Period 1 Period 2 Relative Index
Period 1 2/1 Period 2
Brand A 125.0 150 160 1.067 133.3
Brand B 150.0 225 250 1.111 166.7
Brand C 125.0 140 ‐‐
Replacement 1 ‐‐ 160 180 1.125 140.6
Elementary Aggregate Index (geometric 132.83 1.101 146.2
Mean)
78
SUBSTITUTION AND QUALITY CHANGE
Table 7.4 Quality adjustment using overall mean imputation
Variety Price Price in Price in Price Price Index
Index Period 1 Period 2 Relative Period 2
Period 1
overstating price change. Alternatively, had the price variety (Brand C), an imputed price relative between
statistician assumed the price of Brand C remained period 1 and period 2 is calculated using the
$140 (linked to show no change); its imputed index geometric mean of the price changes (price
would be $140.6, a substantial understatement. relatives) of the two varieties (Brand A and Brand B)
for which actual prices were collected.21 This yields
7.42 Note that if the long‐term price relative
(1.067 x 1.111)1/2 = 1.089. This is multiplied by Brand
formula is used to compute price indices, the base
C’s period‐1 price of $140 to get an estimated
price must be adjusted by the ratio of the new
(imputed) period‐2 price of $152.4. The amount of
variety’s price to the old variety’s price in the
the quality adjustment is the difference in period 2
overlap period and the adjustment for quality
between the price for Replacement 1 and this
change is equal to the ratio of their prices in the
estimated price ($180 – $152.4 = $26.6). Note that
overlap period (see Appendix 7.2).
this procedure provides an estimate for Brand C
Overall mean imputation (sometimes from matched pairs. Replacement 1’s price will not
referred to as the non‐class mean or, in the be used until the next period (period 3). The price
relative for the elementary aggregate is the
context of the European Harmonised Index geometric mean of the price relatives of the three
of Consumer Prices, the bridge overlap varieties (i.e. including the price relative for the
1/3
method) imputed variety) (1.067 x 1.111 x 1.089) = 1.089 as
shown in Table 7.4. Applying the aggregate’s price
7.43 Overall mean or non‐class mean imputation
relative to its period‐1 price index yields the period‐2
(also referred to as “linking” or “splicing”) imputes
index. (Note that the geometric mean of the
an overlap price for the old variety in the current
varieties’ price indices yields the same result‐ (133.3
period by reference to the price changes between
x 166.7 x 136.1)1/3 = $144.6)
the previous and current periods of similar items.
7.47 The overall mean procedure assumes that the
7.44 The latter are used to impute a price change
pure price change from the replaced item to the
for the old variety, which can then be used to obtain
replacement item is the same as that for the
the imputed price. The ratio of the imputed price for
composite of all other goods in the same group. It is
the old variety and the price of the new variety in
used frequently because of its simplicity, but there
the current period is the estimated quality
can be an inherent bias built into the methodology,
adjustment.
particularly when major model changes are
7.45 An estimate of the price of the missing variety occurring. The direction and extent of this bias is a
can still be made even though the price of the matter for debate but depends on whether the true
replacement variety may not be known for the quality adjusted change in price is bigger or smaller
previous pricing period. The example in Table 7.4 than the measured price changes of the items used
illustrates how to make such an estimate and to in the imputation. Major price changes can
implicitly adjust for quality differences using the frequently occur at the time new varieties or models
overall mean price change for the elementary of a product are introduced. This is quite common,
aggregate. for example, with new vehicles, household
appliances, electronic equipment, and clothing
7.46 To derive the required price relative the price
items. As the new varieties are introduced there may
statistician can estimate the price of the original
variety by multiplying the price of Brand C in the 21
The same result is, of course, obtained using the price relative
previous period by the average price change of the
of the geometric average prices:
other varieties available this month. For the missing 1/2 1/2
(160 x 250) / (150 x 225) = 1.089.
79
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 7.5 Class mean imputation where one replacement is not comparable
Variety Price Price in Price in Price Price
Index Period 1 Period 2 Relative Index
Period 1 2/1 Period 2
still be a substantial supply of the old varieties, become obsolete, while the new models
which are showing little price change or may actually (represented by the replacements) tend to have a
be declining in price. In consequence, using the old higher price before falling. This may not be such a
varieties’ price changes to impute the price changes big issue in developing countries where new
for new ones will underestimate the true price products appear in the retail market relatively late at
change for the new varieties and cause a downward a “mature” price. Using class‐mean imputation adds
bias in the price index. The use of the overall mean complexity, but reduces two types of bias referred to
imputation procedure, in which all observations in earlier: bias from ignoring quality change altogether
the elementary aggregate are used, is not and treating all price movements as price change;
recommended for such cases. It is also possible, and bias from over over‐adjusting for quality change by
can be argued (but less convincingly) that the use of treating some pure price change as quality change.22
the overall mean leads to an upward bias because
7.49 The previous examples assumed that there
the price changes associated with models that are
were no differences among the product varieties
unchanged in quality will be further along the
within the elementary aggregate. Now assume that
evolutionary cycle and therefore will be rising less
Brand A is a variety that was available in both
rapidly An alternative imputation procedure, called
periods and that Brand B is a model that is new in
“class mean” imputation, avoids some of the
period 2 and that was deemed comparable in quality
problems associated with bias.
to the previous model priced in period 1. The
Class mean imputation replacement for Brand C (Replacement 1), however,
is deemed not comparable in quality to Brand C, so
7.48 Class mean imputation is similar in procedure class mean imputation uses only Brand B’s price
to the overall mean imputation but uses only the change to impute a price for Brand C. It does not use
price changes of “comparable” replacements to Brand A’s price change, because Brand A is an old
impute the overlap price, the replacements being variety and is therefore likely to have a price change
limited to those that have exactly the same price‐ which differs from new varieties such as Brand B. In
determining characteristics, or those items with this case the price relative is 1.111 and the imputed
replacements that have been declared comparable price, $155.6. See Table 7.5.
after review or have already been quality‐adjusted
through one of the "explicit" methods. For example, 7.50 The amount of the implicit quality adjustment
when the arrival of a new model of a particular make is the difference between the prices for Replacement
of motor vehicle forces price collectors to find 1 and Brand C ($180 – $155.6 = $24.4). In general,
replacements, some of the replacements will be of class mean imputation can be used with any
comparable quality, others can be made comparable targeted group of observations. It is often found that
with explicit quality adjustments, but the remaining some varieties are available all year with little price
ones will need imputed prices. Class mean change. They maintain constant levels of quality
imputation calculates imputed price relatives using from period to period. At the same time, other
only the prices of comparable and, where varieties go through model change, where new
appropriate, explicitly quality‐adjusted varieties or models are introduced periodically throughout the
models. In general, it does not use the prices for the year. Sellers normally increase the price when they
varieties or models that were not replaced, because
these are likely to be different from those of new 22
Clothing, items usually require additional procedures to control
models. The prices of old models tend to fall as they substitutions depending on the season of the year, e.g.,
autumn/winter versus spring/summer clothing.
80
SUBSTITUTION AND QUALITY CHANGE
Table 7.6 Quality adjustment using matched models method
Variety Price Price in Price in Price Price Price in Price Price
Index Period Period Relative Index Period Relative Index
Period 1 1 2 Period 1 Period 2 3 Period 2 Period
to 2 to 3 3
Brand A 125.0 150 160 1.067 133.3 170 1.063 141.7
Brand B 150.0 225 250 1.111 166.7 265 1.060 176.7
Brand C 125.0 140 – (1.089) (136.1) – – –
Replacement 1 180 – 136.1 200 1.111 151.2
(for Brand C)
Elementary 132.83 1.089 144.6 1.078 155.8
Aggregate Index
(geometric mean)
81
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
and resulted in a change at the elementary those areas where it is not available, the product,
aggregate level of 15.1% ([1.067 x 1.111 x 1.286]1/3 = along with its weight, could be dropped from the
1.151). The quality adjustment methods have, sample, so that it does not enter into the index
therefore, removed the price statistician’s best calculation. The disadvantage of this method is that
estimate of the monetary value of the quality change the product cannot be added back to the sample
(given the information available) from the overall easily once it is dropped. It will have to be treated as
price difference, so that the price index measures a new product and added using the methods
only pure price change. Adjustment for quality discussed in Chapter 8 of this Handbook. Thus, for
change is an art for price index practitioners but practical reasons, the statistical office should be very
ignoring quality change can result in a substantial cautious about dropping products from the sample if
overstatement of price change, for instance when there is a likelihood that they might return.
price increases due to quality improvements
7.59 The price of a missing product should be
associated with new products are included in the
imputed from period to period using the price
price index. On the other hand, attempting to
change of the class, subgroup, or group in which the
correct for this problem by the use of a link‐to‐show‐
product belongs (that is, the price change at the next
no‐price‐change method when replacement varieties
highest level in the aggregation structure). The CPI
are introduced can result in over‐adjustment for
Manual especially warns against using the carry‐
quality change (with some pure price change being
forward technique (repeating the last available
counted as quality change and being excluded from
price) under these circumstances (see paragraphs
the price indices) and thus an understatement of
7.71 and 9.50 of the Manual). Repeating the last
price change. The bias can go either way.
available price assumes that there has been no price
change for the product, when, in fact, it is known
Replacing products during periods of that prices are changing where the product is
supply shortages available.
7.57 Most statistical offices have rules regarding 7.60 In cases of severe supply shortages, a key
the length of time that they will estimate a price for product may be missing in all areas of the country. In
a missing product. Usually this is between 2 and 6 such circumstances, the statistical office will need to
collection periods, after which time they assume the assess whether the product is likely to return
product is permanently missing. This is a good rule sometime in the future. If it appears likely to return,
of thumb to follow. During extended periods of then the standard waiting period for selecting a
supply shortages, however, it can be problematic. substitute could be suspended, and its price imputed
The shortage may result in long queues of buyers from other prices for many periods. If it appears that
trying to purchase the products and, possibly, a the product will never return, then it should be
parallel (black) market where the prices are treated as permanently unavailable and a substitute
considerably higher. The prices recorded in the CPI product selected for price collection. If there is no
should ideally be those actually paid by consumers, substitute product available, then the product and
so prices should reflect those both in regular and, its weight will need to be dropped from the index
where practical, in parallel markets. The fact that calculation. If it has permanently gone and no
prices for products experiencing a shortage may rise reasonable substitute is available, then the product
rapidly under these circumstances is not a reason to should have no part in the calculation of the CPI. It is
exclude them and their price changes from the CPI. wise, however, to maintain the sample size by
In practice many CPIs follow the prices in a sample of replacing the item with another relevant product in
shops, making it hard to add prices from new (and the same category.
perhaps less legitimate) sources.
7.61 Data from recent household budget surveys
7.58 Products may not be available in specific or retail sales information could be used for
markets or in certain regions of the country, and it identifying additional products if data are collected
may not be known when or if the missing products in sufficient detail. However, the statistical office
will return to these markets. As long as the product should be careful about using weights from these
is available in some markets, the statistical office sources to update the weights in the CPI because the
should consider imputing the price of the missing weights should reflect spending patterns in a normal
product based on the price trend in a neighbouring or stable period (see the CPI Manual, paragraph
area or based on the change in the national average 4.44) and avoid unusual periods such as those with
price calculated from those areas where the product shortages.
is available. When the product returns, prices may
again be collected in those areas. Alternatively, in 7.62 A major concern in these situations is that the
sample size may diminish significantly, with a
82
SUBSTITUTION AND QUALITY CHANGE
corresponding decrease in the precision of the index. target sample. Thus, for each published index series
Very few countries are able to produce estimates of at the class, group, or division level where the
variance and bias associated with a declining sample, sample falls below this threshold, the statistical
but a simple rule related to sample size may be office discontinues publication until the sample size
helpful to determine which index series should can be increased. When published series are
continue to be published under these circumstances. suppressed, the statistical office should notify users
A number of statistical offices use a 50 percent rule that the reason for not publishing the series is due to
whereby they discontinue publication of CPI series in diminishing availability of representative products
which the sample size falls below 50 percent of the and sample of prices.
83
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 7.1
Flowchart for making decisions on quality change
/
NO
YES Has the specification changed?
YES
Expert judgement
YES
Is a replacement
Can the quality difference be explicitly Available?
quantified?
Quantity adjustment
NO
YES - Use an explicit
adjustment
Production cost
Hedonic techniques
NO
YES
Carry-forward
84
SUBSTITUTION AND QUALITY CHANGE
Appendix 7.2 Adjustments Using Long‐term Price Relatives
7.2.1 The discussion and examples in the main valuing the quality difference as $25 in period 1. We
body of the chapter assume that the statistical office compare the period 1 price of Replacement 1, which
is using the short‐term price relative index formula has the new quality, to Brand C with the old quality
to calculate the elementary aggregate. Many ($140). (Again, this comparison is made for the first
countries use the long‐term price relative formula: period in which the prices of brands with the two
1n
qualities are available.) The QA factor is 1.1786
It = ∏ (pi ,t pi ,0 ) × 100
n
($165 / $140), which is then applied to the Brand C
base price ($112) to derive the new base price of
i =1
$132. This base price is used in the calculation of the
Where It is the current elementary aggregate index, period‐2 long‐term price relative for Replacement 1.
pi,t is the current (period t) price of variety i, and pi,o is Table 7.2.1 demonstrates the compilation of the
the price of variety i in the base period (period o). elementary aggregate index using the long‐term
relative formula.
7.2.2 Since there typically are no expenditure
weights within elementary aggregates, each 7.2.5 In the overlap example – see Table 7.1.2 ‐
observation has the same weight, 1/n, where n is the Replacement 1’s period‐1 price ($160), which
number of varieties for which prices are collected includes the cost of the new quality, is the price
within the elementary aggregate. The long‐term which is compared with the price including the cost
formula is an average of long‐term (current period to of the old quality, which is Brand C’s period‐1 price
base period) price relatives. ($140). The quality adjustment factor of 1.14286 (=
$160 / $140) is applied to the old quality base price
7.2.3 Compare this formula to the short‐term ($112) to derive the new base price of $128, which is
formula that uses the average of short‐term (current used in the calculation of the period‐2 long‐term
to previous period) price relatives. Quality price relative for Replacement 1. Table 7.2.2
adjustment using the short‐term relative is easier demonstrates the compilation of the elementary
because the price statistician is estimating the value aggregate index using the long‐term relative
of quality difference between two varieties observed formula.
in two adjacent periods ‐ the adjustment is made
either to the current or previous period to get the 7.2.6 In the overall mean example period 2 is
correct short‐term price relative. The long‐term used for the quality comparison: Replacement 1’s
(current to base period) formula, in which the price period‐2 price ($180) includes the cost of the new
relative of each observation is an estimate of its quality and Brand C’s Period‐2 estimated price
change in price from its price in the base or ($152.4) includes the cost of the old quality. (Note
reference period, requires that quality be kept again that the comparison is made using the same
constant over this long period. This is more difficult time period; it would not be valid to compare
to do. It requires the assumption that the relative Replacement 1’s price in period 2 to Brand C’s price
quality difference is the same in the base period as in period 1 because the price of Brand C, if it were
at a later ‐ often much later ‐ period. Quality available, would most likely have changed similarly
adjustment with the long‐term relative formula calls to the other prices within the elementary
for the statistical office to calculate a quality aggregate.) The quality adjustment factor of 1.1811
adjustment factor equal to the ratio of the price (=$180 / $152.4) is applied to the old base price
(either collected or imputed) of the new‐quality ($112) to estimate the new base price ($132.3).
variety to that of the price of the old‐quality variety Table 7.2.3 demonstrates the compilation of the
in the same time period and then apply the factor to elementary aggregate index using the long‐term
the base price so that the base price will represent relative formula.
the new quality. The long‐term price relative can 7.2.7 In the class mean example period 2 is also
then be calculated using the price of the new quality used for the quality comparison; again
variety to the quality‐adjusted base price. Replacement 1’s period‐2 price ($180) includes the
7.2.4 In the previous examples, the period‐1 price cost of the new quality and Brand C’s period‐2
for Brand C is $140 and the price index in period 1 is estimated price ($155.6) includes the old quality
125.0, so that the base price must have been $112 cost. The quality adjustment factor of 1.1576 ($180 /
([$140 / 125] x 100). In the explicit adjustment $155.6) is applied to the old base price ($112) to
example the price of Replacement 1 with the new estimate the new base price ($129.6). Table 7.2.4
quality is $165 in period 1, which was derived by demonstrates the compilation of the elementary
85
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
aggregate index using the long‐term relative to Brand C’s period‐1 price ($140) to derive an
formula. estimated price ($152.4) for Brand C that has the old
quality. (Note again that the comparison is made
7.2.8 The calculations for the matched model
using the same time‐period ‐ period 2.) The quality
example are similar to those for the overall mean
adjustment factor is 1.1811 (=$180 / $152.4), which
method. Period 2 is also used for the quality
is applied to the old base price ($112) to estimate
comparison because it is the first period that allows
the new base price ($132.3). Table 7.2.5
one to be done. Replacement 1’s period‐2 price
demonstrates the compilation of the elementary
($180) includes the cost of the new quality and the
aggregate index after quality adjustment, using the
price of Brand C in period 2 is estimated by using the
long‐term relative formula.
geometric mean of the short‐term price relatives of
Brands A and B (1.089). The price relative is applied
Table 7.2.1 Calculation of quality‐adjusted EA index using explicit adjustment method
Variety Base QA
Price in Price in Price Price Index
Price Factor
Period Period Relative Period 2
1 2 2/1
Brand A 120 – 150 160 1.333 133.3
Brand B 150 – 225 250 1.667 166.7
Brand C 112 – 140 –
Replacement 1 (132) 1.1786 (165) 180 1.364 136.4
Elementary Aggregate Index 100.0 1.447 144.7
(geometric mean)
Table 7.2.2 Calculation of quality‐adjusted EA index using overlap method
Variety Base QA Price in Price in Price Price Index
Price Factor Period 1 Period 2 Relative Period 2
2/1
Brand A 120 – 150 160 1.333 133.3
Brand B 150 – 225 250 1.667 166.7
Brand C 112 – 140 –
Replacement 1 (128) 1.14286 160 180 1.406 140.6
Elementary Aggregate 100.0 1.462 146.2
Index (geometric mean)
7.2.3 Calculation of quality‐adjusted EA index using overall mean method
Variety Base QA Price in Price in Price Price
Price Factor Period 1 Period 2 Relative Index
2/1 Period 2
Brand A 120 – 150 160 1.333 133.3
Brand B 150 – 225 250 1.667 166.7
Brand C 112 – 140 (152.4)
Replacement 1 (132.3) 1.1810 – 180 1.361 136.1
Elementary Aggregate Index 100.0 1.446 144.6
(geometric mean)
86
SUBSTITUTION AND QUALITY CHANGE
Table 7.2.4 Calculation of quality‐adjusted EA index using class mean method
Variety Base QA Price in Price in Price Price
Price Factor Period 1 Period 2 Relative Index
2/1 Period 2
Brand A 120 – 150 160 1.333 133.3
Brand B 150 – 225 250 1.667 166.7
Brand C 112 – 140 (155.6)
Replacement 1 (129.6) 1.1571 – 180 1.389 138.9
Elementary Aggregate Index 100.0 1.456 145.6
(geometric mean)
Table 7.2.5 Calculation of quality‐adjusted EA index using matched model method
Variety Base QA Price in Price in Short‐term Long‐ Price in Long‐term
Price Factor Period Period Price Term Period 3 Price
1 2 Relative Price Relative
Relative
Brand A 120 150 160 1.067 1.333 170 1.417
Brand B 150 225 250 1.111 1.667 265 1.767
Brand C 112 1.1810 140 (152.4) (1.089) – – –
Replacement 1 (132.3) 180 – 1.361 200 1.512
Elementary 100.0 1.089 1.446 1.558
Aggregate Index
(geometric
mean)
87
CHAPTER 8
New Products
Introduction • Has a recognisable and generally accepted new
benefit to consumers as a result of becoming
8.1 The complication of new products appearing available.
in the market place is covered in Chapter 8 of the CPI
Manual, paragraphs 8.32–8.58. It provides a practical 8.5 The last two bullet points help to distinguish
definition of a new product to distinguish it from an an entirely new product – referred to as a
existing product whose features have changed and revolutionary product ‐ from an existing product
provides guidance on the detection of new products, whose features and, in consequence, “quality” have
the timing of their introduction and methods for changed – an evolutionary product. A revolutionary
their initial incorporation into the CPI. product is an entirely new good or service that is not
closely tied to a previously available product. A
8.2 Paragraph 28 of the ILO Resolution advises revolutionary product tends to be a good or service
that “completely new types of goods and services that is expected to satisfy some need in a new way
should normally be considered for inclusion (in the and is unlikely to fit neatly into an existing CPI item
index) only during one of the periodic review and re‐ category. For example, a mobile telephone, whilst in
weighting exercises”. It then goes on to offer the one way an extension of an existing flow of service
advice that a new model or variety of an existing (telecommunication), has a dimension of service
product that can be fitted within an existing which is new (it provides the opportunity to make
elementary aggregate should be included “at the “mobile” calls away from a fixed telephone) and is a
time it is assessed as having a significant and distinct product from existing land‐line telephone
sustainable market share”. services (it is a step change in technology). It is
8.3 This chapter of the Handbook addresses some therefore an example of a revolutionary product.
of the practical issues of planning for the Examples of evolutionary products would be new
introduction of new products, the timing of models of household appliances such as refrigerators
introduction and the methods of incorporating them and washing machines where improvements in
into the CPI. It also deals with the special issue of the quality are introduced from time to time.
incorporation into an index of services previously Evolutionary products can also be newly added
provided free. brands of currently available products such as a new
type of canned fish or electronic appliance which
differs from those currently available. For example, a
Definition of New Products current brand of canned fish may consist of certain
types of fish (mackerel, salmon, or tuna) and then a
8.4 An entirely new product, in contrast to a new new variety for one of the canned fish is introduced
product variety which is essentially a replacement of which is packed in water rather than oil.
a previously popular product, represents a good or
service that: 8.6 The focus of this chapter is on keeping the
• Was not included and could not be included in basket of goods and services that are priced up‐to‐
the price index during the initial selection of the date and relevant. It covers both truly and
current market basket and which is now completely new products, i.e. those which are
available for possible inclusion in the index. revolutionary, as well as evolutionary new products
• Cannot be easily linked to the service flow or and also goods or services previously provided free
production technology of existing goods and and thus previously excluded from the CPI. It does
services. That is, it represents a distinct not deal directly with substitution and quality
departure from previously available products in change when a good or service unexpectedly
so far as it is a step change in terms of disappears. This is the subject of Chapter 7.
technology or utility to the customer.
88
NEW PRODUCTS
Planning for the introduction of new model will become, as measured by its popularity
will only be determined over time.
products
Further discussion of the methods for introducing
8.7 There are three sets of circumstances in into the CPI basket replacements for products
which new products and services are included in the which no longer exist and disappear from the
CPI: shop shelves is presented in Chapter 7.
89
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
90
NEW PRODUCTS
namely, the old weights do not need to be re‐scaled product varieties is to add them within the existing
when a new product class is introduced when an old elementary aggregate in which they appear (or
basket is still being used which does not include the belong). This is referred to as sample
new product class. This re‐scaling of weights is supplementation. For instance, take the case where
somewhat arbitrary (in reality it is not known exactly a new canned fish, tuna packed in water, has been
which other expenditures are being reduced recently introduced to the retail market and has
relatively as the new product is purchased) and may significant popularity in the shops. This new variety
lead to credibility issues. can be added to the existing sample in the
elementary aggregate for canned fish as shown in
Methods of incorporating new Table 8.1.
Table 8.1 Example of sample supplementation
Period 1 Period 1 Period 2 Period 2 Period 2 Period 3 Period 3 Period 3
Variety Price Price Price Price Price
Index Price Price Rel. Index Price Rel. Index
Canned mackerel
(in oil) 125.0 50.00 51.00 1.0200 127.5 51.00 1.0000 127.5
Canned anchovies
(in oil) 133.3 45.00 47.00 1.0444 139.3 48.00 1.0213 142.2
Canned tuna
(in oil) 150.0 50.00 52.00 1.0400 156.0 52.00 1.0000 156.0
Canned salmon
(in oil) 145.5 55.00 55.00 1.0000 145.5 57.00 1.0364 150.8
Canned herring
(in oil) 130.0 40.00 42.00 1.0500 136.5 43.00 1.0238 139.8
Canned tuna
(in water) ‐‐ ‐‐ 60.00 140.6 62.00 1.0333 145.3
Elementary
Aggregate Index 136.4 1.0307 140.6 1.0190 143.3
91
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 8.2 Example of targeted replacements
Period 1 Period 1 Period 2 Period 2 Period 2 Period 3 Period 3 Period 3
Variety Price Price Price Price Price
Index Price Price Rel. Index Price Rel. Index
Canned mackerel
(in oil) 125.0 50.00 51.00 1.0200 127.5 51.00 1.0000 127.5
Canned anchovies
(in oil) 133.3 45.00 47.00 1.0444 139.3 48.00 1.0213 142.2
Canned tuna
(in oil) 150.0 50.00 52.00 1.0400 156.0 ‐‐ ‐‐ ‐‐
Canned salmon
(in oil) 145.5 55.00 55.00 1.0000 145.5 57.00 1.0364 150.8
Canned herring
(in oil) 130.0 40.00 42.00 1.0500 136.5 43.00 1.0238 139.8
Canned tuna
(in water) ‐‐ ‐‐ 60.00 156.0 62.00 1.0333 161.2
Elementary
Aggregate Index 136.4 1.0307 140.6 1.0229 143.9
92
NEW PRODUCTS
underlying assumption is that all the new varieties level remain fixed at the base period level and the
have experienced, on average, the same price weights within the classes at the elementary
movement as that for the elementary index. A base aggregate level are changed when new elementary
price for each new variety is estimated by deflating items are added whilst being constrained to add to
the period 2 price for each variety by the elementary the unchanged class weight.
aggregate index in period 2 (104.1, in other words a
8.21 An overlap approach similar to sample
4.1% average price rise for these varieties is applied
rotation can be used where a new sample is selected
from the base period). For example, the Sony 25” TV
and an elementary aggregate is added. Prices are
base price is estimated as 600 / 104.1 x 100 =
collected for both the old and new sample in the
576.64. This base price is then used to calculate the
same period and the old sample is used for
variety level index in future periods. Note that the
compiling the current period index (period 2) and
elementary level indices are calculated using the
the new sample for the next period (period 3). This is
simple geometric means of price relatives.
illustrated in Table 8.4.
8.19 The higher level index at the class level and
8.22 For each of the two elementary aggregates
above is compiled using a base‐weighted arithmetic
in period 1 a new variety to price in period 2 is
mean of the elementary indices. The weights for
selected together with a sample of mobile phones
each elementary aggregate index appear in the last
for the new revolutionary item. In period 2 the old
column of Table 8.3 and are used to weight the price
sample is used to calculate the elementary aggregate
index for each period to derive the class index (the
indices and to compile the class level index. Thus,
last line in Table 8.3). For example, the period 3 class
the indices for telephones and fax machines are
index is calculated by multiplying the elementary
aggregated using the old weights for the elementary
index by its corresponding weight and summing
indices to derive the period 2 class index ([101.5 x
them to get the class index ([95.8 x 0.3] + [103.2 x
0.6] + [94.3 x 0.4] = 98.6).
0.5] + [89.4 x 0.2] = 98.2).
8.23 In period 3 the elementary and class level
Revolutionary products indices are compiled using the new sample of
Adding a new elementary aggregate products and varieties along with the new set of
weights for each component. The telephone index in
8.20 In the previous example, the sample of period 3 (99.6) is calculated by using the geometric
products was outdated and new varieties were mean price relative for the three new varieties
selected within existing elementary aggregates. It (0.9811) multiplied by the period 2 price index for
will occasionally happen that new revolutionary telephones (101.5). The same calculation is used to
products arise that are not covered within the scope derive the, period 3, elementary index for fax
of existing elementary aggregates but do fall within machines (0.9967 x 94.3 = 94.0).
the more widely‐defined COICOP classes. Adding a
new elementary aggregate and re‐distributing the 8.24 The new elementary index for mobile
weight for the COICOP class to all the elementary phones has no period 2 index to use so the period 2
aggregates (or items) now included, is a way of class level index (98.6) is used as the mobile phone
including such products. For example, assume that index, on the assumption that the elementary index
currently telephones have a weight of 60% and fax for mobile phones would have changed by the same
machines 40%, respectively, within class 08.2.0 percentage, on average, as the other products within
(telephone and fax equipment) and that the latest the class. Note that this value is also used as the
information from importers of telephonic equipment starting index for each of the variety indices within
indicates telephones to households (i.e. where sales mobile phones. The period 3 mobile phones
to businesses have been identified and excluded) elementary index is calculated as 98.0 by using the
now have a market share of 20%, fax machines 10%, elementary level price relative (0.9933) multiplied by
and mobile phones 70%. The statistical office can use 98.6.
this information to introduce a new elementary 8.25 The aggregate index is calculated as the
aggregate for mobile phones. The weights at the geometric mean of the variety indexes.
class levels, including class 08.2.0, remain fixed for
8.26 The period 3 class level index is derived
aggregating to group and division level indices while
using the index for the new elementary aggregates
the relative weighting of the elementary aggregates
along with the new weights for the elementary
within the class level are allowed to change as new
indices ([99.6 x 0.2] + [94.0 x 0.1] + [98.0 x 0.7] =
aggregates are added. Thus, there is a two‐tier
97.9).
aggregation system in which the weights at the class
93
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Introducing new items and higher‐level in the overall share of communications expenditures
weights in the CPI in between basket as a result of the revolutionary new product class.
revisions 8.29 Note also that the index compiler will need
to judge whether it is appropriate to update all the
8.27 The example of adding an elementary above weights on an annual basis, subject to the
aggregate presented in Table 8.4 provides a method availability of expenditure information from sources
of introducing a new revolutionary item index, in this such as those described.
case for mobile phones within the COICOP classes
08.2.0 (handsets) and 08.3.0 (calls). The relevant 8.30 Countries which have participated in a
class index now includes the contribution to price recent International Comparison Programme (ICP)
change of the mobile phones item within the class. round might have the opportunity to use ICP
The class, however, is likely to be under‐represented expenditure data to derive new share weights at
within the corresponding groups (08.2 Telephone higher aggregate levels where further additional
and Telefax Equipment and 08.3 Telephone and work, of relevance to a CPI, has been carried on
Telefax services), the division (0.8 Communication) expenditure shares to provide better and more
and the overall CPI, because its weight does not robust estimates. For example, ICP data should be
reflect the increased expenditure resulting from the available at the COICOP division level and, if
introduction of mobile phones. In most instances, sufficiently timely, could be used to derive
statistical offices are hesitant to change the weights expenditure shares for each of the divisions. These
for the class, group, and division until a new set of shares can then be used to aggregate the division‐
weights for all items can be obtained from a recent level indices to compile the total CPI. For those
Household Budget Survey. Statistical offices will divisions where no new items are added, the share
typically include mobile phone handsets in class weights within each level remain the same for
08.2.0 and mobile phone services in class 08.3.0 aggregating item indices to the class level and class
without changing the weights for higher‐level indices to the group level. Introducing new share
aggregates. weights at the division level, however, comes at a
cost because it involves rebasing the index. The new
8.28 If the statistical office has no plans for weights cannot be used directly to compile the
conducting a Household Budget Survey in the near division 08 index and the all‐items CPI. The statistical
future and is concerned about the possible impact office must follow the procedures discussed in the
on the statistical integrity of the CPI from not rebasing section of Chapter 10 to introduce the new
changing the weights for the class, group, and weights and to compile the division 08 index and the
division, there are alternative sources it can consider total CPI. This involves compiling the current and
for updating weights. To update the weights for previous period division 08 index and the all‐items
high‐level aggregates, the statistical office first needs CPI using the new weights, calculating the one‐
to estimate weights for each class within the group period price relatives, and then applying these price
where the new product is added so that these class relatives to the previous period’s published index for
indices can be aggregated to the group level. division 08 and the total CPI. This procedure must be
Likewise new weights are needed for each group to used in all future periods also.
compile the division‐level index. In the example for
mobile phones, each group within division 08 8.31 Table 8.5 contains an example of
contains only one class so weights are needed for introducing new weights at the aggregate level when
just groups 08.1, 08.2, and 08.3. The postal authority the new items for mobile phones have been
may be able to provide estimates of postal revenues introduced in groups 08.1, 08.2, and 08.3. The new
for group 08.1. Importers of telephones, fax share weights for these three groups in column 3
machines, and mobile phones may be able to sum to 100. The price indices for period 1 (columns 4
provide revenue data for group 08.2. Regulatory and 5) below the total CPI level are the same, but
authorities can be a source for revenue data on they are aggregated using different weights. As a
telephone and mobile services as well as internet result, the index for division 08 (Communication)
fees. Alternatively, the national accounts may have differs between column 4 (197.9) and column 5
expenditure data already compiled for these groups. (192.0). The total CPI also differs because of the
The next step is to use the revenue information to different weights — column 4 (386.6) using the old
calculate relative shares for each group and use the weights and column 5 (393.4) using the new weights.
share weights to aggregate group indices to the As the difference between these values is solely due
division level. Note that the above procedure will to weighting effects, the re‐weighted index value
still not be completely satisfactory if it does not should not be published ‐ it should only be used to
incorporate, into Division 08, increases or decreases calculate the current period price change. Footnotes
94
NEW PRODUCTS
should be provided to explain the introduction of the 8.36 The solution is to go back to the standard
new weights. Laspeyres index formula using quantity weights
instead of expenditure weights, and using price
8.32 To derive the period 2 indices for division 08
levels rather than price relatives. The newly‐charged
and the all‐items CPI in column 7, the indices are
product or service can be treated as if it were
compiled using the new weights and the price
already included in an existing section (or item)
relatives between period 1 and period 2 (column 6)
index with a zero price but with non‐zero quantity
are calculated. The price relative for division 08
equal to its consumption in the base period. The
(0.999229) is applied to the period 1 published index
index is then adjusted from the point of introduction
for 08 Communication (197.9) to derive the period 2
of the positive new price to take on the new
Communication index (197.8). Likewise, the price
expenditure. The adjustment is as follows::
relative for the total CPI (0.997954) in column 6 is
applied to the period 1 published index for the Total Iu × EXPu + 100 × Qo × Pt
CPI (386.6) to derive the period 2 All Items CPI Ia =
(385.8). EXPu
Where:
8.33 These same calculations, using the price Ia = adjusted index;
change in the re‐weighted 08 Communication index
and total CPI, are repeated for all future periods (see Iu = unadjusted index;
columns 8 and 9 in Table 8.5). EXPu= average weekly household expenditure in the
base period for the index;
Geographical dimension Qo= quantity of the newly‐priced service used in the
base period; and
8.34 New products may appear in various Pt = price of the newly priced service.
markets at different times. For example, many
products will first appear in the capital or large cities 8.37 In practice, it is not necessary to know Qo
before being introduced (if at all) in smaller towns. and Pt explicitly if their product, the expenditure on
Thus, it may not be feasible to add new products to base year quantity at period t, is known or can be
all CPI areas at the same time. They may have to be estimated.
introduced in different CPI areas gradually as their
popularity spreads throughout the country. The 8.38 After an introductory period, the product
above methods should be used to introduce new may merit a separate index.
products in each CPI area as appropriate. For 8.39 Some illustrative examples follow.
example, a new product may first be added to the
capital city basket and then later to larger cities in Eye‐tests for adults
other regions. It may never become popular in rural 8.40 Assume that these were free until April
areas ‐ in such a case, it should not be added to the 2007 but were charged for from that date on. The
rural basket. In principle, a household budget survey fees can be incorporated from that point by
should provide the necessary background adjusting the paramedical services section index of
information, but the problem is that, frequently, the CPI (COICOP 06.2.3) to take account of the new
rural areas are excluded. charges for the remainder of 2007 until eye test
Services Previously Provided Free charges can be introduced as a new item in that
section in the usual way at the time of a regular
8.35 Sometimes services which have hitherto updating of the CPI basket and chain linking.
been provided free at the point of provision can
8.41 The first stage in calculating the adjustment
become chargeable. Examples are the introduction
is to estimate the expenditure per CPI household
of fees for the provision of certain health services,
arising from the introduction of eye test charges, for
university tuition fees, and parking charges. The
instance as implied by the number of free checks
problem for the CPI compiler in these cases is two‐
consumed in the base period. Price quotes can then
fold:
be collected from opticians and the average price
• There is no weight in the base period calculated. Where some people are still not charged
(expenditure is zero). for an eye test, for example those from poorer
households, the number of tests paid for per CPI
• There is no base period price with which to household can be estimated using official estimates
compare the new price to create a price relative. of the proportion of adults paying for eye tests per
year.
95
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
8.42 For instance, the number of eye tests paid o Living at home or away from home.
for per index household per week may be calculated
o If living away from home, whether they
as:
are living in communal or independent
0.22 × 1.389 accommodation.
= 0.006
52 But in practice, most households would regard
dependent students as part of their household
Where it has been estimated from other data
even if attending an institution away from
sources that: home. This can justify a decision to treat all
22% of adults will pay for their eye tests (and 78% students in higher education as within the
scope of the CPI. But the issue is not clear cut if
will continue to get them free).
the CPI excludes “institutionised” households.
The number of adults per CPI household is 1.389. • Scale of fees. In the case of goods or services
8.43 The expenditure per CPI household per provided or partly paid for by government,
week can be estimated by multiplying the average widely accepted CPI standards indicate that the
price of an eye test by 0.006. amount relevant for inclusion in a CPI is the
charge made at the point of acquisition or
8.44 The adjusted paramedical services (COICOP consumption, i.e. the amount actually paid by
06.2.3) index is then calculated using the formula the individual household not the full economic
already referred to above: cost of the service. In this particular example, it
Iu × EXPu + 100 × 0.006 × Pt is assumed that students are liable for an
Ia = amount between zero and a maximum set by
EXPu
the government depending on their own and/or
Where: their family income. The price recorded, and the
la = adjusted index for paramedical services (COICOP index weight, should thus be the average of that
06.2.3) actually paid by or on behalf of the students.
Iu = unadjusted index for paramedical services The Ministry of Education may be able to supply
(COICOP 06.2.3) forward estimates.
EXPu = base period expenditure on paramedical • Timing. The assumption is made that all fees are
services (COICOP 06.2.3) paid when billed although in practice it is
Pt = average price for eye tests at month t. possible that some students may pay some time
after the due payment date. In this example, it
University Fees may be assumed that fees are paid at the
8.45 The introduction of student fees can raise a beginning of the academic year, and therefore
number of conceptual issues relating to the coverage included in the October CPI index.
of the index and the service paid for. Assume that • Method of incorporation. Initially, the index may
from October 1998, the start of the academic year, be combined with private education fees, in
new students on full‐time higher education courses order to compute an adjusted index as
had to contribute up to $1,000 a year towards the described in the earlier paragraphs. A decision
cost of their tuition (rising to $1,125 a year in then needs to be taken whether at some future
October 2003), the actual amount depending on point higher education fees and private
their own and, if appropriate, their parents’ or education fees are represented by separate
spouse’s income. Previously, university education item indices. Two considerations apply: user
had been universally free. needs and the robustness of the separate
8.46 There are four specific issues requiring indices.
consideration:
Parking charges
• Index coverage. The CPI is intended to reflect
8.47 Assume that parking charges in the inner
the average spending pattern of households.
areas of the capital city were introduced for the first
The definition of household in the case of
time in a country in February 2003, and were first
students might be considered to vary according
included in the March 2003 CPI. Assume that at the
to whether they are:
time of introduction it is estimated that $110m will
o Dependent on family or independent be generated over a year from a standard charge of
(depending on age and whether $5 levied on cars while the annual revenues
married). generated from charges which include a residents’
96
NEW PRODUCTS
discount of, say, 90% is estimated at $6m. Also parking charges. If such a sub‐index does not exist,
assume that it is estimated that there will be a 12.5% then the most appropriate index in the same COICOP
reduction in non‐resident traffic and that in 10% of class should be chosen – e.g. road tolls for road
cases employers will pay the parking charges of their bridges or motorways. If it is assumed that the base
non‐resident employees. It can be argued that as a period expenditure on pre‐existing parking charges is
CPI is a base‐weighted, fixed basket index that does 28.7 cents per CPI household per week, then the
not take into account substitution away from a combined expenditure, including the new parking
service as a result of a price increase, the estimated charge, is 35.7 cents. Assuming no change in pre‐
$110m from the standard charge should be existing parking charges, this gives an increase of
increased, by the estimated reduction in traffic of about 24% in the sub‐index, due to the introduction
12.5%, to $125.71m. of the new parking charge, as shown below. If the
index for the pre‐existing parking charges is Iu, then,
8.48 This figure is then reduced by 10%, to
the adjusted index, Ia, is calculated as:
remove revenue from parking fees paid for by
employers, giving an annual expenditure of Iu × 28.7 + 100 × 7
approximately $113.1m. To this must be added the Ia =
28.7
$6m revenue which is raised from resident
households giving a total of $119.1m. This is then 8.50 The treatment of services previously
converted into an estimated expenditure per CPI provided free can be contentious. There is a lack of a
household per week, in this case 7.0 cents, if we universally accepted method and this can lead to
assume that there are 32.57 million households in inconsistent treatment. For instance, it can be
the base population. argued that increasing the forecast revenue by the
estimated reduction in usage, as in the parking
8.49 This increased expenditure must be
charges example, is counter‐intuitive to most users
accounted for by adding it onto an existing sub‐
of a CPI and can be hard to justify.
index. In this particular example, the most
appropriate sub‐index would be one covering other
97
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
98
NEW PRODUCTS
99
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 8.5 Example of introducing new weights for higher‐level aggregates
1 2 3 4 5 6 7 8 9
Period 1 Period 1 Period 2 Period 3
Old New Index Index Price Index Price Index
COICOP Code Weight Weight (old wts) (new wts) Relative (new wts) Relative (new wts)
Total CPI 100.00 386.6 385.8 386.8
Total CPI using new
weights 100.00 393.4 0.997954 392.6 1.002562 393.6
01 Food and Non‐
alcoholic Beverages 67.29 58.23 345.2 345.2 342.5 343.7
02 Alcoholic Beverages
and Tobacco 0.54 0.77 453.2 453.2 455.1 454.1
03 Clothing and
Footwear 1.37 1.70 376.9 376.9 375.0 376.0
04 Housing, Water,
Electricity, Gas and
Other Fuels 7.91 8.45 572.4 572.4 576.8 577.2
05 Furnishings,
Household Equipment
and Routine Household
Maintenance 2.54 2.71 401.5 401.5 402.0 402.8
06 Health 1.47 2.12 215.1 215.1 217.3 217.3
07 Transport 6.06 8.25 685.9 685.9 687.7 686.5
08 Communication 2.14 197.9 197.8 197.5
Communication using
new weights 4.15 192.0 0.999229 191.8 0.998570 191.5
08.1 Postal Services 0.47 0.24 201.1 201.1 201.1 201.1
08.2 Telephone and
Telefax Equipment 6.54 37.35 179.8 179.8 177.9 177.5
08.3 Telephone and
Telefax Services 92.99 62.41 199.2 199.2 200.1 199.9
09 Recreation and
Culture 1.79 2.55 348.2 348.2 349.6 350.3
10 Education 1.43 1.98 433.7 433.7 433.7 435.9
11 Restaurants and
Hotels 3.41 5.67 411.2 411.2 413.2 415.5
12 Miscellaneous
Services 4.05 3.42 391.3 391.3 393.1 397.8
100
CHAPTER 9.1
Special Cases ‐ Housing
Introduction National housing market structures and practical
measurement issues are also important
9.1.1 Paragraphs 10.4 to 10.50 of the ILO considerations in determining which approach to
Manual give a detailed exposition of the conceptual apply in practice.
and practical issues relating to the measurement of 9.1.3 In a national context the inclusion of
owner‐occupier housing costs. The manual is not owner‐occupier housing costs will improve the
prescriptive in recommending a particular approach representativity and relevance of a CPI. When used
but does recommend that “the approach chosen in an international context, for example to measure
should align with the conceptual basis which best economic convergence, the inclusion of owner‐
satisfies the principal purpose of the CPI”. It also occupier housing costs should, in principle, enhance
makes the point that alternative conceptual the comparability of national CPIs across countries.
treatments can have a significant impact on However for this purpose two major concerns have
measured inflation in the short‐term. Like the been expressed. First, including owner‐occupier
Manual, the 2003 ILO Resolution is not prescriptive. housing costs could impact on the inflation rate and
The treatment of rented accommodation is its volatility. Secondly, it could add to cross‐country
considered straightforward and not to warrant divergence in inflation rates and may make it more
special mention. This section first looks at the difficult for an individual country to meet inflation
treatment of owner‐occupier housing costs and then convergence criteria set for a block of countries, due
at the costs borne by tenants. The latter are often to differences with regards to the relative
used to impute owner‐occupier housing costs. importance of each country’s owner‐occupier
housing market and divergences in the respective
Owner‐occupier housing costs movement in house prices across countries. This
could be the case, for instance, if an inflation target
9.1.2 The treatment of owner‐occupied housing was set by the European Central Bank for the Euro‐
is one of the most difficult issues faced by compilers zone, which included owner‐occupier housing costs.
of consumer price indices. There are a number of 9.1.4 It can also be noted that, depending on the
alternative conceptual treatments and the choice methodological approach used, data on owner‐
between them can have a significant impact on the occupier housing costs may be not as timely as other
overall index, affecting both weights and, at least in data for the CPI and the compilation of
the short‐term, the measured rate of inflation. Some representative owner‐occupier housing costs indices
of these costs, namely the expenditure on minor might only be feasible on a quarterly basis. Both
repairs and maintenance, local property taxes and aspects would impact on the underlying statistical
the costs of services are straightforward, can be quality and usability of the CPI.
directly measured and there is no reason to
introduce a special methodology outside the usual 9.1.5 All these considerations need to be taken
CPI protocols. However, as regards the main into account by a national statistics institute in
elements of owner‐occupier housing costs – the deciding which approach to adopt once a decision
purchase of the property and to a lesser extent, the has been made to include owner‐occupier housing
depreciation cost or cost of major repairs ‐ there is costs. The criteria for choosing which approach will
no universally accepted method of treatment. The include: alignment with user needs and the main
absence of any firm consensus concerning the purpose of the CPI; consistency with the rest of the
appropriate treatment of such costs partly reflects CPI; alignment with international practices (although
the fact that national CPIs are often constructed to in the case of housing costs there is no unique
serve several distinct purposes, from monitoring the agreed methodology); public acceptance; practicality
economy to adjustment of incomes or state benefits. of implementation.
101
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
102
SPECIAL CASES ‐ HOUSING
two methods, it is more likely that there are data loans, while changes in house prices will affect only
which can be readily drawn upon to construct the those buying a house in the current period. Thus a
appropriate weights and the price index. The price price indicator consisting of the current interest
measure would be the change in the average applied to a standard‐sized mortgage to a standard‐
equivalent rent for properties similar to those which sized house would not be appropriate. An
are owned i.e. a sample of private rents appropriate indicator involves two components ‐ the
appropriately stratified and weighted, and the rate of interest and the average amount of mortgage
weight would be the average equivalent rent for debt outstanding. To calculate the average
owner‐occupied homes in the base period, imputed outstanding debt at any one point in time can be
on the basis of actual rents for private rented problematic as it consists of a large number of
dwellings, adjusted at least in theory to allow for any individual debts, some from mortgages taken out
quality difference between owned and rented recently and other from mortgages taken out some
property. More detailed guidance on the application time ago at historic prices and with some of the debt
of this approach is given below in paragraphs 9.1.13 paid back. It is unlikely that all countries, and most
to 9.1.18. developing countries, will have the necessary data to
apply this method. Returning to more fundamental
9.1.11 Moving on to option (2) ‐ the payments
issues, some argue that the problem with this
approach ‐ it is sometimes argued that this option is
approach is that it includes a major (explicit) cost of
more consistent with the traditional approach to CPI
owning a home, namely mortgage interest, but it
construction, which is a carry‐over from a time when
does not include a major offsetting (implicit or
the CPI was mostly used as a compensation tool. It
imputed) benefit, namely possible price appreciation
also has much to commend it from the point of view
or capital gains on the home. This neglect of this
of public acceptability. It measures costs directly,
benefit is particularly troublesome when there is
thereby avoiding imputation. In addition “mortgage
moderate or high inflation in the economy: the
interest” is more likely to be understood than “rental
observed mortgage interest cost can increase
equivalence” and, unlike the latter, the index will
compared with other costs and give a very
reflect changes in house prices and interest rates.
misleading picture of the homeowner’s true long run
But there are also issues relating to (2). The items of
costs of living in the home (because the offsetting
direct expenditure by purchasers of property are:
benefit is neglected). But the counter‐argument is
mortgage interest payments; repayment of capital;
that the capital gains of owning a house which
large repairs (associated with depreciation from
appreciates in value is of limited relevance when
wear and tear as properties get older). A CPI should
people have to bear such costs from current income.
only relate to consumption items and should exclude
This is, perhaps, a case where the measurement
cash disbursements or expenditure which is in the
approach which is adopted will vary depending on
nature of a saving or “investment” and the
the use and purpose of the CPI.
acquisition of a house will normally over a period of
time represent a substantial capital asset – a point 9.1.12 The argument in favour of (3) ‐ the net
which emerges when comparing the position of acquisition approach ‐ is that it is the closest to the
owner‐occupiers with tenants who rent. It can “acquisition” approach which has traditionally been
therefore be argued that the capital element of adopted for other parts of a CPI and is more
mortgage repayments should be regarded as an appropriate for a CPI being used as a general
investment or saving rather than consumption indicator of current economic conditions. But the
expenditure and should therefore be excluded from method draws considerable criticism from those
the index. The question arises over whether the who require a CPI as a compensation index, as
weight and price indicator should be net of any tax neither the weight nor the price indicator properly
allowances for mortgage interest payments. It is reflect the shelter costs of owner‐occupiers. For
recommended that in accordance with the principle instance, a rise in interest rates would not be
that a CPI should be based on the amounts actually reflected in a net acquisition cost index. Also there is
paid the weight and price indicator should both be a lack on consensus about whether the price of land
based on payments after tax relief. Perhaps the should be included or excluded. It has been argued
biggest barrier to the adoption of this method is that that in principle the cost of the land should be
it requires a large volume of data, which may or may excluded because it is a non‐depreciating capital
not be available to the compiler. Mortgage interest asset, i.e. it represents the capital element of the
payments will be affected by changes in both house purchase, and similar to the treatment of
interest rates and house prices in different ways. capital repayments on mortgages should be
Changes in interest rates will affect all of those regarded as an investment rather than consumption
buying a house apart from those on fixed‐interest expenditure. It is important to remember that in the
103
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
System of National Accounts dwellings are treated as in the base period, or from asking an expert to
capital, not consumption, items. Some countries use provide on a monthly basis the current equivalent
this argument to exclude the acquisitions approach rents for a sample of houses with different
as a valid method for a CPI. characteristics which are “representative” of the
owner‐occupier housing stock. More advice on rent
Rental equivalence (imputed rents) – an surveys is given later on in this section where the
option for many developing countries measurement of the costs borne by renters is
9.1.13 For many developing countries a significant discussed.
proportion of the housing stock consists of newly 9.1.17 In each case, stratification by type of
constructed buildings on family owned land or of old dwelling (house or flat), location (region or area,
stock which has been significantly upgraded. There urban or rural), plus other characteristics which will
can also be a significant element of owner‐ influence rent is important so that the rents data can
constructed housing. Construction may take many be combined to reflect the overall composition of
years and at any time a significant proportion of the owner‐occupied property. Other stratification
houses could be considered incomplete. The use of variables will include such things as the total size of
formal mortgage finance is often very limited but the plot, floor area and number of rooms, whether
informal finance may be used. House construction there is mains water, an internal WC and mains
can vary from shanties built on compacted soil with electricity, the material used in construction and
salvaged materials to substantial multi‐room whether the building is of traditional design. The
dwellings built on concrete foundations with price statistician should seek the advice of an expert
concrete blocks. Amenity levels can vary from active in the field of renting domestic property, such
virtually none to the elaborate. Housing mobility, as a housing corporation, to ascertain the most
particularly with owner‐constructed dwellings, is important rent‐determining characteristics and
usually very low and consequently the markets for should bear in mind the need to keep these to a
rental or sale of owner‐constructed houses are manageable number. Weights information can be
limited and there is very little movement between derived from the latest Housing Census or Census of
the two. However, in principle the estimation of the Population and Housing. In practice this information
price of owner‐occupied housing services is the same may not be up‐to‐date due to the change in the
for owner‐constructed housing as for third party owner‐occupied housing stock which can occur in
constructed housing but the measurement problems the time period between censuses. Where this is the
are accentuated. case special surveys may need to be conducted or,
9.1.14 The above complications mean that formal particularly in urban areas including townships, use
records will rarely be kept of the cost of building the made of planning applications to update the latest
new dwelling or of upgrading an old house, for census.
example, by incorporating running water, an internal Double counting
WC or additional rooms. Formal transfers of
ownership sometimes do not take place, formal 9.1.18 Where expenditure on repairs,
valuations are often not available and methods of maintenance, local property taxes, water charges
financing can be informal through the family or may etc. are included in rents, these costs should not be
simply not be recorded or records not kept centrally. included elsewhere in the index. Thus it is important
Thus in these circumstances it is not possible to that Household Budget Surveys determine if the
calculate mortgage interest payments (including or renter household receives any additional services,
excluding notional interest payments to relatives), or such as electricity or use of facilities outside the
to estimate net acquisition costs. housing unit, such as off‐street parking. It is also
important to learn if the household must pay any
9.1.15 The lack of such basic information often additional costs, such as taxes, that the owner of the
means that the rental equivalence or imputed rent dwelling does not pay. The value of any
approach is the only practical option for supplementary items provided and the cost of any
incorporating owner‐occupier housing costs in a CPI, items borne by the tenant should be allocated to
whether owner‐constructed housing or third party their proper CPI category. For example, the dwelling
constructed. may have water supplied at cost by the landlord: in
9.1.16 The price indicator for imputed rents can this case, CPIs may either leave landlord‐supplied
be derived either from a readily available price series water in the rent or move an estimated value for it
for rents, weighted to reflect the current from the rent index to an index for water, but this
composition of the stock of owner‐occupier housing, must be done consistently in the weights and the
which can then be applied to the rental equivalents rent survey. Leaving supplementary items in the
104
SPECIAL CASES ‐ HOUSING
rent avoids the need to adjust the weight, but leaves such as a labour force survey. In this case a specific
the potential problem that if the landlord ceases to rent‐based auxiliary questionnaire can be distributed
provide them, the CPI compiler will need to adjust to those tenants who participate in the main survey.
the values of the rents collected in the rent survey. Clearly the respondent should be knowledgeable.
For example, some occupants may not be the ones
Rented accommodation who actually pay the rent and may be quite
unsuitable as respondents. The important feature of
9.1.19 Unlike owner‐occupiers, renters buy any such survey is that it provides rents at multiple
shelter from others who own the dwellings that the time intervals but because people may move it will
renters occupy. Consequently, there is a market not necessarily follow the rents of a fixed basket of
transaction to observe and the cost of rented housing units. In these circumstances, the rent data
accommodation is relatively easy to observe in the should be stratified by rent‐determining
market. characteristics, so that the average rent increase for
a house with pre‐assigned characteristics or for a
Weights “fixed basket” of house types can be determined.
9.1.20 Obtaining the weight for renter‐occupied 9.1.24 If rents change rather infrequently, it may
housing in the CPI basket is relatively be more efficient to use a relatively large sample
straightforward. Household Budget Surveys (HBS) collected less frequently than every month even
typically collect data from households at their place though the CPI may be computed monthly.
of residence. The surveys obtain the rents from
9.1.25 Where a survey of rental dwellings is
those residences that are renter‐occupied, and the
undertaken a sample can be drawn from any frame
statistical offices estimate the annual expenditure on
that contains the residential units of an area. This
rents using standard methods. The HBS should also
may be the Census of Population (if it has a shelter
inquire whether the household rents other housing
component), postal lists, or street directories.
in addition to its main residence, perhaps near to a
job or in a holiday location. 9.1.26 If the population census can provide
information on the average rent or dwelling value by
9.1.21 It is important that the HBS determines if
geographic area, sampling of the areas by probability
any additional services are included in the rent. This
proportional to rent/value will increase the final
is to ensure that no double‐counting takes place and
sample’s representativity. Where a sampling frame is
that a consistent approach is taken on where the
not readily available, a grid can be placed on a map
expenditure is accounted for in the CPI (see earlier
of the area and a sample of cells in the grid can be
section).
selected. The grid method can also be used as a
Prices – rent surveys second stage of sampling after selecting larger areas
using available information on rents or values.
9.1.22 The rent to be recorded is the amount that Several countries, including developing countries,
the household actually pays including taxes and have used satellite images of the selected areas to
excluding subsidies. If the rent is subsidized or taxed, obtain an accurate view of the housing in the cells or
the amount the household pays will not be the same target areas. Ideally, all the dwellings in the selected
as what the landlord receives so although cells should be enumerated and from this a random
information on rents may be collected from any sample selected for the data collector to visit to
knowledgeable respondent – this may be one of the determine if they are tenant‐occupied, and, if so, to
dwelling’s occupants, its landlord or the landlord’s obtain their rents and initiate them into the sample.
representative, the latter two can be problematic for The initial selection of target areas for enumeration
a CPI. In order to respect the “fixed basket” principle should be small enough to be manageable and
of the CPI, an effort should be made to continue relatively homogeneous and large enough to allow
collecting prices from the same dwelling units over for a big enough initial sample selection to allow for
time. Despite the perceived difficulties, a non‐response and the fact that some dwellings will
longitudinal survey of rental units will often be the be owner‐occupied. Where an enumeration exercise
best solution to ensure that the proper rents are is not possible, for example because of the cost,
collected24. judgmental or purposive sampling may be used but
9.1.23 Alternatively, it may be possible to collect in this case it is particularly important that the
rents from an existing continuous household survey individual cells on the grid are relatively
homogeneous, i.e. that there is little variation in the
type, size and quality of the dwellings, in order to
24
With longitudinal studies, the same units are observed at
regular intervals over long periods of time.
105
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
1 + rcollected 2
the sum of the weighted sample rents in period t to t t
t −m rcollected
the sum for the same dwellings in period t‐m: t
rˆ =r × t −m t −m
rcollected 1 + rcollected
missing missing
2
−1→t
∑ ⎡⎣ w × r ⎤⎦i i
t
× (Δ )
−1→t
Δ tRent = change in rents from period t‐1 to
t −m t −1→t m
period t
t
rˆ
missing =r missing Rent
t = the rent of sample dwelling i in
r
i period t 9.1.34 This imputed rent is not used until period
t+m. Notice that it cannot be computed until after
t −1→t
period t, when the value for Δ Rent is known.
t − m = the rent of sample dwelling i in
ri period t‐m, (the last time a rent for
that dwelling was collected)
106
SPECIAL CASES ‐ HOUSING
107
CHAPTER 9.2
Special Cases ‐ Own Account Production
Introduction prevailing on the market, unless there is some
reason to conclude that market prices are not
9.2.1 Own production is considered within the relevant or cannot be reliably observed or there is
scope of the CPI and can account for a significant no interest in using hypothetically imputed prices”.
proportion of a country’s household consumption. This chapter discusses these issues in more detail.
However, as discussed below, there are components
of own account production that are never included Background
in the CPI and other components for which
alternative treatments are possible. The issue is 9.2.4 Goods and services produced by
discussed at length in the CPI Manual, (paragraphs households for their own consumption are called
1.175 to 1.182 and 3.74 to 3.89). Before considering own account production. Own account production
what is excluded and the options available for the may be considered within the scope of a general
part of own account production that may be consumption CPI and should not be ignored
included within the scope of the CPI, it is necessary especially where it can account for a significant
to distinguish a number of concepts and to explain portion of a country’s household consumption.
some of the problems and choices that must be However, for practical reasons, own account
addressed. production of services is excluded from the scope of
the CPI.
9.2.2 The Manual makes a distinction between:
9.2.5 In some developing countries most
• Intermediate consumption. In the context of
households report some own‐account production.
own account production, this refers to the
Food is the area where this is most frequently
goods and services which households use in the
observed. For instance, it is not uncommon for 50
process of producing other goods and services.
percent and more of food consumed to come from
• Final consumption. Correspondingly, this refers own production in some developing countries. For
to goods and services produced for direct low‐income households living in rural areas, the
consumption where utility is derived by the share can be much higher. Own account production
household through the act of consumption. is therefore a particular issue for poverty analysis.
It goes on to indicate that in principle a CPI should 9.2.6 Three broad areas of household own
measure output prices in respect to own account account production can be distinguished:
final consumption but notes conceptual and practical
• Goods produced for own consumption. These
difficulties with this, associated with the fact that
goods are mainly produced by subsistence
there are no transactions or prices. The CPI Manual
households. They include basic food items such
then points to the more practical option of using
as rice, wheat or maize, vegetables, fruit, dairy
goods and services acquired in the market place as a
products, fish and meat. Some households may
proxy for household production activities as if they
also produce clothing and items of furniture for
themselves were final consumer goods and services.
their own use.
But it also points to the special problems associated
with subsistence agriculture and housing. • Owner‐occupied housing services. These services
are produced by all households that own25 the
9.2.3 The ILO Resolution is not prescriptive on
dwelling in which they live. See Chapter 9.1.
the inclusion of own account production in a CPI but
does advise that “when the consumption from own • Own account household services (excluding
account production is within scope of the index, the owner‐occupied housing services). These
weights should be based on the value of quantities services are produced by all households. They
consumed from own production” and that
“valuation of consumption from own account 25
This includes households that are purchasing their house using a
production should be made on the basis of prices
mortgage.
108
SPECIAL CASES ‐ OWN ACCOUNT PRODUCTION
109
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
110
SPECIAL CASES ‐ OWN ACCOUNT PRODUCTION
compensation index will rise because basic foodstuff Construction of a “general
prices cost more, but the rise will be moderated
because the price of imported manufactured goods consumption” index
has not increased to the same extent. The rise in the
monetary index will be moderated further because 9.2.22 To include subsistence households and
of the fall in the price of farm inputs. The general consumption in this index it is necessary to construct
consumption index will show the greatest increase appropriate consumption weights and to decide how
because the weight given to basic foodstuffs will be subsistence consumption will be priced. We consider
much higher, reflecting subsistence consumption, both aspects in turn.
and no weight will be given to farm inputs. 9.2.23 The normal source for subsistence
9.2.20 Which index in this example is ‘”correct”? consumption weights will be a household budget
If properly constructed, all three are “correct” in the survey or a survey of subsistence households. To
sense that each addresses a particular user need. measure subsistence consumption, households are
They are different simply because they serve asked to record the quantities consumed of own
different purposes. The general consumption index account goods. To obtain a value weight a “price” is
would give the best picture of how the drought was applied to the product. Usually this will be the price
affecting the country as a whole. The fact that the of the product in a relevant market. Most countries
price of farm inputs had fallen would not reflect the with significant subsistence production will prepare
problem faced by subsistence farmers who had to estimates of the value of subsistence consumption
find ways of providing food for their families and classified by product. This will be prepared for use in
were unable to benefit from the fall in farm input poverty analysis and possibly also for use in
prices. The monetary index would give the most compiling the national accounts. When preparing
relevant measure for the country as a whole of the weights for a general consumption index these
change in prices of goods exchanged within the estimates of consumption should be combined with
market. For the purposes of monetary policy the estimates of purchases of the same products by non‐
general consumption index would overstate the level subsistence households to obtain the total
of inflation but for the purposes of understanding consumption weight for each product. It is important
the impact of the drought on the price of to exclude weights for any market purchases of
consumption it would not. The converse is true of items used as inputs into subsistence production, for
the monetary index. The compensation index would example fertilizer, tools and herbicides. These
not reflect the full impact of the drought on all products are excluded but only in respect of
households but it would reflect the impact of the households for which subsistence production is
drought on indexation households. The general estimated. For all other households they are
consumption index would overstate the impact of included in the weights and classified within
the drought on indexation households and the recreational expenditure (COICOP 09.3.3.)29
monetary index would understate the impact; both 9.2.24 An alternative estimation procedure for
would be biased if used in the context of indexation. obtaining weights is to use the Commodity Flow
9.2.21 Each country has to make its own decision Approach. This is a generalised approach which uses
as to which of the three types of index is appropriate domestic production and import/export figures in
in its own circumstances. Countries with well the national accounts to adjust consumption
developed price statistics systems could consider expenditures which have been under‐ or over‐
compiling all three types of index. They would need reported in the Household Budget Survey (see
to educate users about their respective roles. Most Chapter 4). It can also be used to estimate own
countries will opt for one or possibly two types of account production although it relies on survey
index. In that situation national statistics institutes estimates of total production, including own‐
have to make a balanced decision as to which index
(or indices) will be the most useful in meeting the 29
For non‐subsistence households, garden produce may be
country’s needs. treated as a recreational outcome rather than the result of a
production process, depending on the motivation and
circumstances of production. Using this treatment, purchases
such as tools or fertilizers are gardening or recreational expenses
and not inputs into the production of food. For some households
treatment of subsistence households and subsistence the line between subsistence production and recreational activity
consumption. If there are changes in the relative prices of the will be unclear. Statistical agencies should adopt practical rules
selected product groups, the choice of index type will affect the based on national circumstances for making this distinction. For
outcome. Each index will tell a different story concerning the example, urban food production for own consumption may be
impact of the drought on households, particularly subsistence treated as recreational and rural food production for own
households. consumption may be treated as subsistence production.
111
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
account, so that conversion factors can be estimated subsistence production is included in this index as a
and applied to HBS expenditure data. proxy for subsistence consumption and to include
that consumption would involve double counting.
9.2.25 The price of basic foodstuffs may vary,
particularly between urban and rural areas. This will
reflect availability, demand, transport costs and Construction of a “compensation”
other factors. The most appropriate price to apply to index
subsistence consumption would be the price in a
market near to where the subsistence consumption 9.2.29 If it is decided to compile a ‘compensation
takes place. Typically this would include many rural index’ it is unnecessary to construct subsistence
areas. If a country collects basic foodstuff prices in consumption weights and to decide how subsistence
both urban and rural areas, the most appropriate consumption would be priced because an index
price would be an average of the rural prices designed for the indexation of wages or benefits will
weighted so as to reflect the distribution of generally exclude subsistence households from its
subsistence agriculture. In practice a simple average scope.
of rural prices might provide a reliable estimate,
particularly if the selection of rural outlets is roughly
self‐weighted.
Own account household services
9.2.26 Similar pricing considerations apply to the
(excluding owner‐occupier housing
prices used when compiling the results of a services)
household survey to estimate subsistence
consumption. Price statisticians can assist the 9.2.30 Services produced by households include
household survey statisticians with these estimates. such things as the preparation of meals, the care of
If the price statistician is producing a general children, the sick and the elderly, cleaning and
consumption index they should ensure that the maintenance of dwellings and the transport of
prices used to value subsistence consumption are household members. Housing services are provided
consistent with the prices used in construction of the by the stock of owner‐occupied dwellings.
price index.
9.2.31 The CPI Manual recommends that goods
used as inputs into services produced within the
Construction of a “monetary” index household (e.g. cleaning agents, fuels or cooking
ingredients) should be treated as consumption. It
9.2.27 One of the main challenges in including also recommends that the actual outputs are
subsistence households in this index is to construct excluded from the scope of the CPI and that no
appropriate consumption weights. This is because attempt is made to impute expenditures to them.
this index uses purchases of inputs used in producing Home cooked meals are a service output produced
subsistence outputs as a proxy for subsistence within the household. The CPI Manual recommends
consumption. Additional price information is only against imputing expenditures in respect of these
required for input items that are unique to meals. Similarly it recommends against imputing
subsistence households. If items are purchased by expenditures to other services produced within the
non‐subsistence households and are already priced household such as when parents transport their
for CPI purposes, the latter price information can be children to school or care for a sick child. These
used for the subsistence inputs component of the treatments are consistent with the SNA. Although it
monetary index, such as spades, seeds etc. may be conceptually more appropriate to regard the
outputs of household production as consumption
9.2.28 The normal source of weights for inputs
rather than the inputs, data in respect of the outputs
into subsistence production will be a household
of these areas of household production would
survey, such as a general household budget survey
require many assumptions and imputations. As a
or a survey of subsistence households. When
result they would be of little practical use and so are
preparing weights for a monetary index these
excluded from the CPI and the SNA production
estimates of consumption should be combined with
boundary as well.
estimates of purchases of the same products by non‐
subsistence households to obtain the total 9.2.32 It should be noted, however, that a long‐
consumption weight for each product. It is important term bias can result if households increase their
to exclude weights for subsistence consumption purchases of services and decrease their production
which may have been prepared as part of the for own account consumption and this is not
household survey. Subsistence consumption is addressed in an updating of weights. For example, if
excluded because the weight for inputs used in households purchase more take‐away or restaurant
112
SPECIAL CASES ‐ OWN ACCOUNT PRODUCTION
food instead of preparing food for their own purchases and other household purchases. This is
consumption, over time the relative expenditure on because labour costs have a greater impact on take‐
take‐away and restaurant food will increase and the away and restaurant food prices than on food
relative expenditure on food ingredients will ingredient prices. As noted above, although the
decrease. If long‐term labour costs increase more household contributes labour when producing
rapidly than basic food prices, there will be a long‐ household services, this is not included in the index
term downwards bias to the index (and conversely if because of the problem of putting a price on
they increase less rapidly) unless the impact of this household labour. Expenditure weights should be
change in behaviour is reflected in a revision of the reviewed on a regular basis.
weights both for own account production associated
113
CHAPTER 9.3
Special Cases ‐ Services
Introduction Background
9.3.1 This chapter covers a selection of the more 9.3.4 Services are a large, diverse and growing
important services in a CPI – health, education and part of consumer spending. As Table 9.3.1 indicates,
public transport. The principles described can also be many consumption categories according to COICOP
applied to other services. Housing, aspects of which include elementary aggregates of service items.
can be considered as a flow of services, is dealt with
9.3.5 COICOP includes the full range of services
separately in 9.1 because of the special issues
covered in the national accounts. In principle, all
relating to its treatment conceptually and to the
these should be included in a CPI compiled as a
associated measurement difficulties.
general economic indicator, but some countries
9.3.2 Chapter 10 of the CPI Manual provides choose to make some services out of scope for a CPI
comprehensive advice on the measurement of even if they are included in the national accounts.
services including detailed guidance on the COICOP categories that are often excluded from the
measurement of financial services, property CPI are games of chance and financial intermediation
insurance and telecommunications. services because they are not considered by some as
consumption items. Illegal services and fines for
9.3.3 The ILO Resolution gives no specific
violating laws are also often excluded from the
guidance on the measurement of these services but
scope, the former mainly for practical reasons and
it recommends that all kinds of goods and services
the latter because they are not part of Household
consumed by the reference population should be
Final Consumption Expenditure in the SNA.
included in a CPI. It is thus drafted on the implicit
assumption that all these services should be 9.3.6 Services present some conceptual and
included in the CPI. methodological difficulties for CPI compilers. The
main feature that distinguishes services from goods
is, of course, that they are not tangible. In addition,
Table 9.3.1 Distribution of elementary aggregates between services and goods according to
COICOP
COICOP
Category Services Goods
01 Food and non‐alcoholic beverages 0 11
02 Alcoholic beverages, tobacco and narcotics 0 5
03 Clothing and footwear 2 4
04 Housing, water, electricity, gas and other fuels 8 7
05 Furnishings, household equipment and routine household
maintenance 3 9
06 Health 4 3
07 Transport 8 6
08 Communication 2 1
09 Recreation and culture 7 14
10 Education 5 0
11 Restaurants and hotels 3 0
12 Miscellaneous goods and services 11 4
TOTAL 53 64
TOTAL GOODS AND SERVICES 117
114
SPECIAL CASES ‐ SERVICES
many types of services are not available for price feasible. This is particularly so where the people
collectors to examine directly. In some service providing the service are part of the informal
categories there may be a great variety in specific economy and are employed on a casual basis. In
services. Service providers may tailor them to such cases the only approach available to the
individuals to make them more suitable e.g. a doctor price statistician may be to impute a price or
modifying a treatment for a particular patient. price change from some other item, elementary
Services may be applied directly to consumers, as in or higher aggregate. For instance, the pay rate
health care, or to consumers’ durable goods, as in of an office cleaner or a maid in a hotel may be
the case of repairs, or supplied directly for used as a proxy for the cost of employing a maid
consumers’ use, as in the case of entertainment. in a private household.
Services may be bundled with goods, as with repair
9.3.8 Guidance on the construction of price
warranties for vehicles or mobile telephones.
indices for a selection of the more important
9.3.7 As a result of these complications, services is given below.
difficulties can arise both in sampling and in price
collection. Health, education and social
• Sampling. The “tight specification versus loose protection services
specification” debate can be acute for many
services. It is generally advised to follow a loose 9.3.9 In many countries, various government
specification when sampling and tight units or non‐profit institutions serving households
specification when pricing. A general (NPISH) will sometimes finance and pay for the full
specification is used when first establishing what or partial provision of a significant proportion of
will be priced in the outlet in order to allow the goods and services. Health, education, and social
price collector to find something representative protection are often recognised as the sectors where
of what the selected service provider sells. Then, these types of goods and services are typically
once the price collector and the respondent produced.
have settled on a particular item, a tight
specification is used. At that point the price 9.3.10 The overall position is as follows:
collector should describe the item in detail, • Expenditure is included into the CPI only in
noting any features that are likely to affect the regard to the households’ expenditure on these
price. Ideally the same price collector should services, i.e. payments that are made as a direct
return to visit the same respondent to collect all consequence of purchase of individual goods or
future prices. Clearly this will not always be services. These prices should be net of direct
possible, but having a good description of a reimbursements. Reimbursements refer to
tightly specified item should result in “like‐to‐ payments to households by government units,
like” price comparisons whenever possible and social security administrations or non‐profit
will help the most suitable replacement to be institutions serving households (NPISHs) that are
found should the original item be discontinued. made as direct consequences of purchases of
The “Structured Product Descriptions” (SPDs)30 individually specified goods and services, initially
used by the International Comparison paid for by household.
Programme provide a useful tool for describing
items within categories precisely and • Obligatory payments are excluded from the
completely, and can be adapted for use with the scope of the index (for example, employee’s
CPI. contributions to social security schemes). These
are collected to finance social security schemes
• Prices. For some service categories it can be and these payments do not directly relate to
difficult to obtain prices because there are provision of goods or services. Reimbursements
either few or no clearly‐defined or established that do not directly relate to goods or services
outlets. Examples include domestic services are not in scope.
such as in‐home child care, maid services, and
private protection services. For these the CPI • Other payments or rebates to households by
can obtain expenditures from the household government units, social security
budget survey, but finding a stable sample of administrations or NPISHs in the form of
sellers from which to obtain prices may not assistance to reduce household expenditure,
such as housing allowances to tenants or
30
payments due to sickness, disability, the care of
http://siteresources.worldbank.org/ICPINT/Resources/ch5_Annex elderly relatives or scholarships to students, are
1.xls has an example of a Structured Product Description
115
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Figure 9.3.1 Health, Education and Social Protection – total expenditure and the CPI coverage
B: Covered by public (and
A: Not covered by private) insurance scheme
public or private
insurance b: Public subsidy /
reimbursement from
public insurance scheme
b': Provided for
free
b": Reimbursement
to consumer, private
insurance
C: Covered by
private insurance c: Reimbursement
scheme from private
insurance
Total expenditure (all sectors) = A+ (B+b+b’+b’’) +(C+c).
Government and/or non‐profit institutions’ expenditure = b+b’
Private insurance sectors expenditure (claims by household sector) = b’’ + c
Household sector’s expenditure net of public subsidies, gross of private insurance claims (CPI
Scope) = A+ (B+b’’) + (C+c)
considered as social benefits in cash. They are depending on the country, any or all of the
treated as income transfers to households and following: food stamps, subsidised housing,
do not constitute reimbursements prescription drugs, and job training programs.
Clearly, these in kind transfers can contribute
9.3.11 For the background we need to go back to
substantially to the standard of living of the
national accounts conventions. Goods and services
individual households that receive them.
acquired by households and for which the
government or NPISH provides a subsidy either 9.3.13 Depending on a particular country’s
through a full or partial reimbursement should be circumstances it could be that the public authorities
measured net of direct reimbursements. For have decided to partly or fully finance the provision
example, individuals that are part of a particular of certain goods and services. Regardless of the
socio‐economic group may be eligible for a full or practice, those expenditures incurred by
partial refund for dental care; if the refund covers government or non‐profit institutions to finance
the full cost of the dental service, then the social transfers in kind are unambiguously outside
expenditure weight would nil and no prices would the scope of a CPI (although it can be argued that it
need to be monitored for the purpose of the CPI. is desirable to take them into account when
estimating a comprehensive and “true” cost‐of‐living
9.3.12 Individual goods and services provided at
index. However, when the consumer pays part of the
no charge or at a subsidised price to individual
cost associated with the provision of such goods and
households by governments or non‐profit
services, then this element is within scope of the CPI;
institutions are generally described as social
although subsidised, this expenditure may account
transfers in kind and costs are included in
government expenditures31. Examples could include,
social security funds) and non‐profit institutions serving
31
Looking at it another way, services provided free as social households (NPISHs), whether purchased on the market or
transfers could, in principle, be regarded as part of household produced as non‐ market output by government units or NPISHs.
consumption expenditure but with a zero price. Social transfers in The items included are: social security benefits, reimbursements;
kind consist of individual goods and services provided as transfers other social security benefits in kind; social assistance benefits in
in kind to individual households by government units (including kind; transfers of individual non‐market goods or services.
116
SPECIAL CASES ‐ SERVICES
for a large part of a household’s final monetary on the institutional arrangements for providing
consumption expenditures. Take for example a them. Government‐provided free medical care is
subsidised rental unit for which the monthly cost of out of scope, but many countries adhere to some
providing the rental service is $800 and where, after form of two‐tier health care system whereby some
the government subsidy, the actual rent paid by the health care services, often for a small fee, are
tenant is $500. While the $300 subsidy is excluded provided by a private system that co‐exists with the
from the scope of the CPI, the rent of $500 paid by public system or where medical services are
the tenant is a legitimate consumer expenditure that subsidised partially by the government but the
should be included in the CPI. consumer pays part. In both cases fees paid by the
users are in scope. Employer‐provided medical care
9.3.14 Users’ expenditure should be included in
is beyond the scope of the CPI ‐ a transaction is not
order to properly reflect changes in prices in case
involved and it is treated in the national accounts as
governments and non‐profit institutions decide to
remuneration in kind.
introduce charges for services that were previously
provided at no charge, or vice versa. 9.3.21 Although health is a broad term, for the
construction of an elementary aggregate in a CPI it
9.3.15 To clarify the CPI coverage of the health,
generally refers to medical care provided by medical
education and social protection sectors, Table 9.6.2
professionals, para‐professionals or medical
divides the total expenditure on the basis of who
institutions.
bears the cost.
9.3.22 COICOP divides the health care sector into
9.3.16 Table 9.3.2 highlights possible
classes according to type of medical care provider ‐
combinations of payment and reimbursement
doctors, hospitals, etc. This facilitates sampling, price
systems. In practice the different mixes between
collection and index construction. The goods and
publicly financed social security schemes, employee
services covered by the Health category (COICOP
or worker financed social security schemes and
category 6) consist of the following sub‐categories.
social security schemes financed by individuals may
be more complicated. 06.1 Medical products, appliances and
equipment
9.3.17 Thus, the subsidised nature of the health,
education and social protection sectors is a 06.1.1 Pharmaceutical products
challenge for price index compilation. The national
06.1.2/3 Other medical products, therapeutic
index methodologies have to be built around the
appliances and equipment
national social insurance system.
06.2 Out‐patient services
9.3.18 Note that the procedures used in the
treatment of tariffs, as described in Section 9.3, are 06.2.1‐3 Medical and paramedical services
often applicable for pricing schemes encountered in
06.2.2 Dental services
health, education and social protection services.
06.3 Hospital services
9.3.19 Finally, it should be noted that to it is likely
that detailed in‐depth research will need to be 06.3.0 Hospital services
undertaken to identify and measure newly
9.3.23 In addition, it also includes health services
introduced significant charges. Some of the goods
purchased from school and university health centres.
and services in health, education and social
protection are difficult to measure at constant 9.3.24 The weights are the amounts consumers
quality. The social security systems can also cause spend on each type of provider. In countries where
complications. The CPI compiler is advised to keep in the government does provide a substantial portion
close contact with their policy colleagues to obtain of medical care free of charge, the relative
the information which is necessary in order to make importance of medical care in the CPI is less than its
informed decisions about which charges should be expenditure share as measured by the national
included in the index and how they should be accounts.
measured, and also to gain access to relevant
9.3.25 Some critics have noted that the COICOP
sources of data.
categories are activity classes — not classes
Health services (doctors and according to consumers’ purposes. These critics
state a preference for categorisation by types of
dentists) medical condition and are of the view that the CPI
9.3.20 How to handle health services in the should measure the cost of treating a disease or of
compilation of individual countries’ CPIs will depend obtaining a fixed outcome after treatment rather
117
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
than the medical service or treatment itself, which, that the input approach ignores medical advances
in their terms, is an input. But it should be noted that and that it also loses sight of the fact that the
the “medical condition approach,” which would patient, unarmed with medical knowledge, generally
define elementary aggregates for categories of looks to buy a cure to a medical condition rather
medical conditions, is not sufficiently developed to than a particular course of treatment. Under the
be recommended at this time. The recommended treatment approach the price statistician or collector
procedures are as follows. firsts selects a specific medical problem by selecting
a disease or injury for which a patient recently
Sampling received treatment and then follows the price of
9.3.26 The general CPI practice is to select treating that disease or injury regardless of medical
samples of providers within each COICOP category procedures or medicines used. For example, if
(doctors in Medical Services, dentists in Dental treating the disease or injury initially requires five
Services, etc.), and then choose one or more service visits to the doctor, the CPI price is the cost of five
items for each sampled provider. The principle of visits. However, if subsequently the doctor reports
“loose specification when sampling and tight that he or she is now using a new way of treating the
specification when pricing” is followed. When first condition that requires only two visits, then from
visiting a doctor or other medical provider, the price that point the cost of two visits is included in the CPI.
collector should find out what services are provided 9.3.29 Services in health are included into the CPI
or medical procedures are performed and select a only in respect of household expenditure on these
representative sample with guidance from the services, i.e. the payments that are made as a direct
provider. The price collector should describe them consequence of purchase of individual services. It
as completely as possible, and then continue pricing follows that these prices, whether subsidised or not,
them as long as possible or until they are no longer should be net of direct reimbursements. Obligatory
part of the sample. One approach to the initial payments, such as employees’ contributions to social
sample selection of services or medical procedures is security schemes, are out of the scope of a CPI.
for the price collector to ask the medical practitioner These are collected to finance social security
or dentist what was provided recently to a “typical” schemes and do not directly relate to provision of
patient with a representative medical condition or to goods or services. Reimbursements that do not
ask the respondent to describe a simple service they directly relate to goods or services, i.e. where the
recently performed. When returning to obtain future recipient has the opportunity to spend the money on
prices, the price collector should collect the price for something unrelated to the treatment which is the
that identical service, even if the doctor or dentist subject of the reimbursement, are also excluded.
has not performed it recently. If, however, the
respondent has not performed it for a year or says 9.3.30 Both approaches – the input approach and
that he will no longer perform that particular the treatment approach – may involve “quality
procedure, the price collector should find a adjusting” the prices where a change of treatment or
replacement procedure, preferably for the same course of medication results in a change in outcome.
purpose. Headquarters should then judge if it can be For instance, if in the above example the reduction
treated as a comparable item. The need to price to two visits to the doctor is associated with a
many types of medical services from the same shorter course of treatment and with the patient
doctor reduces where it is known that price being in less pain and suffering fewer adverse side‐
movements among various medical services move effects, then the “quality” adjusted price should see
more or less closely together. a steeper price reduction due to the better
treatment.
Pricing
Coverage of Medical Insurance
9.3.27 There are two approaches to pricing. The
traditional input approach treats medical items as 9.3.31 The existence of medical insurance
just consumption items, without regard to their confuses the situation further. Some consumers buy
effectiveness in preventing, curing or ameliorating medical care directly from medical care providers,
an illness or injury. The approach prices particular while others buy medical insurance that pays for
medical items, such as an annual check‐up by a some or all of their medical care. It is, perhaps,
doctor, or a particular surgical procedure, such as an useful to review exactly what kind of product the
operation in a hospital or clinic, and follows the cost consumer is actually buying. Medical insurance may
over time. be considered a way of prepaying for likely future
medical expenses, a way of reducing the risk of
9.3.28 The alternative approach, the treatment catastrophic expenses (i.e. giving peace of mind),
approach, is a partial response to increasing criticism and a way of reducing the total expected cost of
118
SPECIAL CASES ‐ SERVICES
119
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 9.3.2 CPI Weights: medical insurance and medical care elementary aggregates
Health
Total Insurance Hospitals Doctors Drugs
premiums
Budget Survey data 1800 1000 300 300 200
= CPI weights with gross insurance
Insurance industry data 900 500 300 100
CPI weights with net insurance 1800 100 800 600 300
CPI weights with no insurance 1800 0 856 633 311
9.3.40 The price relative for the net insurance weights yields a small aggregate for net insurance
premiums aggregate can be developed in the same premiums, as shown in the third row. Alternatively,
way as under the gross premiums method, using a the CPI could allocate all insurance premiums to the
sample of insurance policies. The problem of holding medical care aggregates, as in the last row.
the policies constant over time remains, of course,
but this is less troublesome because the weight is Education
much smaller. Alternatively, a measure of the
change in the insurance companies’ net earnings 9.3.44 The scope of the CPI is limited to payments
(premiums less payouts) can be used. This may be actually made by consumers. Consequently, fully
volatile and net earnings could actually be negative publicly‐funded education is out of scope of the CPI.
in particular years, and so statistical offices who use However, there are usually some minor ancillary fees
this approach often take an average figure over a for materials or services, such as pens and paper,
number of years. In addition, such a measure and sports activities that students must pay for and
requires data that insurance companies may be these should be included in the index. Out‐of‐pocket
reluctant to provide. educational expenses for tuition and related costs
are treated as standard consumption items. In these
9.3.41 It is important that the CPI reflects the circumstances the CPI weight for education is the
prices that insurance companies pay as well as those out‐of‐pocket expenditures as reported in the
prices that self‐paying consumers pay “out of Household Budget Survey.
pocket”, particularly so given the fact that there can
be significant price differentials. Providers, however, 9.3.45 The goods and services included in the
may be reluctant to reveal or have difficulties in Education category (COICOP category 10) cover
calculating what the insurance companies actually educational services only. As well as the main
paid, so it may be difficult to collect these prices, and channels of education, this includes:
it is rare for CPIs to use them.
• Education by radio or television broadcasting
Weights • Educational programmes, generally for adults,
9.3.42 In the first row of the example in Table which do not require any special prior
9.3.2, the Household Budget Survey reports that instruction, in particular vocational training and
consumers spent (from their own pockets ‐ cultural development
government‐provided benefits are not in scope) a • Literacy programmes for students too old for
total of $1800 on medical care, $1000 on insurance primary school including out‐of‐school
and the rest on direct payments to medical secondary education for adults and young
providers. Under a gross premiums approach, these people and out‐of school post‐secondary non‐
are the weights for insurance and medical care tertiary education for adults and young people.
categories, i.e. standard CPI weights: expenditures
reported in the HBS of what consumers paid out of 9.3.46 It does not include:
their own pockets. • Expenditures on educational materials, such as
9.3.43 To obtain weights for a net premiums books (09.5.1) and stationery (09.5.4), or
approach more data are required. The second row education support services, such as health care
has the insurance industry’s spending on medical services (06), transport services (07.3), catering
care ‐ the insurance companies spent $900 in services (11.1.2) and accommodation services
benefits and retained $100 to cover their costs and (11.2.0).
their profits. Using this industry data to construct CPI
120
SPECIAL CASES ‐ SERVICES
• Driving lessons (07.2.4); recreational training offered in exchange for work, e.g. teaching
courses such as sport or bridge lessons given by assistants, or that require participation in sports
independent teachers (09.4.1). teams should not be reflected in the index as they
are a form of income.
9.3.47 The same principles apply as in health
services. The compiler is advised to refer to COICOP 9.3.51 The treatment in the CPI of scholarships to
for guidance. particularly talented students or tuition assistance in
the form of financial aid to students from certain
9.3.48 In many countries, the government
socio‐economic backgrounds such as low‐income
partially subsidises education, particularly higher
households depends on the method of payment:
education, and students pay a portion of the cost of
their education and this portion is in the CPI’s scope. • Those that are paid directly to students to assist
The tuition fees faced by students are not always the them in bearing the cost of the full fee but
full cost of education, and in these cases the CPI which the students can use, if they wish, to
weight should not and does not represent the full finance something else, are transfer payments
cost of providing education to the population, just that the CPI should not reflect.
the cost to the individual of acquiring an education.
• Where the scholarship or subsidy results in
Changes to the amount of the government subsidy
students being subject to reduced fees, i.e. they
can lead to changes in the tuition charged to
are billed for a reduced amount, it is the
students and the index should show any resulting
reduced fee which should be priced for the CPI.
change in the fees. If a student goes to a private
school, where there is no subsidy, the total Weights
advertised price P for fees should be reflected in the
CPI weights and in the price paid, as this is what it 9.3.52 Weights for educational services can be
costs the household to educate the student. In the difficult to obtain. Information from the HBS can be
case of a government subsidy, let s be the share of unreliable and National Accounts principles only
this total advertised price which is actually borne by allocate the costs incurred by households as a part of
the household (i.e. the household is presented with household final consumption expenditure. The level
a bill showing a net price of sP), then the appropriate of detail in accounting is often varied and may not
price for the CPI and for the construction of weights allow the division of the costs to the level of detail
is sP. In this way, the proportion of the cost borne by required for index compilation. Therefore a better
the household will feed through into the calculation source of detailed weighting information may be the
of the CPI, i.e. the CPI will reflect the actual fees paid public institutions in charge of the educational
by the household which, in this case, will be less than schemes.
the basic advertised price. Pricing
Pricing – tuition fees 9.3.53 The recommended approach is to use
9.3.49 The typical item to be priced will be the “student profiles” to price the cost of education for a
cost of a term or semester at a sample of schools. A selection of typical students who are chosen to
school term is usually longer than one month, which reflect different levels of fees and base‐weighted to
is the calculation frequency for most CPIs. Standard reflect their distribution in the particular educational
practice in this case is to collect prices only in institute which is being priced. Where the fee
months when the terms begin, and use that same structure is straightforward it may be sufficient to
price in the intervening months. The pricing months simply collect prices for the posted or advertised
can vary depending on the country or the school. For tuition fees directly from the institution or its
example, if a school has two semesters, one starting website. Any discounted tuition fees can be ignored
in September and the other in January, then tuition where it is known or can be assumed that they
fees should be priced in these months only. In other change at the same rate as the full fees and that the
months the price is “carried forward” i.e. the same proportion of students benefiting from them
price is used for the remaining months of the term. remains constant over time. However, any
The index and the associated inflation rate may assumptions would need to be checked on a regular
display step changes, changing only in the months basis and there are inherent dangers of introducing
when the terms begin. an element of bias in the index if differential changes
in fee structures or changes in the student
Scholarships and other forms of discounting population are missed. The approach should be used
with care. Similarly, the use of average revenue
9.3.50 Schools and universities often reduce the
(total tuition fees divided by the number of students)
tuition for some students. Tuition fee reductions
is not an appropriate way to price tuition fees which
121
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
122
CHAPTER 9.4
Special Cases ‐ Tariffs
Introduction been centrally fixed by the supplier, by the
government, or by agreement to exert influence on
9.4.1 The topic of tariffs is covered in paragraph the consumption patterns by means of appropriately
6.21 of the CPI Manual in the context of the differentiated prices and conditions according to
treatment of variable pricing policies dependent on characteristics of consumers, the level, the structure
demand, and in paragraph 9.14 in the context of the or the timing of the consumption.”. The article goes
assignment of weights. Paragraph 6.21 states the on to define a tariff price, which is “a price within a
principle that “price collection should be made tariff that applies to a component element or unit of
consistently over time and in a way that represents consumption of the good or service in question.” At
consumer purchasing patterns” and that the its most basic, a tariff consists of a list of prices
selection of representative items should “represent based on detailed specifications of the services that
consumer behaviour” and “be weighted by are priced individually but can only be bought as part
consumer spending patterns”. Paragraph 9.14 states of the package.
that in the construction of elementary aggregates, 9.4.4 Economic theory recognises different types
which do not normally involve weights, “a special of price‐discriminating behaviour on the part of
case occurs in the case of tariff prices” where “it is sellers. Consequently, there may exist in the market
appropriate to assign weights to the different tariffs many tariff pricing schemes. Some of the more
or prices in order to calculate the price index for the common variants that are present in the market and
elementary aggregate”. By way of an illustrative which will be discussed in this section of the chapter
example, paragraphs 10.91 to 10.116 of the Manual are as follows:
then discuss the subject of tariffs in some detail in
the context of the construction of price indices for • Peak‐load pricing (also known as “congestion”
telephone services. pricing). This occurs when producers charge
higher fees during periods of greater demand,
9.4.2 The ILO Resolution makes no explicit usually because of the higher costs of
reference to tariffs, but paragraph 58 states that “for production caused by capacity constraints. Peak‐
goods and services where the prices paid are load pricing thus helps in balancing capacity
determined by combinations of subscription fees usage over a period, which decreases the need
and piece rates (e.g. telecommunications) care must for firms to invest in costly infrastructure
be taken to ensure that a representative range of expansions. Such pricing practices are often
price offers are observed” and that “care must also found in the areas of toll roads and bridges,
be taken that prices charged to different types of ferry services, electricity, and long distance
consumers are observed”. telephone calls.
• Two‐part tariffs. These occur when consumers
Background are charged both an entry (or lump‐sum) fee
and a per unit charge. In other words, a fee is
9.4.3 In paragraph 9.14 of the CPI Manual a tariff charged up front for the right to use (or buy) the
is defined as a list of prices for a particular kind of product and an additional fee is charged for
good or service supplied under different terms and each unit that the household consumes.
conditions. A similar but more detailed and possibly Examples where two‐part tariff pricing is often
more helpful definition is provided in Article 2 of applied are:
Regulation (EC) No 2646/98 of the Commission of
the European Communities for the purpose of the o Amusement parks that charge an entry fee
European Harmonised Index of Consumer Prices in addition to a fee for each ride.
(HICP). It states that “a tariff is a list of pre‐ o “Membership” discount schemes or
established prices and conditions for the purchase shopping clubs that require the purchase of
and consumption of one and the same good or a membership card to access the point‐of‐
service, or of similar goods and services, that has
123
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
sale and then the consumer pays for his or The main measurement issues
her purchases at a lower than otherwise
price. These should be covered in a CPI. 9.4.7 Tariff‐type pricing schemes are used
They are not the same as discounts that are particularly for services and as such suffer from a
restricted to particular groups and which number of price‐index problems associated with
are generally ignored in a CPI as they do not services.
apply to all consumers. The latter are
covered in Chapter 6. • Charges for services can be subject to such
complex pricing schedules that it is difficult to
o Landline telephone services where there is
select the appropriate prices for inclusion in the
sometimes a fee to use the service ('line
construction of the price index.
rental') and also a fee per call. The line
rental covers the cost of providing the • Identification or specification of individual
service and the “per minute” or metered products or units of consumption themselves
charge, covers the cost of placing the call on can be difficult, particularly when they are
the network. offered as bundled packages consisting of
o Taxi fares where a variable fare based on several types of micro‐expenditure that may be
distanced travelled is added to the base priced separately but are typically only available
“fixed” initial charge which is not as a part of a package.
dependent on mileage. • Services are often provided under long term
• Block pricing is when prices vary according to contractual arrangements and these may
the volume consumed. Electric utilities, for include different types of “customer loyalty”
example, often charge a per KWh price up to X rebates, clauses for the minimum duration of
units consumed and then charge a different contracts, surcharges for the provision of
price for the remaining units. services not foreseen in the contract, etc.
Loyalty rebates or coupons are usually ignored
• Special group discounts. This is where members due to difficulties of measurement (see Chapter
of certain segments of the population such as 6).
students and seniors may be charged a lower
price for the good or service compared to the • There are often difficulties in accounting for
general population. Bus fares and hair cuts are substitution between different providers of the
good examples of services where these group same type of service, and in accounting for
discounts are applied. quality differences in the services provided.
9.4.5 In some countries, private and public firms • There can also be difficulties in accounting for
have come to rely increasingly on tariff‐type pricing customers moving under their own volition from
as a tool for generating additional revenues or for one tariff to another under the same service
public policy purposes. For example, publicly provider. For instance, when a mobile telephone
administered electric or water utilities may charge a company offers a number of different pricing
lower tariff for the first units of electricity consumed packages targeted at different patterns of
up to a certain level (the first block), followed by a usage, current customers may undertake
higher tariff for subsequent units consumed (the periodic reviews during which they may decide
second block). Such an increasing block tariff to move to a more advantageous tariff for their
strategy is often used, it is argued, to promote better level of usage.
and greater access by lower‐income households to
what is considered a vital service. Private sellers can Treatment of tariffs and the
often increase their profits by charging a higher price
for a product to some consumers and a lower price information requirement
to others if it can successfully segment its customer‐
base. 9.4.8 Although there is no generally agreed
approach about the exact treatment of tariffs within
9.4.6 Clearly, tariff‐type pricing schemes can be the framework of the CPI, the HICP regulation
applied to a wide range of products. The extent of mentioned in paragraph 9.4.3, although developed
the practice will vary according to the country. In for the EU, does offer some general and reasonable
cases such as electricity, the weight of the guidelines which can be applied to the national CPIs
component can be relatively high so it is important of other countries.
to measure accurately the price change for such
commodities.
124
SPECIAL CASES ‐ TARIFFS
9.4.9 According to this regulation, the basic validity of the approach. Neither of the matching
information requirement for a tariff should include approaches is suitable for situations where there are
“all tariff prices and weights which reflect the major changes in the structures of tariffs. The
structure of the consumption of the good or service matched sample methodology cannot usually be
according to the characteristics of the consumers, applied where the structures of tariffs regularly
the level, the structure or the timing of the change.
consumption”.
Unit Values
Price measurement methods 9.4.14 The unit‐value approach, where the overall
average price of one unit, for example of electricity,
9.4.10 There are four basic approaches to tariff for customers of a particular supplier is used for
pricing ‐ matched samples, unit values, customer monitoring prices, instead of following individual
profiles, and the sample of bills approach. elements of the tariff. The contents of the tariff‐
based service are homogeneous (e.g. KwH of
Matched Samples electricity) and the method should be used only in
9.4.11 Matched samples are where a full tariff list such cases. The unit value is calculated using overall
or an element of the tariff structure is treated as a revenue and quantity data at a higher level that does
“product specification” and re‐priced in subsequent not distinguish between different tariffs or
periods. This mirrors the traditional CPI customers. It attributes all the differences between
methodology of matched pairs. The simplest form of different packages to price alone i.e. quality
traditional matching methodology would be to take differences between different pricing approaches
the tariff price lists of some major companies and are assumed to be zero or insignificant. This may not
follow the changes in the entire pricing scheme, be the case where, for example, the reliability of
preferably with accurate weighting information for continuity of supply varies between suppliers. A
the different elements of the price list. This option particular advantage of this approach is that it can
may be feasible in markets of limited competition be easier to compute. But because of this mixing of
where the pricing structures should be quite stable. quality and quantity, unit value indices are generally
There are practical challenges associated with not considered ideal for constructing CPIs and are
ensuring that exactly the same product specification best avoided unless it can be proven that their
is priced each month, i.e. that the matched pairs weaknesses have no significant numerical impact.
methodology is adhered to. There is usually limited Customer Profiles
data about the customer base. In practice the service
to be priced using a full price list usually also has to 9.4.15 Pre‐specified consumer profiles. The idea of
be priced in a rather simple manner without too consumer profiles is to define the product
much differentiation on the basis of customers’ independently of a single producer’s or supplier’s
characteristics, i.e. for the sake of simplicity some tariff structure. Instead, a more general formulation
variations in tariff are sometimes ignored. of the consumer’s behaviour is constructed based on
information (for example sales information provided
9.4.12 Another version of matching methodology by the industry as a whole) which can be used to
is to select some elements of a tariff as define a range of typical consumers, irrespective of
“representative items” and re‐price them in supplier. Different suppliers’ services are priced
subsequent periods. For each representative item, through these typical consumers. A unit value price
detailed specifications combined with information is calculated for each consumer profile and then this
supplied by the provider, are constructed. For unit value is recalculated over time. The advantage
instance, for air fares this could be for each airline of this approach is that it does not require an actual
carrier, a non‐refundable and non‐changeable airline sample of bills, only the details of the different tariffs
fare from one pre‐specified location to another, with and some information on usage by “typical”
pre‐determined outbound and inbound dates customers. Also the price statistician could
chosen by time of day and day of week, including all potentially define consumer profiles by class of
surcharges). household, e.g. prices for the poor versus the well
9.4.13 The representative tariff element has the off could be computed.
advantage that it can be applied to different service
Sample of Bills Approach
providers, at least in principle. It becomes possible to
incorporate new service providers into the index 9.4.16 The sample of bills approach. This is a more
without significant difficulties. However, the choice refined version of the customer profile approach
of only a few items as representative may limit the where a level of service activity from an actual
125
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
sample of customers is priced each month rather tariff changes, such as reduced unit charges
than defining profiles representative of the average when a minimum threshold of consumption is
monthly activities of a range of customers (see reached.
above). This can be done, for example, by selecting a
9.4.17 It should be noted that there can be an
sample of customers from each category of
inherent time‐lag due to bills being issued after the
customer, the latter being chosen to reflect the
index for the relevant time period has been
structures of the tariffs of the corresponding service
compiled.
providers. For instance, a sample of customers and
their bills might be drawn from low, medium and 9.4.18 In each of the above approaches the
high volume consumers of the product. The resulting resulting calculation should be based on prices and
index measures the cost of the current billing weights which reflect the structure of the
period’s consumption (normally over a month or a consumption of the goods or services according to
quarter) at prices charged in the index period the pre‐determined characteristics of the consumers
compared with prices charged in the base period. and the level, the structure or the timing of the
The sample of bills approach has a number of consumption. The corresponding checklist of price‐
differences compared with the customer profile relevant characteristics for tariff‐based expenditures
approach, most particularly: may include:
• It takes into account in‐year variations in • The time‐pattern of use of the service. The
consumption, for example, a greater volume of patterns of use may be measured over one day,
international telephone calls during public one week or even one year.
holidays and festivals.
• The volume of use of the service, for example
• It reflects actual customer behaviour by where pricing schemes differentiate between
reference to bills. small and large customers.
• It detects price changes not associated with • The past behaviour of the customer. Particularly
Table 9.4.1 Matched Models: Landline Telephones
Long‐distance 20
Peak 5 2.10 2.50 +0.40 (19.0%)
Off‐peak 15 0.90 1.10 +0.20 (22.2%)
International calls 30
To Cape Town (service 10 3.50 3.50 0.0 (0.0%)
provider A)
To Singapore (service
10 4.20 4.40 +0.20 (4.8%)
provider B)
To Washington
(service provider C) 10 3.90 5.10 +1.20 (30.8%)
126
SPECIAL CASES ‐ TARIFFS
127
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 9.4.2 Consumer Profiles: Mobile Telephones
Calls
Within same mobile network
To different mobile network
To landline
Other services
128
SPECIAL CASES ‐ TARIFFS
service used. In the matched sample approach, the values perform best when there is a high degree of
quality change situation can be thought of as similar homogeneity. To prevent quality differences from
to the situation confronted by a disappearing affecting the price index, the specifications of the
product, i.e. that some pre‐specified element of the “units of consumption” should be as detailed as the
tariff is no longer available for pricing. A replacement available data permit to maximise homogeneity.
should be selected for the disappearing element of
9.4.34 In the example below (Table 9.4.3),
the tariff and the elements compared directly
relating to bus fares for juveniles, the previous tariff
(where applicable) or treated as entirely new
categories have to be ‘transformed’ into the new
products. An illustrative example follows for public
ones. For customers aged from 12 to 16 there was a
transport bus fares.
price rise of 60 percent (from $5 to $8) while for 16
Public bus transport in City X year olds there was a price decrease of 20% (from
$10 to $8). The proportion of customers falling in
Old tariff element. Bus fare from city centre X to these categories, i.e. the previous consumption
Suburb Y, on a Saturday, 10.30 pm. (night tariff, last patterns, should be used to calculate the average
connection) price change, or in the absence of such information
• New tariff element. Bus fare from city centre X the price statistician might make an assumption
to Suburb Y, on a Saturday, 9:30 pm. (night about the proportions based on the available
tariff, last connection). information, including revenue information from the
bus company.
9.4.31 The basis for the direct comparison is the
index compiler’s judgement, that from the Table 9.4.3
consumers’ point of view the change in timing is not
a significant change. Current ticket New ticket
9.4.32 The price statistician could also reason that prices ($) Prices ($)
the quality of the service has changed and an 5‐11 years
allowance for that should be made in the index. But Children
5‐15 years = $5
in drawing this conclusion and making a quality = $5 12‐16 years
adjustment the CPI statistician should not use their Teenagers
= $8
subjective judgement or personal view. In this
16 and above 17 and above
particular example some travellers may consider this Adults
= $10 = $10
change in tariff and timetabling as a reduction in
quality of service if last shows at cinemas finish at
9:45 pm mean that a bus journey is replaced by an Consumer profiles
expensive taxi journey home, but it could well be 9.4.35 The key issue when dealing with quality
that other users of the bus actually welcome the changes in the context of consumer profiles is the
earlier timing because most restaurants close at 9:15 choice of which consumer profiles should be priced
pm and they can now avoid waiting for a bus until through the period of tariff change and/or change in
10:30 pm. in the cold. So for them it is a quality service provider. The example of a profile given
improvement. If possible the price statistician below is based on a situation where broadband
should canvass the views of users for an indication of internet providers increase their download and
whether the change is considered to be for the upload speeds substantially, while keeping prices
better or for the worse. An indication of subsequent constant, or reducing them. In this example it is
changes in passenger numbers may confirm whether assumed that the differentiating factors of an
the initial assessment was correct but even this internet‐provided broadband connection are the
involves some judgement. The decision to use direct download and upload speeds and that this is
comparison, without quality adjustment, is always to reflected in different tariffs (see Table 9.4.4). It
a certain extent judgemental and should therefore therefore follows that the increase in speed is a
be based on explicit lines of reasoning so that it is quality enhancement – in the market place it is a
transparent and can be justified. price‐determining characteristic. As a result of the
increase in speed, it will take less time to download
Unit values
and upload files. However this time advantage will
9.4.33 The unit value approach regards quality only be noticeable when downloading or uploading
differences related to different pricing schemes as large files.
implicit price differences. The detection of quality
change depends on the degree of detail in the
specification of “the unit of consumption”. Unit
129
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 9.4.4 Internet Services
9.4.36 The assumption is made that the product and high volume users (in each period) determines
versions provided are also, in effect, descriptions of the volume associated with this fee. The key point is
consumer profiles: that a unit price for the ‘volume of use’ is calculated
for each class of consumer, in essence, a form of
• Low volume users who are mainly interested in stratification is applied. The base period and current
having access to the internet when needed. period situations are as follows.
• Medium volume users who require and pay for 9.4.39 In the far right‐hand column of Table 9.4.6
some additional functionality. (the second of the tables below), the total price of
• Heavy users who are ready to pay for additional the new scheme has been re‐calculated using the
performance. user‐based structure – in this case the features of
the packages provided – for the comparison period.
9.4.37 It can therefore be argued that the fee, for The quality adjusted price change, which allows for
low volume users and their associated quality changes in different schemes, is obtained by
download/upload speeds, is a price for access to the dividing the price of the adjusted new package by
service while the more expensive versions also the price of the old package in the base period, the
contain a price for the use of the service. latter being given in Table 9.4.5 ‐ (49.85/56.38 =
9.4.38 The price for the use component – which, 0.88), i.e. a 12 percent price decrease.
as described above, only exists for medium and high 9.4.40 The direct comparison of the two tariff
volume users – is derived by subtracting the fee for schemes, disregarding the quality change, would
the cheapest available subscription from the total result in price change 54.84/56.38 = 0.97, i.e. a 3 per
fee. Subtracting the download speed of the low‐ cent decrease in price.
volume user from the speed provided to medium
Table 9.4.5 Internet Prices: Base Period
Table 9.4.6 Internet Prices: Comparison Period
T0 Share of Download Total Price Price Volume Unit Total Total price
products speed price of the of use of use price price of of the
access of use the scheme,
scheme T0 user
structure
Light 0.25 384 34.90 34.90 0.00 0 0.000 34.90 34.90
Basic 0.5 1024 52.25 34.90 17.35 640 0.016 52.25 47.91
Fast 0.25 2048 79.95 34.90 45.05 1664 0.022 79.95 68.69
Average price of the operator (weighted): 54.84 49.85
130
SPECIAL CASES ‐ TARIFFS
New service providers and new introduced (for example, multimedia messages or e‐
mail on cell phones). These should be introduced
products into the pricing schemes or consumer profiles by
chaining, once they have significant market share
9.4.41 From the consumer’s point of view many (see Chapter 8).
tariff‐priced services are relatively uniform.
Electricity delivered to the consumer is almost
homogeneous by definition, the main difference in
Classifications
service level being reliability of supply. The same
9.4.44 CPI classification systems in use by many
may hold true for other services, like telephone
countries, including COICOP, are not constructed
services or internet‐services, despite the service‐
with a stratification structure that is sufficiently
provider’s attempts to differentiate their products
detailed to reflect the various tariffs for those
and make their pricing structures more and more
commodities that are subject to such pricing
opaque. Therefore different suppliers of tariff‐priced
practices. For example, electricity under the COICOP
services could be considered as providing the same
hierarchy appears as a product Class (4‐digit COICOP)
type of service and treated as substitutes for each
in its own right (04.5.1) and according to its official
other.
description includes: associated expenditures such
9.4.42 Changes in the market mix of a clearly as hire of meters, reading of meters, standing
homogeneous product from different tariff charges, etc. A finer breakdown is not given in
structures and suppliers should be incorporated in COICOP. However, when sellers of a commodity,
the index. Index construction will require such as electricity, use tariff pricing, the accuracy of
information about the market share of different the CPI can be improved if the price index for this
producers and the various service plans. As a general commodity is constructed in a way that reflects as
principle, for homogeneous services the price impact accurately as possible the market realities. In other
of a new service provider should be shown in the words, a class‐level price index should be comprised
index, i.e. the target price measure for of a number of sub‐indices, each one corresponding
homogeneous services is the unit price in the overall to its particular tariff price. This may require the use
market for the service, quality adjusted as necessary. of a specially designed classification for the purpose
9.4.43 In some tariff‐priced parts of the retail of stratification within an elementary aggregate.
market new service elements are frequently
131
CHAPTER 9.5
Special Cases ‐ Seasonal Products
Introduction excluding all the possible ‘problematic’ products
from a CPI is not a solution in the context of an index
9.5.1 The topic of seasonal commodities, prices whose purpose is to reflect changes in all
and weights is covered in Chapter 22 of the CPI consumption prices. If these products have some
Manual and is mentioned in various paragraphs in importance in the index basket then there is no
other chapters. It distinguishes between “strongly” justification for ignoring them. This section looks at
seasonal commodities which are available only part the alternative ways of dealing with strongly
of the year when “in season” and “weakly” seasonal seasonal products.
ones are available throughout the year but their
prices (and their consumption) fluctuate with the Background
time of the year. “Weakly” seasonal commodities
generally do not require any direct intervention by 9.5.4 Climate, traditions and institutional
the compiler. The seasonal fluctuating prices of arrangements are the main causes of seasonal
“weakly seasonal products” will typically be captured unavailability. Fresh fruit and vegetables often have
in the index although they are not without their a particularly marked seasonal purchasing and
problems for users of a CPI. For instance, when the consumption pattern and certain fruit and
“weakly” seasonal commodity is “out of season”, its vegetables may not be available at all at certain
price may be unusually high or low and the annual times of the year. Other products which can display
basket will reflect these unusual price fluctuations, some seasonality include clothing, water and fuel.
leading to seasonal fluctuations in the overall index. The list of seasonal commodities cannot be expected
This volatility can cause “statistical fog” with the to be uniform across countries. For examples,
analysis and understanding of inflation. For some oranges may be available for purchase year‐round in
purposes users want a CPI which measures the warmer climate countries but only in certain times of
underlying price change and not these seasonal the year or at a premium price in cooler climates.
fluctuations. The use of moving averages can help as Certain religious and other festivals can also be
can various measure of “core inflation” (see Chapter associated with goods or services whose
11). But it is the “strongly” seasonal products which consumption is limited wholly or partially to the
pose the biggest problem for the index compiler. festival period, such as Christmas trees, or products
which are in high demand or especially produced,
9.5.2 The 2003 ILO Resolution recognises that
such as some gifts given at the end of Ramadan.
seasonal products should be included in the CPI
basket but may need special treatment. It states that 9.5.5 Strongly seasonal items are the most
“the way these products are dealt with should be challenging for index compilers since having
determined by the main purpose of the indices, different bundles of items in the CPI basket in the
national circumstances and the practicalities of months (or quarters) being compared leads to a
compilation”. It then goes on to refer to two breakdown of traditional bilateral index number
methods for their inclusion; a fixed‐weight approach theory. In the context of producing a monthly or
which uses the same weight for the seasonal product quarterly CPI, it must be recognised that there is no
in all months using an imputed price in the out‐of‐ completely satisfactory way of dealing with strongly
season months; a variable weights approach where a seasonal commodities. This chapter presents several
changing weight is attached to the product in various solutions to overcome the problem, which include
months. The same two main methods are covered in traditional month‐to‐month indices (along with their
the CPI Manual for dealing with “strongly” seasonal limitations) and annual approaches, which take
products. advantage of the fact that seasonal items might
appear in broadly the same month every year. Index
9.5.3 There is at present no generally agreed
number theory does provide particularly effective
approach as to the best way to treat strongly
solutions where the focus is to compare prices in
seasonal commodities. Ignoring the issue and
one month with prices in the same month a year
132
SPECIAL CASES ‐ SEASONAL PRODUCTS
earlier. But the estimate of month‐on‐month 9.5.9 The other approach to imputation is to
inflation can vary, depending on the approach which carry forward prices. In other words:
is used, making the analysis of short‐term inflation
trends difficult, since in the absence of price quotes pˆ m = p m −1
in particular months, no month‐to‐month price ratio
9.5.10 The main drawback of the fixed weights
can be compiled without imputing a price. The
approach is that the annual fixed weights will not be
results will depend on the method of imputation.
representative of the monthly consumption pattern.
Alternatively, a zero weight can be attached to the
Oranges might be available for sale only in some
missing product but this also raises issues and
months but the fixed weights approach “pretends”
hinders month‐to‐month comparisons.
that they are on sale throughout the year and
9.5.6 The difficulties that are raised by the introduces artificial prices which do not actually exist
existence of “strongly” seasonal items and their in the market place but which are based on the price
seasonal unavailability can be tackled by choosing movements of a “comparable” product, such as
one of two main approaches: bananas, that in reality may be of limited
comparability. The limited comparability of the
• The fixed weights approach. Allocating fixed
“comparable” product can lead to an extreme
annual weights, assuming that seasonal
fluctuation in the oranges sub‐index when oranges
products are to be treated in the same way as
return to the market and are available again for
all other products.
direct pricing. But that said, in practice the fixed
• The variable weights approach. Allocating weights approach does normally give reasonable
variable weights, according to the consumption smooth 12‐month rates of changes. The imputation
pattern found in the base period. of prices by reference to the most “similar” group of
products normally reflects reasonably well the price
Fixed weights approach inflation expectation when the product returns to
the market and thus is often considered appropriate
9.5.7 This approach, where the weights remain for the medium and long‐term measurement of
constant over all months of the year and which inflation. However, the use of carry‐forward prices
imputes a price when the product is out‐of‐season can, of course, bias the month‐to‐month changes to
and not available to price, is theoretically consistent zero.
with the concept of a fixed basket. However, it raises 9.5.11 There are alternative methods for the
the issue of the choice of imputation method for the imputation of prices to the one described above. For
unobservable out‐of‐season prices. The most example, another option is to apply the following
commonly used approach is to impute a price using procedure:
the last available observed price (that is “reliable”)
and multiplying this last available price by the • In the first month of the out‐of‐season period
amount of price inflation for the most “similar” the price of an out‐of‐season product is
group of products that has taken place since the estimated by a typical or average price observed
time of this last available price. The latter might be, during the in‐season period.
for instance, a “similar” product or group of • From the second month on, the price of an out‐
products, which is in season. The missing prices can of‐season product is estimated by applying the
be estimated using the monthly rate of change in the price change for a set of related products or a
prices of the set of products within the same COICOP product for which prices have been observed for
class or using an appropriate sub‐set. Higher level the current month to the estimated price from
aggregates inflation is likely to be influenced to a the previous step (i.e. the typical or average
large extent by factors that are not so relevant for price observed during the in‐season period).
the seasonal product. Using products from the same
COICOP class or group also enhances comparability 9.5.12 In summary, the advantage of the fixed
with the variable weights approach, thus increasing weights approach where missing prices are imputed
statistical integrity. is that it is easy to explain and implement and keeps
the annual basket methodology more or less intact.
9.5.8 The updating index could be an elementary
one that uses Dutot or Jevons averaging, depending 9.5.13 A word of caution is necessary at this
on the formula used for the CPI at the elementary point. It should be noted that ideally whatever
aggregate level or it could even be the CPI as a whole method is applied to deal with seasonal items should
if there is no “similar” group of products. be “self‐correcting”, i.e. if the item after the out‐of
season period reappears at the same price as it left
the market, then the method should ensure that this
133
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
will take the index back to 100. Similarly, if the price • Month‐on‐month price changes reflect not only
reappears 10% higher, it should take the index to changes in price relatives but also changes in
110 etc. Carrying forward is self‐correcting, provided consumption baskets (i.e. the weights). This
that the price statistician includes the full price makes it difficult to interpret month‐on‐month
change from the last carried forward price to the changes in the price index.
new price (assuming constant quality) or, if quality
9.5.17 In addition, concerns have been expressed
has changed, the full quality‐adjusted price change.
about the fact the seasonal weights are determined
by past consumption patterns so that abnormal
Variable weights approach seasonal fluctuations are not taken into account.
These fluctuations may be significant and undermine
9.5.14 This approach introduces into the index the index (although it can be argued that such
changing seasonal consumption patterns, reflecting abnormal fluctuations do not warrant special
the reality of the retail market. It is usually applied compilation procedures even when known in
within a COICOP category, the weight for the latter advance). Also, the variability in the precise timing of
being held constant. The underlying assumption is the seasons from one year to the next means that
that total expenditure on the relevant COICOP the imputation of prices is not totally avoided. For
category (often synonymous with an elementary instance, if unusual weather conditions delay the
aggregate) does not vary between different times of appearance in the market of, say, oranges, then
the year, the main tendency being for expenditure to prices would need to be imputed for the months in
switch between “similar” items only (consumers will which oranges are unavailable but where they have
tend to buy more apples if oranges are not been allocated a non‐zero weight. It should also be
available). With the variable weights approach there noted that the variable weights approach implicitly
is a different basket for each month, the general allocates the weight of out‐of‐season products to
principle being that an item’s weight in a given those products that are available for price
month is a reflection of the expenditures associated observation in the current month, i.e. products
with the items in that month. Linking the monthly which are in‐season. This means that the variable
baskets provides the means of calculating the weights approach impacts on the implicit weight
monthly changes for baskets that differ from month given to non‐seasonal and in‐season products
to month. sharing the same COICOP group or elementary
9.5.15 As well as reflecting the market, the aggregate – which could be the case in reality.
variable weights approach also has the advantage of 9.5.18 The variable weights approach can be
minimising the practice of price imputation, as prices based on one of two options:
will not be imputed for those months when the
product is not available for purchase. Prices are • A seasonal basket at base month prices
observed only in months where weights are above • A seasonal basket at base year prices (the
zero. For products that have prices observed in two ‘Rothwell index’).
consecutive months, the monthly changes of the
product price indices are computed using matched 9.5.19 Along with the previously discussed (fixed‐
samples. When weights change from month to weight) Lowe index, the Rothwell Index (also known
month, but remain positive, this does not have any as the Bean and Stine Type C) is the formula that is
influence on the price observation process. When a used by a number of national statistics institutes.
product has a positive weight after the weight has Calculating this type of index using an artificial
been zero for some months, the product index is dataset indicates that these indices can have smaller
compiled by matching the price observation in the or larger seasonal movements than the Lowe index
first month of the new season with the observations and, contrary to earlier studies, are not always less
from the last month of the previous season and volatile. Paragraph 22.11 of the CPI Manual
applying the difference to the last index of the describes the Rothwell index in more detail.
previous season to give the article price index for the 9.5.20 It is strongly recommended that the set of
first month of the new season. products defined as seasonal should not vary from
9.5.16 The variable weights approach has two year to year, unless strongly justified on grounds of
particular disadvantages necessity to keep the sample representative.
134
SPECIAL CASES ‐ SEASONAL PRODUCTS
Fixed or variable weights? Estimation of weights
9.5.22 Comparing the fixed and variable weights 9.5.26 In the context of the variable weights
approaches it can be seen that the two approaches approach, it may not be practical or possible to
are not necessarily unrelated. The imputation of create seasonal baskets for all goods and services
prices based on the price movements of similar which are subject to seasonal effects. For this reason
products – the fixed weight approach ‐ can be seen the variable weights approach is normally applied
as a form of re‐weighting where further weight is only to products for which seasonal weights data are
given to the price movements of products which are available and for which the expenditure weight is
available for pricing. But the two approaches are not, significant. For instance, some countries will thus
of course, equivalent. limit their seasonal basket to certain food items such
as fruits and vegetables. But there are additional
9.5.23 A ‘perfect’ solution to the treatment of
reasons for keeping the use of variable weights to a
seasonal items does not exist, particularly where the
minimum. These are the relations between
existence of seasonal commodities (and strongly
variations in the overall index level and changing
seasonal commodities especially) means that it is
weights on the one hand and the use of fixed
impossible to compute a completely satisfactory
monthly weights for non‐seasonal products on the
month to month index that accurately measures
other hand. This can be achieved in the first place by
month to month price change.
defining seasonal products only on the basis of non‐
9.5.24 If the focus of the CPI is the accurate availability during part of the year. Products with
measurement of annual price inflation, then the fluctuations in the consumption level between
problems associated with strongly seasonal months, but which are available all year, have prices
commodities mostly disappear (although changes in which can be measured and are not abnormally high,
seasonal patterns from one year to the next should not be considered seasonal products in the
continue be problematic32). current context.
9.5.25 The advantages and disadvantages of each 9.5.27 The Household Budget Survey (HBS) has
method should be considered and, if possible, the traditionally been the favoured source for seasonal
impact on measured inflation assessed for or monthly CPI weights where the information for
reasonableness before a particular method is recurring expenditures, such as food, is collected
implemented. The use of the index will also be an with a one or two week diary that is continuous over
important consideration. For example, if the main the year. Consequently, the monthly expenditures
purpose of the index is for indexation then the fact for many products that are deemed seasonal can be
that the variable weights approach means that the derived as long as the needs of the CPI compiler
month‐on‐month price change will reflect not only have been discussed and taken into account in the
changes in price relatives but also changes in survey design of the HBS.
consumption baskets may be considered a
9.5.28 The average expenditure per household
significant disadvantage, i.e. users might question
from the HBS for three consecutive years rather than
the meaning of comparing the price of this month’s
just one is often used to calculate the seasonal item
basket with the price of last month’s basket which
weights in order to reduce sampling error and also
might be very different. More generally, if user focus
to reduce the effect of unusual monthly patterns of
is on month‐to‐month change then the variable
consumption in any one year ‐ which can be a
weights approach is recommended, despite the
common occurrence for seasonal foods. The values
problems of interpretation, as the annual weights
for each of the 36 months are price‐updated (see
attached to each month‐to‐month price relative can
Chapter 4 on weights) to the current base month,
be misleading. If user focus is on long‐term index
January say (or alternatively the base reference
32 period, a year say33), or base period, by multiplying
The focus on annual price inflation leads to the possibility of a
further solution ‐ the construction of 12 year over year monthly by the appropriate current January item index and
Lowe indices (or geometric Young indices or four quarter over dividing by the corresponding index value relating to
quarter indices, using seasonal baskets that are appropriate for
each month or quarter. From a practical point of view, annual
33
household budget surveys would have to be augmented to cover There are some problems involved in price‐updating seasonal
seasonal commodities in more detail and this would entail more products to a given month, as some products may be out of
expense. But if a statistical agency is using the Rothwell formula, season in January – thus it may not be possible, or they have to be
then it is already collecting seasonal expenditures information. price‐updated with some imputed/estimated price development.
135
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
136
SPECIAL CASES ‐ SEASONAL PRODUCTS
9.5.36 The presentation of 12‐month rolling prices are observed or used in the compilation for
averages can be regarded as a measure of inflation that product because the product is not available for
for a year that is centred on six months prior to the pricing or only available in small quantities at
last month in the current rolling year. abnormal prices. The latter point is important
because the inclusion of abnormally high “start‐of‐
9.5.37 The Rolling Year fixed base index can be
season” prices relating to low volume sales can
viewed as a basic form of a seasonally adjusted
introduce a downward bias in the index‐ for both the
annual index. It should be noted that this type of
fixed weight and variable weight approaches. It may
index does not measure short term month‐to‐month
also cause difficulties for price collectors attempting
price fluctuations and so there is still a role for a
to find the products. For instance, seasonal fruits
“regular” month to month CPI.
and vegetables may return to the market initially at
9.5.38 The formulation for a Laspeyres index is to low volume and high prices, after which prices fall
calculate the price index, for example, for April 2005, rapidly as supply floods the market. Later, prices
with a base year of 1997. The year January to may rise as supplies fall. Statistical offices should
December 1997 must be used in the denominator analyse past data and construct some experimental
and May 2004 to April 2005 in the numerator, i.e. indices to help inform decisions, which will inevitably
the "rolling year" is used only in the numerator. involve some judgement by the price statistician.
9.5.39 In the case of a rolling‐year chained The geographical dimension
Laspeyres index, the, rolling‐year months are used
both in the numerator and denominator. The 9.5.42 The seasonal availability of products may
January to December whole year is used only once, vary between regions within a large country. There
in order to calculate the first year indices. may also be a difference in supply (and demand)
between markets in rural areas and those in urban
9.5.40 There are many examples in the CPI areas, or between market stalls and more upmarket
Manual to which the reader can refer. The detailed shops catering for the better off. For example, some
formulations are given in Chapter 22 of the CPI prices may be collectable in the capital city even
Manual. though the product is generally not available
elsewhere. Where this is the case, different seasonal
Some common implementation cycles may be applied, particularly if the differences
are marked and regional or urban/rural sub‐indices
problems are being computed. The decision should take
account of the extent of the differences and the
The identification of seasonal items in the needs of users.
basket and the corresponding length of the
seasonal cycle Lack of well‐defined seasons
9.5.43 In practice, seasonal commodities do not
9.5.41 The most important characteristics which always appear and disappear in the same month
can be associated with a strongly seasonal product every year. This presents a problem both for year‐
are: that it is not available for purchase during on‐year monthly comparisons of monthly indices as
certain months (or the quantity available for well as for year‐on‐year comparisons of quarterly
purchase is negligible): that there is a significant indices, but is particularly pronounced for the
variation in demand during the course of the year; former. The consequence of neglecting the
that there are corresponding large fluctuations in importance of carefully pre‐defined seasons can
price. The variations in supply and demand, means contribute to a method, particularly the variable
that prices cannot be or are not observed during weight approach, failing to correctly reflect the price
these months. Seasonal products are associated with changes in the price index in the short‐term.
variations both in availability of products and in their Decisions made regarding the relevance of prices
demand and this can lead to pronounced variations which could have been observed but were not (and
in stocks in outlets, expenditure levels and prices. have not been included in the index or the
Any period of non‐availability for pricing in outlets imputation of non‐existent prices can affect the
should have some typical annual cyclical pattern. If stability of the price index. Thus the fact that
for some special reason a product becomes seasonal patterns change and the need to confront
unavailable in a month where it would normally be whether to accept this or force data into a pre‐
available and sold at a normal price, this does not determined seasonal pattern poses a dilemma to
make it a seasonal product. The out‐of‐season compilers of the CPI. It is particularly important that
period of a product includes the months in which no when data are published, the statistician should
137
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
provide background explanations for the 12‐month • Strongly seasonal products enter the market at
rate of change, and in particular whether this is high prices that diminish towards the end of the
influenced by a change in the timing of sales and in season but, as already argued, can go up sharply
what direction and by what magnitude. before exiting the market. This will also have an
impact which may vary depending on the
Home produced products being replaced calculation approach used.
with imported products when out‐of‐season
9.5.44 The product description is important in Some common seasonal products
these circumstances. The price statistician will need and their treatment
to make a judgement on whether the imported
apple is of the same quality as the home grown one
and if so whether it can be treated as an
Fruits and vegetables
“equivalent” fruit. This judgement should be based 9.5.46 The prices of seasonal fruits and
not just on variety of apple, its general condition and vegetables are strongly dependent on supply
flavour but also on whether the market perceives it conditions, such as weather, and are thus liable to
to be the same and treats it as the same. For show extreme price movements and shifts in
instance, the imported apple may be so highly priced seasons as well as seasonal unavailability. However,
that only a small number of well‐off consumers will changes in specification or quality are not common
buy it. It may then be regarded as being and therefore do not cause the same problems as
unrepresentative and not equivalent to the domestic are experienced when dealing with, for example,
apple, and excluded from the sample. The decision seasonal clothing (see next section).
to exclude the imported apple may be made even if
the apple is of the same variety as the home grown 9.5.47 There is no consensus on whether the
one. It is important, of course, that the price fixed or variable weights approach is the most
collector is properly trained to identify these product appropriate.
changes. Clothing
9.5.45 In order to correctly define the months in 9.5.48 The treatment of seasonal clothing can be
which the price index of the seasonal commodity will far more problematic, especially if fashion is
be calculated from observed prices, particularly in involved. In many countries the price patterns of
the situation where the methodology requires a clothing are the result not just of seasonal
fixed seasonal pattern, the statistical office should availability (for example, winter coats, bathing suits)
examine in depth the historical data on quantities but, especially for fashion goods, of discounts in
and prices for each commodity and undertake a seasonal sales (Chapter 6 provides specific guidance
sensitivity analysis to identify working rules for on the latter). In addition, the problem of extreme
determining the inclusion and exclusion of products price movements is combined not only with seasonal
in the index in specific months: unavailability but by annual changes in the products
• The quantity data are needed in order to decide themselves, in other words, the effect of fashion.
whether it is likely that in any specific month the New stocks of clothes that arrive in the shops at the
product will keep returning to the market every beginning of their season may have styles that are
year. Unfortunately most Household Budget different from those that disappeared at the end of
Surveys record expenditures rather than the previous season, so there is the question of
quantities but other data sources might be at whether the new styles are different in quality from
hand relating to retail sales. the old styles. In addition, prices generally fall over
the course of the season: prices at the end of a
• The prices data will show how inclusion of the season are often discounted as shops clear their
product in the index for the first month in which stocks in readiness for the next season. Of course,
the product returns to the market may for seasonal items, comparisons must be made
contribute to the volatility of the index. For between the new replacement products and the old
example, if the entry price is particularly high products that disappeared at the end of the previous
(say, more than 250% higher than the season, maybe six months earlier. Dealing with these
commodity's last actual price), then it would be issues can be challenging:
reasonable to consider not including the
product for that month, especially as it is • Specification changes. Since it can be difficult to
improbable that the product was widely make like‐to‐like observations for many
purchased at such a high price. categories of clothing, the rules for judging
whether a replacement is comparable to an
138
SPECIAL CASES ‐ SEASONAL PRODUCTS
original item may need to be somewhat relaxed. that was priced during the previous season and
For example, this season’s high‐fashion coat can compare its price directly to the final imputed
be viewed as comparable to last season’s model price of the old variety. It is important that the
unless there are obvious differences in index reflects the full extent of the price
important characteristics such as textile difference between the last month in which a
replacing leather. For high‐fashion items, only price was imputed and the first month of the
changes in compositional and material new season. If the replacement has a different
characteristics, if significant, should be treated level of quality, then the price should be
as quality changes – see Chapter 7. The primary adjusted using one of the quality adjustment
characteristic of most high‐fashion items is the methods presented in Chapter 7.
product brand, which incorporates the fashion
• Continue price collection using this new variety
element. Keeping the same or an equivalent
for the new season.
brand is often the key to measuring the fashion
element. (See CPI Manual paragraphs 10.66– 9.5.50 The reader should refer to Chapter 6 for
10.72.) further guidance on the treatment of “sale” prices.
• Discounted end‐of‐season prices. In the case of a
product showing temporary discounts or Intractable problems and serious
promotions, where it seems likely that the price challenges?
will return to its normal level after a short
period, then the discounted price should be 9.5.51 In the context of constructing a month‐to‐
included and no special method is needed to month index that accurately reflects price inflation
handle it. If, however, the discounting is facing households, there is no fully effective method
seasonal in nature and intended to help the of dealing with large monthly fluctuations in prices
shop clear stocks of old styles, then the issue is and quantities generated by strong seasonality. The
more problematic because, unless specific fixed and variable weights approaches each have
procedures are applied to ensure a return to merits as well as defects. If user focus is on month‐
normal prices, the index will be subject to a to‐month change then it has been argued that the
systematic downward bias. The exclusion of variable weights approach is probably the preferred
such situations is an important consideration in method as it avoids the problem associated with the
defining the in‐season period. Products that fixed weights approach where the annual weights
have flaws or are shop‐soiled should not be attached to each month‐to‐month price relative can
included. be misleading. But the fixed basket has been
9.5.49 As a result of the above complications, compromised unless year‐on‐year comparisons are
some statistical offices employ the following made. From a presentational point of view, indices
procedures during the period covering the time showing the twelve‐month inflation rate assist in
when a “seasonal” clothing item is not available. This trend analysis but do not easily identify changes in
adopts the fixed weight approach but imputes the monthly trends. It has already been noted that
missing prices by applying price movements (using month‐to‐month indices using maximum overlap are
the overall mean or the class mean – see Chapter 7) prone to downward bias.
to the last available normal price: 9.5.52 Indices can be seasonally adjusted using
• In the first month that the item is unavailable, seasonal adjustment factors or rolling year indices or
record the price at its “normal” level; in other by traditional approaches such as the use of three
words, in the first month impute the last month moving averages. The latter are covered in
available normal price. Chapter 11 on Special Indices.
• Impute a price during the period of 9.5.53 It is recommended that statistical offices
unavailability, for the second and subsequent undertake pilot studies with historical data before
months, by applying the monthly movements making a decision on which approach to use. Finally
for clothing items for which prices are available it should be noted that when estimating the
to the last normal price. contribution of a sub‐component of the CPI to the
change in the All Items CPI, the way this is calculated
• When the normal season resumes, select a is different for a seasonal component. This is dealt
replacement as similar as possible to the variety with in Chapter 14.
139
CHAPTER 9.6
Special Cases ‐ Second‐Hand Goods
Introduction Furthermore, putting the national accounts
argument to one side, the use of gross expenditure
9.6.1 The purchase of second‐hand goods can be weights and the inclusion of disposals also would be
important for some categories of household. inconsistent with an index based on acquisitions and
could clearly overstate, potentially by a substantial
9.6.2 Paragraph 31 of the 2003 ILO Resolution amount, the resources that are devoted by
on consumer price indices, states that “the households to acquiring second‐hand goods. It is
expenditure weights for second‐hand goods should therefore not appropriate for either a cost‐of‐living‐
be based either on the net expenditure of the index (COLI) or a non‐COLI (for example, a cost‐of‐
reference population on such goods, or on the gross goods index (COGI)37.
expenditure, depending on the purpose of the
index”. A CPI is generally understood to be a price 9.6.4 In nearly all cases net weights should be
index that measures the changes in the prices of used38.
consumption goods and services acquired and 9.6.5 The CPI Manual elaborates four different
should normally use weights consistent with this scenarios on the issue of weights, all of which are
concept. The Consumer Price Index Manual (2004) based on the “net expenditure” concept:
provides more detailed guidance on the appropriate
weights in the CPI. Paragraphs 3.127 to 3.129 1. Directly from another household. The net
reiterate that second‐hand goods are in scope and expenditure is zero as the transaction is
introduce the “net expenditure” concept, in other between households. It follows that these
words total purchases less sales, for weights. The purchases should be excluded from a CPI.
latter is in line with the national accounts concept of 2. From another household via a dealer. This is
“household final consumption expenditure” as where dealers purchase second‐hand goods
defined in the System of National Accounts (SNA) from individual households and then re‐sell
1993. them usually after carrying out renovation,
9.6.3 The use of gross expenditures as weights cleaning or maintenance. Theoretically these
for changes in the prices of second‐hand goods in a purchases should be included with a “net”
CPI is inconsistent with the SNA35. The SNA states weight reflecting the difference between the
that sales of second‐hand goods (i.e. partly used buying and the selling price which is deemed to
durables) have to be accounted for and are treated
as negative expenditure in the SNA to be consistent
with the treatment of the original purchases36. purchases an ex‐rental car then the whole value of the car is
recorded as HFCE (and negative capital. The various scenarios are
35 described in more detail in paragraph 9.6.5.
The SNA does not directly treat household durables as
household assets or their acquisition as capital formation. The one 37
A cost‐of‐goods index measures the percentage change in
exception is dwellings (see footnote 2). However, through a
expenditure a household requires to purchase a fixed basket of
recognition that household durables are goods that can be used
goods and services. As its name implies it seeks to measure the
repeatedly or continuously over time, the SNA does indirectly
change in cost. In contrast, a cost‐of‐living index measures the
recognise them as a household asset, despite their purchase being
percentage change in expenditure needed to maintain a
treated as consumption. So by implication durables are subject to
household’s standard of living. As its name implies it s objective is
the SNA guidelines relating to expenditures on gross fixed capital
to measure the change in the cost –of‐‐living.
formation. These guidelines explicitly define the latter as 38
There is one important exception that occurs when the user
acquisitions less disposals of fixed assets.
cost approach is used for the measurement of owner‐occupier
housing costs. The loan costs associated with house purchase
36
More particularly, the SNA recommends that sales of second‐ enter the index. Under an acquisition approach owner‐occupied
hand goods are to be recorded as an expenditure within the housing would be based on a net acquisition cost basis i.e. the
purchasing institutional sector and a negative expenditure on the cost of purchasing all newly built houses or converted dwellings or
part of the selling institutional sector. So, any sale/purchase existing dwellings newly acquired by the household sector (for
between households has no impact on HFCE, unless an example, private purchases of houses previously owned by the
intermediary such as a car dealer is involved, in which case the government) less disposals of houses (such as demolitions or sales
dealer’s margin adds to HFCE. On the other hand, if a household of private houses to government).
140
SPECIAL CASES ‐ SECOND‐HAND GOODS
represent the “service” the dealer is giving the justification for including acquisitions but arbitrarily
buyer. excluding disposals. Such a procedure would be
illogical and inconsistent with the SNA as a whole.
3. Directly from another sector, i.e. from another
Thus, putting aside some highly specialised uses, in
enterprise or from abroad. The appropriate
general there does not appear to be a circumstance
“net” weight is household purchases from these
in which use of gross expenditures can be justified in
other sectors less any sales to them.
a mainstream price index for household
4. From an enterprise or from abroad via a dealer. consumption, especially one intended to be used to
Following the same principles as applied above, adjust rates of compensation for changes in the cost
the appropriate “net” weight consists of of living.
household purchases from dealers less
9.6.8 From the above discussion, guidelines for
household sales to dealers plus the aggregate
the measurement of second‐hand goods in a price
value of dealers’ margins on the products that
index can be derived whose practical implications
they buy from and resell to households39.
are most pronounced for durable second‐hand
9.6.6 One consequence of using net weights, goods such as houses where the treatment is
with sales and purchases being treated in the same dependent on whether a use, payments or
way, is that it is necessary to reverse the sign of the acquisition approach is adopted (for further details
price changes for second‐hand goods sold by see footnote 1 and paragraphs 10.4 to 10.50 of the
households for the CPI to properly reflect changes in CPI Manual and chapter 4 of this Handbook). The
the cost of living with the consequence that price measurement of housing costs is considered
changes for second‐hand goods sold directly from elsewhere in this chapter. Where the second‐hand
one household to another will carry a zero weight. good is a semi‐durable good, such as for second‐
Thus for both a COLI and a COGI the text in the hand clothes and cars, its treatment is not practically
Manual can be simplified and “household” affected by the whether the use, payments or
transactions in second‐hand goods can be divided acquisition approach is adopted.
into three groups:
9.6.9 Some goods such as vehicles may be sold
1. Transactions between households. The net by households to dealers who subsequently resell
expenditures are zero. The changes in the them at a higher price to other households. This
prices of the goods concerned carry no weight implies that households’ net expenditures on goods
and have no effect on a CPI. that are purchased indirectly from other households
via a dealer should be positive. It can be argued that
2. Purchases by households from units in other
the net expenditures should be treated as purchases
sectors including the rest of the world. The price
of intermediate services provided by the dealers and
changes are weighted by the values of the
not as purchases of second‐hand goods. The weight
purchases or expenditures incurred.
carried in the overall CPI is the same whichever
3. Sales by households to units in other sectors interpretation is adopted although the estimated
including the rest of the world. The price price changes might be different.
changes are given negative weights equal to the
9.6.10 Weights are generally derived from
values of the receipts from the sales.
Household Budget Surveys, which may or may not
9.6.7 In what circumstances can the use of gross cover second‐hand goods or may not have
expenditure weights be justified? As already stated, information to separate the weights of second‐hand
one consequence of using gross expenditure for items from new items. Explicit weighting is
weights would be that the weights carried by recommended even when based on approximate
second‐hand goods in the index would be greater estimates of expenditure. Administrative records of,
than if net expenditures were to be used and too for example, the value of imported second‐hand cars
large compared with other goods and services as it and of changes in second‐hand car registrations may
would overstate in relative terms the amount of provide an alternative source of information to
household resources taken up by transactions in compute approximate weights. Weights can be
second‐hand goods. If the price of a second‐hand estimated “top‐down” by taking total sales and
good increased, the index would reflect the making an approximate apportionment between old
purchasing household being worse off but not the and new goods or “bottom‐up” by computing
selling household being better off. Similarly, from a separate estimates and totalling them.
national accounts perspective, there is no
9.6.11 There are several practical difficulties in
39 pricing second‐hand goods when using the
This covers both imports and domestic sales of former company
cars. traditional approach of sending price collectors to
141
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
a) Items • Periodically conducting a full enumeration of the
relevant outlets selling second‐hand goods at a
9.6.12 A number of factors need to be taken into particular location and then selecting one at
account when choosing a representative sample of a random for price collection each month. The
specific second‐hand good: difficulty with this approach is that it is not only
• The items chosen must be generally available resource intensive but also may not be
and easy to find by price collectors. particularly efficient in circumstances where it is
unlikely that the item priced in a particular
• The item description must be sufficiently outlet at the previous price collection will still be
detailed for the item to be easily identified both available the following month as it will have
initially and on repeat visits to outlets and to been sold. (Second‐hand goods are by definition
provide sufficient guidance when a replacement unique). In addition it is not a practical option
needs to be chosen. when outlets are moveable, which can often be
the case for second‐hand semi‐durable goods
• The item must be representative of the second‐
(especially unbranded clothing).
hand market.
9.6.13 For most second‐hand goods the selection • A form of quota sampling where the price
of precisely which items to be priced is likely to be collector visits a pre‐selected location, for
purposive. To facilitate this, the price statistician will instance a market which is known to sell goods
need to identify not only which characteristics in a mid‐price mid‐quality range, taking a
physically differentiate one item from another but “random” walk around the stalls until a second‐
also which factors have a significant impact on price, hand item meeting the required description is
including, in particular, the condition of the good found. This may be the preferred method,
concerned. The latter may need to be deduced by especially under the circumstances described at
observation based on a number of price influencing (a) above and is a departure from the
attributes. For instance, with clothing there may be conventional CPI practice where the same outlet
noticeable thinning of the material. The attributes is visited each month and the same good is
may be determined in part indirectly, for instance in priced. Outlets and other points of sale for
the case of a car by reference to its age. These specific types of second‐hand goods, such as
should make up part of the item description and be clothing, can often be identified by their specific
used when selecting an item to price each month. location on a market day. Choosing the
appropriate day of the week is also important
9.6.14 Specifications should be kept up‐to‐date to because street traders may change their
reflect the second‐hand market so that problems position on different days .
relating to sample depletion and “forced”
replacements are minimised. Forced replacements 9.6.16 In practice a mixture of the two
are to be avoided particularly for cars as explicit approaches described above might be desirable and
forms of quality adjustment are problematic40. possible.
9.6.17 Under both approaches, stratification by
b) Outlets outlet type ‐ for example, shop, market stall and
street trader ‐ is recommended to keep the sample
40
For example, option costing, which is commonly employed for representative.
new cars, cannot be applied to second‐hand cars, and hedonic
regression techniques require a rich data source which is unlikely
to be available. In these circumstances, expert judgment is often
used to inform quality adjustment for differences in technical
specifications, mileage or condition of vehicle.
142
SPECIAL CASES ‐ SECOND‐HAND GOODS
143
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
prices. A copy of the type of questionnaire sent to 9.6.25 If the price trend of the corresponding new
retailers is given at Appendix 9.6.2. good is used as a proxy then the expenditure weight
used in compiling the index must, of course, reflect
Prices of new goods expenditure on both new and second‐hand goods.
9.6.24 The use of new goods as a proxy for old One approach is to include the weight of the second‐
should be kept regularly under review to identify hand good with that of the new good. Another
anything that might challenge the historic price would be to keep the second‐hand item separate
relationship, for instance, where new safety with its own weight and use it with the price index
regulations or environmental laws reduce the value for the new good. The latter is, perhaps, more
of second‐hand cars compared with new ones. transparent.
144
SPECIAL CASES ‐ SECOND‐HAND GOODS
Appendix 9.6.1: Example Price Collection Checklist for Second‐Hand Clothing
Category: Second‐hand Clothing
Product Name: Men’s Branded T‐Shirt
Product Code: 123.321
Preferred Quantity and Unit of Measurement: 1 Unit
Product Description:
o Product Presentation: No Package
o Quantity: 1 Unit
o Fibre Type: Cotton 100%
o Units per package: One
o Type: T‐Shirt
o Brand/label category: Well known. Indicate. Not
brand imitation.
o Sleeve Length: Short sleeve
o Fabric design: Single colour
o Neck style: Round neck
Collector: ...................................................................................................................
Date: ...................................................................................................................
Market: ...................................................................................................................
Product Detail (9 or specify as appropriate) :
Brand:
Quality: Perfect
Minor Wear and Tear(e.g. some fraying at edges)
Major Wear and Tear (e.g. some holes or major repairs)
Fabric design: Small logo
Large printed design
Multi‐coloured fabric (e.g. stripes)
Neck Style: Round
V‐neck
Outlet Type: Licensed outlet
Temporary stall
Street vendor
Other Comments:
145
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 9.6.2: Example Price Collection Letter to Retailer
Bureau of Statistics
P.O Box 1
Date XX/XX/XX
Mr B Graham
Dealer, Second‐Hand Clothes
Market A
CLOTHING PRICE INDEX
Every month the Bureau of Statistics collects clothing prices for the purpose of constructing the above
mentioned Index and for this purpose we would request you to advise us on the following prices as at insert
date.
We would remind you that it is important for you to let us know if the size of the bundles or the quality of the
content has changed from the previous month. Any such changes should be recorded in the remarks column.
ITEM TYPE PREVIOUS CURRENT REMARKS
PRICE PER PRICE PER
BALE BALE
(an
international
logo)
Brand 2 350
(a national
brand)
(an
international
logo)
Brand 2 200
(a national
brand)
.....
146
CHAPTER 10
Index Calculation
Introduction • The calculation of price indices for elementary
aggregates. These so‐called elementary indices
10.1 Chapter 9 of the ILO manual covers the are the lowest level of aggregation where prices
calculation of consumer price indices including the are combined into price indices. It is often also
elementary aggregates, the averaging of these using the lowest level of aggregation for which explicit
the relative values of the elementary aggregates as expenditure weights are available and used.
weights, to obtain higher‐level indices, and the Each elementary aggregate consists of a set of
construction of the consumer price index itself from goods or services, usually from within a COICOP
a weighted average of the higher‐level indices. It class, and are normally chosen to be as
provides a general description of the methods used homogeneous as possible. The elementary
and gives a general insight into the properties of the aggregate can include stratification by region
different index number formulae that can be used. and by shop type (see chapter 5).
This is supplemented by a detailed exposition of • The aggregation of these elementary price
index number theory in Chapter 15 and on indices to higher level indices using relative
elementary indices in Chapter 20. The ILO manual levels of consumer expenditure as weights. At
points to the better statistical properties associated this stage, a Laspeyres‐type index formula is
with the geometric mean whilst acknowledging that typically chosen, thus reflecting the fact that for
one of the few weaknesses of the geometric mean is most price indices only basket reference period
that compared with an arithmetic mean it is much (as opposed to current period) weights are
more responsive to large price drops. available. This explains why the CPI is often
10.2 Paragraphs 39 to 47 of the ILO resolution described as a fixed‐weighted index or a fixed‐
describe the different stages of index construction. basket index.
The resolution advises against the use of the 10.4 Accordingly, and using the arithmetic mean
arithmetic mean of price relatives because this for purely illustrative purposes, the index Pt/0 in
formula often produces biased results. Nevertheless, period t and which uses period 0 as the price
there is some scope for the use of the ratio of reference period, is a Laspeyres‐type or fixed base
arithmetic mean prices particularly where the weight index, which can be expressed algebraically
elementary aggregate is made up of relatively as:
homogeneous items and the opportunity for
substitution is limited. It recommends that the ∑p q t
i
b
i
geometric mean be used particularly where there is Pt /0 = i
(10.1)
a need to reflect substitution in the index or where
there is a large dispersion in prices or price changes.
∑p q
i
0
i
b
i
41
For a CPI there are three kinds of reference periods: the weight
reference period; the price reference period; the index reference
period. See paragraph 9.81 of the CPI manual.
147
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
An algebraically equivalent expression for equation directly concerned with the calculation of weights,
10.1 is: nor in the design of the sample for price collection.
These issues are addressed in chapters 4 and 5
∑ (p / p ) w
t 0 respectively.
i i i
Pt /0 = i (10.2)
∑w i
The compilation of indices for
i elementary aggregates
where w i = p q
0 b
i i
10.10 The price statistician is confronted with three
10.5 Expressed in this way, equation 10.2 issues in constructing the CPI:
represents the weighted average of elementary price
•
( t 0
)
indices pi / pi , in this case ratios of average
Lack of access to the prices for all products in
the marketplace. Instead a sample of prices is
prices i.e. they represent the weighted average of collected (usually purposive i.e. non‐
elementary price indices relating to each individual probabilistic).
item i. The weights are the expenditure shares for
item i in the basket reference period which is • Incomplete expenditure data at the lowest level
expressed at base period prices (i.e. hybrid of detail, so un‐weighted elementary aggregates
expenditure weights). A price relative is the ratio of a are calculated at the most basic level.
product’s price at a given point in time to its price at • Changes in the sample of prices occur within the
another point in time ‐ in this particular case using elementary aggregate as a result of new and
average prices. disappearing varieties.
10.6 Formula 10.1 is not a “genuine” Laspeyres 10.11 Elementary aggregates are defined as the
price index. For the CPI to be a Laspeyres index, the smallest aggregates for which reliable weights based
base period would have to coincide with time 0. But on consumer expenditure data are available. The
current period expenditure data are only available to latter are used to aggregate the price indices for
the index compiler with a lag, sometimes as long as elementary aggregates to obtain higher‐level indices.
two years or more. Consequently, expenditure data Elementary aggregates can also serve as strata for
for some previous period are used, usually drawn 12 the sampling of prices.
or more months in the past and therefore the price
reference and weight reference periods do not 10.12 The essential tasks of the index compiler in
match. compiling the elementary indices are two‐fold:
10.7 The elementary aggregate forms the basic • To organise the large set of monthly price
building block of a CPI and can have an impact on observations in a way that allows the calculation
the overall index through its influence on the of price indices at detailed levels that are
performance of different aggregation formulae, so representative of the entire population of
the choice of elementary aggregate and the choice transactions during a given period.
of formula used in aggregation can be of critical • To calculate those indices in a way that gives the
importance. most accurate results, given that no weights are
10.8 Operational and computational issues are available at the most detailed levels.
also important considerations in the aggregation of 10.13 In practice, elementary aggregates are
elementary price indices to higher level indices. By composed primarily in terms of groups of
the splicing and chaining approach, the weights can homogeneous products but the decision in the short
be changed as frequently as desired and is only or medium term on what should be the lowest level
limited by the availability of weighting information of product definition for an elementary aggregate is
and the availability of operational resources. In the governed mainly on the existence of weights. The
end, the CPI becomes a series of linked indices which reliability of the weights at these disaggregated
are chained together thus ensuring the series’ levels may be variable ‐ they may be based on survey
continuity. data, or they may be estimated from other data
10.9 This current chapter provides guidance both sources such as administrative data. The important
with regards to the calculation of price indices for point to note is that for the purpose of index
elementary aggregates, and the aggregation of these construction, no elementary aggregate should be
elementary price indices into higher level indices all created for which there is no reasonably reliable
the way to the all‐items CPI. The chapter is not overall weight information. This means that some
elementary aggregates may need to cover more than
148
INDEX CALCULATION
one representative item. For instance the price price behaviour. For instance, where hand
statistician may not have weights for a gloves are included in an elementary aggregate,
representative item, such as bananas, let alone it may be that the price trends of leather gloves
stratified by bananas sold in supermarkets versus may unexpectedly diverge from those of
other outlet types in each region. In such cases woollen gloves as a result of divergences in the
where lack of weights prevents the use of a more costs of raw materials.
narrowly‐based elementary aggregate it is important
• The potential for substitution. The choice of
to construct the elementary aggregate to be as
products to include within an elementary
homogeneous as possible. For instance, in the
aggregate should also account for the degree of
northern hemisphere if it were not possible to
substitutability in response to a change in
construct an elementary aggregate for bananas an
relative prices. For example, an elementary
elementary aggregate for fresh tropical fruit would
aggregate defined as fruit may include apples
likely be better than an elementary aggregate for
and pears, if it is known that consumers
fresh fruit in general.
regularly substitute one for the other as a
10.14 In the longer term, the availability of weights reaction to changes in their respective prices. An
for CPI category should not be an exogenous aggregate for clothing would most certainly
constraint on the definition of elementary need to be divided into more narrowly defined
aggregates. The weighting information required to elementary aggregates simply because many of
construct CPI baskets is something that should be its items are not considered close substitutes,
negotiated with the officials responsible for e.g. hats and shirts. Substitutability also
household survey estimates or whatever other influences the choice of the most appropriate
source is used to establish CPI basket shares. index formula to use in the construction of the
index for the elementary aggregate. This issue
10.15 Note also that the choice of products within
will be discussed later in the section about index
the elementary aggregate will depend on the
formulae.
classification system being used. For instance,
assuming that the international classification COICOP 10.17 Clearly, the more narrowly defined the
is adopted, then elementary aggregate products elementary aggregate, the more homogeneous the
should fall within one of the COICOP classes, such as products in the elementary aggregate should be. For
“garments” (COICOP 03.1.2). The decision on the example, an elementary aggregate for women’s
number of elementary aggregates within the outerwear would include such products as blouses,
“garments” classification, will depend on the dresses, skirts, hats, overcoats etc. The latter are
availability of expenditure weights. A typical choice quite different types of product, and may not be
may be whether to define separate elementary considered to be particularly homogeneous. An
aggregates for: men’s outerwear, men’s underwear, elementary aggregate just for women’s dresses is
women’s outerwear, women’s underwear, and likely to be much more homogeneous and is a more
children’s clothing ‐ or at some greater or lesser desirable goal if expenditure weights for such a
detail than this. grouping are available.
Homogeneity Selection of representative items
10.16 What is meant by “homogeneity”? There are 10.18 This is covered in Chapter 5, with additional
three main dimensions from the perspective of a CPI advice on the related issue of price collection
compiler. All three are related and the compiler provided in Chapter 6, and the reader should refer to
should always attempt to maximise the homogeneity these chapters for a fuller discussion on the subject.
in the CPI sample. Suffice it to note here that an elementary aggregate,
such as women’s outerwear, is usually disaggregated
• Purpose. The products should have a broadly
into a number of “representative items”. This step is
similar purpose ‐ using COICOP (which is a
necessary since the level of product definition in a
classification of products by purpose) should
typical elementary aggregate is too broad to serve as
assist in ensuring this.
a detailed specification for price collectors. For
• Prices. Products should be selected in such a instance, a price collector cannot be expected simply
way that the price movements (and to a lesser to collect a price for “women’s outerwear” ‐ some
extent price levels) are likely to be reasonably examples of women’s outerwear must be carefully
similar. This cannot always be achieved in specified. In doing so, a balance is struck between
practice and there is no guarantee that future the cost of collecting, checking and processing the
price behaviour will be the same as previous prices, against the practical difficulties that might be
149
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 10.1 Example of stratification by region and outlet‐type
Region
Outlet‐type 1 2 3 4 All regions
A a b c d
B e f g h
All outlet types
confronted in the field and the accuracy of the of the weight information. For example, cell “a” in
elementary aggregate index. Table 10.1 refers to razor blades in small
independent shops in the North Region. In any
Stratification of elementary aggregates particular price collection period there will be a
10.19 As mentioned in Chapter 5, elementary number of prices collected in order to populate this
aggregates are frequently defined in three elementary aggregate. There may be several towns
dimensions ‐ product, region and outlet‐type. For in the North Region covered in the CPI sample, and
example, there may be an elementary aggregate for in each town there may be several small
razor blades in small independent shops in the independent shops. In each small independent shop,
Northern Region of a country. collectors may be required to monitor the prices of
two different types of razor blade – say, a well
10.20 Table 10.1 shows a simplified example which known universally available brand, such as Gillette,
assumes, for an undefined product within an together with a local brand. In practice there may
elementary aggregate, stratification by region and be, say, as many as 50 or more prices collected for
outlet‐type. elementary aggregate “a” each month (or as little as
10.21 In this example there are 4 regions: North, 3) but for illustration purposes Table 10.2 assumes
South, East and West. There are also two outlet‐ that there are 14 prices.
types: small independent shops, and other outlet‐ 10.23 To compile an index for universal brand
types ‐ making a total of eight strata, each of which (Gillette) razors in Shop 1 in Town A is
will be defined as an elementary aggregate. Note straightforward – according to Table 10.2 there was
that it may not be possible in some countries to find no change in the index between months 1 and 2,
the data necessary to have such a fine breakdown while the index registered an 8% rise between
for a particular product while in some other months 2 and 3. To calculate an index for razor
countries such a detailed stratification may not be a blades in small independent shops in the North
necessary enhancement to the survey design. The region however, the index compiler is confronted
table must be adapted and reduced according to a with a choice of methods for aggregating the prices
country’s particular circumstances. The letters a to h and thus calculating the elementary price index.
represent the estimated expenditures in each
stratum on the product concerned. In this example 10.24 This chapter looks at the three main methods
there are 4 regions, say North, South, East and West, used by NSIs for calculating the elementary
and 2 outlet‐types, say small independent shops and aggregates. Other methods are discussed in Chapter
other outlet‐types ‐ making a total of 8 strata, each 9 of the Manual. It should be noted that selecting
of which will be defined as an elementary aggregate. the correct formula is critical; otherwise this could
have a significant impact on the overall CPI, thus
undermining its reliability and credibility of the
Calculation formulae for elementary index.
aggregate indices
10.25 The index compiler has three main
aggregation methods to choose from for the
10.22 The method of calculating the indices for the
elementary aggregate is dictated by the availability
150
INDEX CALCULATION
Table 10.2 Product: Razor blades in the North Region
Prices
Product Outlet‐type Month 1 Month 2 Month 3
construction of the elementary aggregate price prices or price ratios, thus generating an index
indices of a consumer price index. rise of 0.80% in both cases.
• The ratio of the arithmetic mean of prices (or 10.26 In many countries, prices for at least some
Dutot index). This is often referred to as the products are collected more than once a month.
ratio of averages (RA). Referring to Table 10.2, Petrol is a good example where the prices are often
the compiler takes the arithmetic mean of all quite volatile and its weight relatively large. It is
the prices in month 1 (17.93) and compares it inappropriate to use these individual prices in the
with the arithmetic mean of all prices in month calculation of the price index for the elementary
2 (18.00). The ratio of these two means is 1.004 aggregate as this will imbalance the sample of price
(i.e. an index increase of 0.40%). quotations. Instead the prices should be averaged
before use in compiling the elementary price index.
• The arithmetic mean of price relatives (or Carli
For a monthly CPI, it is usually sufficient to take a
index). This is often referred to as the average of
simple arithmetic average of the price quotes for
relatives (AR). The compiler calculates each of
each specific item in each specific shop to obtain a
the 14 price ratios (or price relatives), from
representative (monthly average) price for use in
month 1 to month 2, and takes the arithmetic
compiling the elementary price index. This strategy is
mean of all the ratios. Again using the price data
also more transparent for the users. Using a
given in Table 10.2, this gives an index rise of
geometric average of the intra‐month price
0.86% ‐ twice the estimate obtained with the
observations should best be avoided however. It can
average of relatives.
be shown that when calculating the mean of two
• The ratio of the geometric mean of prices (GM) extreme values, say 0.001 and 0.990, that the
(or Jevons index). The compiler carries out arithmetic mean (0.49505) provides a more credible
similar calculations to the ratio of averages and result for the representative price than that obtained
the average of relatives but using geometric with the geometric mean (0.004987).42 Note that
rather than arithmetic means. This is done by
taking the 14th root of the product of all 14 42
This follows from Schlömilch inequality.
151
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
using the arithmetic mean formula for averaging the discounted price then replication will bias the index
intra‐monthly price observations, should not affect downwards.43
the compilers decision as to the formula to use for
10.29 The alternative and technically preferred
constructing the elementary price indices, which is
option to replication is to select outlets in proportion
separate issue (i.e., Jevons, Dutot, or Carli – See the
to their estimated size (turnover). Thus, in the above
Choice of Formula later in this chapter).
example, for each supermarket as large as Town A
Implicit weights within an elementary Shop 1, the sample would include three of the
aggregate – replication smaller types of supermarket. However, this may not
be practicable, and the additional price collection
10.27 Although the lack of explicit weights below costs may not be justified. Also the same judgments
the elementary aggregate level means that the about relative size are involved. But this overcomes
expenditures in Table 10.1 cannot be further the potential bias associated with replication of an
subdivided, implicit weights within an elementary unrepresentative price quote and also has the
aggregate, through the replication of price quotes, advantage that the effective replication factor can
provides a further way of improving the accuracy of vary according to product.
the elementary aggregate indices. For instance, it
may be known, from simple observation, that Town Choice of formula
A Shop 1 is much larger and has a higher sales 10.30 As already indicated there are three main
volume than Town A Shops 2 and 3. Therefore, it aggregation methods for the construction of the
would seem logical to give more weight to the price elementary aggregates of a consumer price index.
changes in Shop 1 than to those in Shop 2, rather
than the equal weighting given by the three • The ratio of the arithmetic mean of prices (or
aggregation methods shown above. This can be done Dutot index). This is often referred to as the
through the method known as “replication”. ratio of averages (RA).
Effectively, the method gives a larger weight to Shop • The arithmetic mean of price relatives (or Carli
1 than the other shops in Town A. The allocation of a index). This is often referred to as the average of
weight by replication can only be done by informed relatives (AR).
guesswork, because no data is available on the sales
of razor blades in different outlets. For example, it • The ratio of the geometric mean of prices (GM)
might be observed that Shop 1 is about 3 times (or Jevons index);
larger than the others on the basis of numbers of 10.31 These aggregations can be expressed
staff employed or on total shop area. In such a case algebraically as follows:
the prices recorded in Shop A could be replicated
three times. Thus, the first two rows in Table 2
would be each repeated twice. This replication does
∑p t
i /n
RA: I
t /0
= i
(10.3)
not need to be done explicitly. The formula used to
calculate the means can easily be adapted to include
∑pi
0
i /n
the prices in the first two rows three times instead of
just once as for the others. ∑p t
i / pi0
10.28 In this example, the results for the three AR: I
t /0
= i
(10.4)
n
methods described above would be 0.31%, 0.67%
and 0.62% (compared with 0.40%, 0.86% and 0.80% 1/n
⎛ ⎞
respectively). The replication makes a significant 1/ n ⎜⎜ ∏ pit ⎟⎟
⎛ p ⎞
t
⎝ i ⎠ (10.5)
GM: I = ⎜⎜ ∏ i0 ⎟⎟ =
difference in this example, as would be expected. t /0
But such an approach should be exercised with some 1/n
⎝ i pi ⎠ ⎛ 0⎞
caution. The index compiler needs to be aware of ⎜⎜ ∏ pi ⎟⎟
the potential for replication to introduce bias into ⎝ i ⎠
the index if for some reason an unrepresentative 0 0
price is replicated. For instance, if the price of one where prices p1 to pn are obtained in the base
razor blade is recorded in a very large supermarket 1
period and matching prices p1 to pn are obtained
1
and replicated a factor of one hundred, while this
for the same commodities in a subsequent month t.
particular razor happens to be unrepresentative and
because of low sales volume it is offered at a
43
The calculation system could be designed to allow for the use of
explicit weights, which would be essentially the same as
replication.
152
INDEX CALCULATION
10.32 The CPI Manual (see paragraphs 9.25 to 9.38) most appropriate if the base price reflects the
includes a detailed discussion of the underlying relative importance of each price quote and the
economic assumptions and properties of these and quantities purchased remain the same, i.e. they
other formulae. Only a short summary of the most are not influenced by subsequent price
salient points is given in the following sub‐sections of movements. Under these conditions, the
this chapter. Laspeyres price index remains the “appropriate”
economic price index and RA can be justified in
The economic theory and market a fixed base situation provided that equal
assumptions behind each formula quantities are purchased in the base period.
10.33 In theory a price index can be compiled for • The GM assumes equal expenditure weights
each individual product by computing the price that are constant over time. This is true both in
relative: a fixed base or chained context.
153
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Some axiomatic properties the direct index between the same two periods,
indicating the potential for in‐built bias.
10.37 The axiomatic approach to index numbers
attempts to determine the most appropriate formula • The Dutot formula, or RA, does not suffer from
to apply by reference to specific axioms or tests that the same defects as Carli. But it has other
the index ought to ideally satisfy. Paragraphs 1.53 to disadvantages. If the elementary aggregate
1.84 of the ILO Manual discuss these in some detail includes products which have widely differing
and this is supplemented by further material in price levels, the products which are more
Chapter 16. Three of the more critical axioms or expensive (in the chosen units of measurement)
tests – the “time reversal” test, the will get a larger implicit weight, and hence have
“commensurability” test and the “transitivity” test – a greater influence on the elementary aggregate
are considered here together with the issue of zero index. A further complication is that the Dutot
prices. formula is accurate only if all the constituent
products are measured in the same units (i.e. it
• The Carli formula, or AR, fails the “time reversal” fails the so‐called commensurability test). This is
test. If prices in period 2 revert to the initial an important result, and the table below
prices in period 0, then the product of the price illustrates the effect, using an imaginary
change going from period 0 to 1, times the elementary aggregate called “seasonings”,
change going from period 1 to 2, should be unity comprising two representative items, salt and
‐ i.e. the index should end up where it started. pepper. In most countries pepper is much more
This does not happen with the Carli index: in expensive than salt if measured by weight,
fact, unless the prices in the two periods are though in practice much less pepper is
exactly proportional, the product of the two consumed than salt. Table 10.3 shows an
Carli indices will end up being greater than one, extreme example of both of the above defects.
with the result that there is potential for upward
bias when the Carli formula is used. The Carli • Being multiplicative, the Jevons formula cannot
also does not satisfy the “transitivity” test – the use a price of zero. This is not always listed as an
chain index between two periods does not equal axiom but it is nonetheless important to bear in
Table 10.3 Elementary Aggregate: “Seasonings”
MONTH
1 2 Month2/Month1
A) DIFFERENT UNITS Price Price Price ratio
Pepper (10g) 2.70 3.80 1.407
Salt (kg) 2.50 2.70 1.080
Arithmetic mean prices 2.60 3.25 1.250 DUTOT
Geometric mean prices 2.60 3.20 1.233 JEVONS
Arithmetic mean of ratios 1.244 CARLI
Geometric mean of ratios 1.233 JEVONS
B) SAME UNITS
Pepper (kg) 2700 3800 1.407
Salt (kg) 2.50 2.70 1.080
Arithmetic mean prices 1351.25 1901.35 1.407 DUTOT
Geometric mean prices 82.16 101.29 1.233 JEVONS
Arithmetic mean of ratios 1.244 CARLI
Geometric mean of ratios 1.233 JEVONS
154
INDEX CALCULATION
mind. Zero prices may happen if the price of a to increase with the dispersion of the price
product (for example. a government service relatives, so it is desirable to construct the
within education, health or transport) changes elementary aggregates to minimise the variation
from zero to a positive price, or vice versa. In in the price movements within each
such cases, a possible solution is to combine the homogeneous elementary aggregate. The less
item concerned with a related item, and first to variation there is in the price movements the
calculate an average price of the two items less difference the choice of index formula will
before applying the Jevons formula. The make numerically. But the index compiler needs
treatment of the incorporation into an index of to avoid the potential for bias due to the item
services previously provided free is covered in definition being too narrow and excluding
Chapter 8. important parts of the market.
10.38 To summarise the above discussion, it follows • As RA gives more weight to the price relatives
that: for the products with high prices in the base
period the construction of elementary
• The Dutot formula should be used only where
aggregates should also avoid relative large price
the products are homogeneous and where
dispersions in the base period. This is because of
consumers have only limited opportunities to
the difficulty in providing an economic rationale
substitute between products or where the
for such implicit weighting. If the products in the
policy is that substitution is not to be reflected
elementary aggregate are homogeneous very
in the CPI.
few quantities will be purchased at high prices if
• The Carli formula is to be avoided because of the same products can be purchased cheaply
the potential for bias, particularly in its chained either at a different retail outlet or under a
form. The formula fails the time reversal and different brand name. An elementary aggregate
transitivity tests and gives more importance to compiled using RA is not invariant to changes in
prices that rise more quickly. the units of measurement of the individual
products. Thus an RA index of energy products
• The Jevons formula avoids the problems would change if the units of measurement for
associated with the Dutot and Carli formulas petrol changed from gallons to litres while
and is generally regarded to be the most leaving the units of measurement for natural gas
preferable method technically for calculating consumption unchanged. Units of measurement
elementary aggregates. It also implicitly for many commodities are basically arbitrary. It
assumes that consumers substitute between should therefore not be used for heterogeneous
products when their relative prices change (a products where such variations can occur,
realistic assumption in many cases), although including, for instance, pack sizes.
with an elasticity of substitution of one it may
overstate the impact. The Jevons formula is • Sample stratification As the elementary
particularly appropriate where there is a need to aggregates also serve as strata for sampling
reflect substitution in the index or where there purposes, minimising the variance in the price
is a large dispersion in price levels or changes. It relatives within the strata will also reduce the
is generally the more favoured method sampling error. This is often achieved by
worldwide for compiling a CPI particularly as it is additional sample stratification, for example, by
arithmetically well‐behaved and stable, not introducing further elementary aggregates by
being prone to influence from differences in the geographical location and outlet type but this
statistical variance in price levels and trends assumes that the corresponding weights, which
between elementary aggregates. might requires a bigger and more expensive
household budget survey, are available.
Implications for construction of elementary
aggregates and sample stratification • Item descriptions. Item specifications may be
generic or very precise. The approach adopted
10.39 The statistical design underlying the will depend on local circumstances and may
construction of the price index and its elementary consist of a mixture of both generic and specific
aggregates should take into account the axiomatic item descriptions but it should be noted that the
properties of the different aggregation formulae. use of generic item descriptions is likely to
The main points can be summarised as follows: increase the variance in prices. Where generic
item descriptions are used for sample selection,
• Elementary aggregates. The numerical
price collectors should record detailed
differences between the various formulae tend
descriptions of the items being priced.
155
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
10.40 Further advice on Sampling is given in formula most if not all statistical agencies use to
Chapter 5. construct their CPI.
The Laspeyres Index and the Laspeyres‐type
The compilation of upper‐level Index
aggregate indices
10.46 In many references in the literature to the CPI
10.41 The previous sections explored the formula, it is often mentioned that it is a Laspeyres
aggregation of prices at the lower‐level to construct price index. This is not most often an accurate
price indices for the elementary aggregates which depiction of the formula. For a CPI to be a “true” or
are then weighted together to compute the all‐items “genuine” Laspeyres price index the weight
CPI, and any intermediate aggregate price indices at reference period must coincide with time b and then
Class, Group, and Division levels, in the CPI scheme. the following weighted aggregative formulation
would result:
10.42 This section covers the compilation of upper‐
level aggregation, otherwise known as macro‐ ∑p qt
i
b
i
indices. The topic is covered in detail in paragraphs PLAt / b = i
(10.7)
9.74 to 9.126 and 9.131 to 9.138 of the CPI Manual. ∑p qb b
i i
This section provides a summary with some i
supporting examples. where:
The choice of index formula i is the number of products in the index basket.
pib is the price of the ith product at period b, the base
10.43 Higher‐level price indices are constructed as
weighted averages of elementary aggregate indices. or reference period.
When compiling the upper‐level indices by pit is the price of the ith product at period t (t > 0).
aggregating the elementary aggregates or the micro‐
qib is the quantity of the ith product at period b, the
indices, the compiler faces similar choices with
regards to which index formula to use as in the case base or reference period.
of micro‐indices. These various formulae differ in PLAt / b is the Laspeyres price index that measures the
terms of the information content of the weighting
change in the aggregate value between the base
system used to compile index values.
period and period t for a given pre‐defined basket,
10.44 In principle, the choice of index formula used qib of goods and services; the basket is re‐valued in
to calculate a CPI should be determined by the
each period in terms of the prices of that period.
“target” index that the statistical agency has set as
its objective. For instance, in theory one of the 10.47 A “true” Laspeyres price index where the base
superlative index number formulas, that passes the period coincides with time b, is rarely a practical
time reversal test, and is an average of the Laspeyres option when what is needed is a timely CPI, which is
and Paasche formulas (e.g. Fisher formula) or a available to users on an on‐going basis. Some of the
fixed‐basket formula whose basket is an average of difficulties associated with applying the “true”
the base and given period baskets (Walsh, Laspeyres formula for the CPI are:
Edgeworth‐Marshall or Törnqvist indices) should be
• If expenditures for some products are
used for the construction of a consumer price
characterised by a seasonal pattern, then the
index45. However, in practice the compiler often
expenditure at time b, a sub‐annual period, may
lacks sufficient and timely knowledge about the
be unrepresentative of the expenditures at
prices and the quantities in both periods of
other periods.
comparisons to produce such indices. Consequently,
statistical agencies will aim for a more modest but • Expenditure data for periods less than a full year
practical basket formula that is less data intensive; are often unreliable.
such a formula nevertheless provides valuable and
timely insight as to the behaviour of prices in the • The reference period for the expenditure data
economy. are not in sync with the chosen basket update
period46.
10.45 Most of what follows focuses on constructing
macro level indices using the class of fixed‐basket
46
Although, this dilemma could also be resolved were the CPI
45
See, for example, ILO Manual paragraphs 15.18 to 15.23 for revisable. A CPI series could be recalculated using the new
averages of the Laspeyres and Paasche formulas, and paragraphs weights from the weight reference period using as the base prices
15.24‐15.32 for the desirable Lowe formulas. those from the same reference period.
156
INDEX CALCULATION
10.48 Price updating of expenditure weights does not ask the household “how many” candles
provides a popular solution to the issue raised in the were purchased during the reference period but
last point (to be discussed in a later sub‐section). rather enquires as to “how much” did it spend on
Once the weights are price‐updated, the resulting candles47. Using value weights as opposed to
price index is more in line with what is commonly quantity weights in the compilation of the CPI is also
recognised as a Lowe index (or Laspeyres‐type price much more convenient because for some products,
index). Most statistical agencies will use this index particularly some services, no tractable quantities,
formula for compiling their CPI. q’s (or unit values) are available. For example,
t /0
property taxes, public transportation, and education
10.49 The Lowe price index formula, ( PLo ) which tuition fees are good examples where the price
compares prices in period t to those in the price statistician cannot derive meaningful unit values or
reference period 0, is written as: quantity weights. Formula 10.9 can be transformed
to account for these issues.
∑p qt
i
b
10.53 An equivalent algebraic transformation of
PLot / 0 = i
(10.8)
∑p q
i
0 b
i
formula 10.9 is:
pit pi0 q ib
P t /b
=∑ 0 (10.10)
where:
∑ pi0 qib
LO
t i pi
p i is the price of product i in period t i
0
p i is the price of the same product in period 0, the pi0qib
10.54 The expression represents, the
∑ pi0qib
price reference period for this index.
157
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
period, b (i.e. the average of the price relatives48). An of the index with equation 10.11 or any other
index using equation 10.11 shows unambiguously formulation for that matter. The p appearing in the
that a composite price index such as the CPI is a formula is not the price associated with an
measure of price change. This formula, which is individually specified item or variety such as those
recognised as the weighted average‐of‐relatives can selected from outlets every month for the CPI.
be computed in much the same way as a simple Instead, these prices would more accurately be
average of relatives but this time it is a weighted described as an estimate of the price of a group or
average. Expressing the price index as a weighted sample of products that make up the elementary
average‐of‐relatives does have other advantages index aggregate. In other words, pi is more
over the aggregative indices, for instance: ∧
precisely defined as a pi , an estimate of the prices
• The price relatives for each individual product or
product group in the aggregate together of sampled products within the stratum of the
constitute a simple product or product group elementary index. These are average prices, based
price index that often provides valuable on a sample associated with the individual product
analytical information. groups at the lowest level of aggregation for which
reliable expenditure weights are available, i.e. the
• When a new commodity is introduced to replace elementary aggregates. In most cases no one single
one that was formerly included, the relative for price will in fact make up an elementary aggregate
the new item may be chain‐linked to the relative except in exceptional cases where an elementary
for the old one, using the existing value weights. aggregate is represented by a single item such as the
The chain‐linked index will retain its existing domestic postal rate established by the national
value weight. postal office for first class mail.
• Values rather than quantities are used in order 10.59 To summarise the discussion so far, the target
to produce weighted relatives that are all in the index is normally the Laspeyres price index, a specific
same monetary units which can be aggregated; form of a Lowe Index. In practice however, because
generally, quantities cannot be directly of the non‐revision policy of most CPIs and the need
employed as weights, although in some cases for timeliness, a chain Lowe (Laspeyres‐type) index
they may be, that is they may be applied in the formula is usually the chosen option by the statistical
calculation of some elementary indices. agencies for calculating their CPI. A Laspeyres‐type
index is easy to explain and understand as it answers
• The basket reference period for an index is
the question how much more or less does it cost to
usually a year, but in extremis may be less.
purchase the same basket of goods and services over
• Different elementary aggregates can be an interval of time (e.g., a month, a year or many
combined, provided they have the same base years). But it is not without it problems and may be
year, to form new special aggregate indices that seen as a practical compromise which departs from
users sometimes request. the ideal.
10.57 Note that weights, w i of the individual Time‐lags and the use of price update hybrid
products that are used in formula 10.11 are defined weights in a Laspeyres‐type Index
as “hybrid” weights that have been price updated
from the basket reference period. (The practice of 10.60 As previously mentioned, the price reference
price updating is discussed in the next section.) and weight reference periods coincide in the
These weights represent the cost of buying the formulation of the true Laspeyres price index.
quantity of commodity i, derived from the basket Formula 10.12 shows the Laspeyres price index from
10.8 transformed to weighted “average of relatives”
qib but re‐valued at prices from
reference period, equivalent.
0
some chosen base period, pi .
pit pib q ib
PLAt / b = ∑ (10.12)
10.58 An issue worthy of special mention is with
regards to the prices that are used in the compilation
i pib ∑ pib qib
i
158
INDEX CALCULATION
value (as opposed to hybrid value) of a commodity in 0 b
Where pi qi is the hybrid value weight
the basket reference period b. Recall, that these
(expenditure), and
observed values are derived from Household Budget
Surveys (HBS) or/and the National Accounts, or pi0qib
possibly other sources. is the basket share for the i th commodity
10.62 In practice, the price base period and the
∑ pi0qib
i
basket reference period will diverge because of an in terms of period b expenditures at period 0 prices.
unavoidable lag associated with processing the The sum of these shares must be rescaled so as to
weight data, a necessary exercise regardless of their equal 1 (or 100) at the new time base which is often,
source, e.g., HBS or National Accounts. To ensure but not necessarily, chosen to coincide with the new
that a commodity’s weight and its price share a price reference period, 0.
common period, the weight is price updated (or
10.66 Note that equation 10.14 is not a true
adjusted) from its reference period to the price
Laspeyres price index but is instead a Lowe price
reference period.
index because the timing of the price reference
10.63 Weights commonly used in a CPI are not 0
period (typically a month), pi now differs from the
therefore the observed weights from their reference
weight reference period (typically a year).
period but can best be described as hybrid (or price‐
updated) weights. Price updating of weights, an 10.67 Equation 10.14 may conceal the subtle nature
often necessary practice, should however be of the price updating exercise. For this reason, a
exercised with a certain level of care49. simple algebraic example is presented here.
10.64 Multiplying the reference period expenditure 10.68 Consider a situation where the weight
b b reference period relates to the year 2001 and the
for that commodity pi qi by its corresponding
objective is to calculate the CPI for December 2002
elementary index for period b to period 0, will yield
(the link period) to April 2007. The index would then
that commodity’s price‐updated hybrid weight.
be calculated as follows:
w = (pib q ib )×
pi0 p April 07q 01
= pi0 q ib (10.13) April 07 Dec 02
pib P =∑ i i
LO i pDec 02q 01
i i
Where: ⎡ ⎛ 01 01 ⎞ Dec 02 01 ⎤
⎢ ⎜ ⎟ April 07
i ⎥ pi
p q p p
= ∑⎢ ⎝
i i ⎠ i
pib q ib : The observed expenditure value for a given ⎥
i ⎢∑⎜ p ⎛ 01 01
q ⎟p ⎞ Dec 02 Dec 02
p 01 ⎥ pi
product at the basket reference period. ⎢⎣ i ⎝ i i ⎠ i i ⎥⎦
April 07
pi0 / pib : The price index for the product that ⎡ 01 ⎤ pi
= ∑ ⎢w Dec 02 ⎥ Dec 02
i ⎣
i
compares its price in the base period, b to its price ⎦ p
i
during the basket reference period.
Where:
w : The hybrid value weight (expenditure) for the
product or product group (i.e. the elementary piDec 02 / pi01 is the price index for product i that
aggregate), which is used for weighting the price compares its price in December 2002 against its price
relatives. It represents the hypothetical cost of level in 2001.
b
buying the quantity qi of a single product or piApril 07 / piDec 02 is the price index for product i that
product group n drawn from the basket reference compares its price in June 2007 with its
0
period b at its base year price, pi . corresponding price in December 2002.
w iDec 02(01) is the reference period weight from 2001
10.65 Given the set of price‐updated weights, the
formula for calculating the CPI from period 0 to (the 12‐month period) that is price updated to
period t can now be written as follows: December 2002 and will be applied to the price
relative of product i.
pit pi0 q ib
PLOt / 0 = ∑ (10.14) 10.69 The price index pi
Dec 02 01
/ pi represents the
i pi0 ∑ pi0qib
i
price of a product (or commodity) in December 2002
compared to its average price in 2001. Where price
relatives are calculated as month‐to‐month ratios in
the course of CPI production, some algebraic
49
See Chapter 4.
159
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
160
INDEX CALCULATION
10.79 Note that the long term Lowe index going period, i.e. a new basket can be introduced at
from month 0 to t – 1 can be written as a sum of any time during the year. This can be an
t −1
terms of the form pi q i /
b
∑p qi
0
i
b
i . The Lowe attractive feature where production difficulties
may have led to an unplanned delay in the
index going from month 0 to month t can be written introduction of the new basket. But introducing
as the sum of the short term elementary indices a new basket during a period of abnormal price
t t −1 movements and levels, such as during sales
going from period t – 1 to t, pi / pi , multiplied by
should be avoided.
p q / ∑ p q . The important feature of this
t −1 b
i i
0
i
b
i 10.81 That said, many statistical offices have no
i
approach to calculating the index for period t is that problems with adopting annual chaining.
this index can be readily calculated by simply
updating the previous month’s index by the short Linking and Chaining
term elementary indices.
10.82 It is generally agreed that weights should be
10.80 Calculating the index using formula 10.17 has
reviewed and eventually revised regardless of the
two significant advantages over a formula similar to
conceptual basis of the CPI. This is to ensure that the
10.16, from an operational point of view.
basket is as representative as possible of the
• The short term relative procedure offers more consumption patterns of consumers to mitigate the
latitude to the compiler for dealing with new incidence of substitution bias, which is inherently
and disappearing products, missing prices, and present in a CPI, especially with a fixed‐basket index
quality adjustments. In other words, the sample where the q‐values are constant over a certain
of products within an elementary aggregate can number of years.
be different in period t+1 to t from those in the
10.83 Chain linking involves constructing a
interval 0 to t. Items can be replaced or
continuous price series by multiplying together price
introduced into the index by linking at period t –
indices that have been constructed under different
1 thus avoiding the need to estimate a price at
baskets or weighting regimes. Unlike an unchained
period 0. For example, suppose that the in
index, a chain‐linked index spanning more than one
month t+1 a certain model car is subject to a
period or chain cannot be represented either as the
quality change over the previous month’s (t)
ratio of the price of a basket in the current period to
model. With formula 10.17, the compiler does
that in the base period or as the weighted average of
not adjust the current period’s price of the car
price relatives. This is because the weights are not
but the previous period’s price. If the current
constant and even the list of items in the basket is
price were targeted for adjustment, then all
variable. This sub‐section will present a brief
subsequent prices would also have to carry the
overview of the chaining and linking exercise.
adjustment to ensure that the index shows no
movement during the later periods as a result of 10.84 Whilst it is necessary to chain a CPI when the
this specific quality change. This is a tedious weights change, it is possible to chain an index more
exercise that can easily be avoided by simply frequently, every month, say. For the ratio of
adjusting the price at month t – 1. averages, provided that the weights and item list
remain fixed this would yield the same results.
When the direct index formula with a long term
However, for the average of relatives the result
relative (10.16) is used, the exercise is even
would usually be that the index would grow more
more operationally awkward. The complier
rapidly than it should because of a phenomenon
0
would have to adjust the base period price, pi known as “chain drift”, which is caused by price
51
in order to record any quality change. “bouncing” .
Consequently a different reference price would
10.85 The general procedure behind chain linking
have to be registered, in order to avoid having
can be illustrated with the following relationship,
to change the index values that were computed
where a series of bilateral price indices are chained
in the interim. For example, 10 adjusted prices
together:
occurring in the future would require 10
different base prices, a situation that is best P t / 0 = P t / 0 × P 2 / 1 × ... × P t −1 / t −2 × P t (10.18)
avoided.
10.86 To further expand and to illustrate the
• Another but less well known advantage, relating simplicity behind the concept of linking by way of an
to monthly chaining (as opposed to annual), is
that any month can potentially be used as a link
51
See CPI Manual, paragraphs 20.60 to 20.61.
161
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 10.4 Splicing and chaining when a new index is introduced with a revised set of weights
Index A Index B
p 00
2000: P
00 00
=∑ wi = 1.0
i p 00
p 01
2001: P
01 00
=∑ wi = 1.2
i p 00
p 02
2002: P
02 00
=∑ wi = 1.5
i p 00
p 03 p 03
2003: P
03 00
=∑ wi = 2.0 2003: P
03 03
=∑ k i = 1.0
i p 00 i p 03
p 04
x1 2004: P
04 03
=∑ k i = 1.2
i p 03
p 05
x2 2005: P
05 03
=∑ k i = 1.6
i p 03
p 06
x3 2006: P
06 03
=∑ k i = 1.8
i p 03
162
INDEX CALCULATION
Table 10.5 Continuous series with time base 2000
163
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
164
CHAPTER 11
Special Indices
Introduction according to season, such as certain fruits and
vegetables. This topic is dealt with in Chapter 9.
11.1 Chapter 13 of the CPI Manual, paragraphs • The process of seasonal adjustment. This is the
13.24 to 13.31, discusses the compilation of adjustment of a CPI to remove seasonal effects
alternative measure of inflation, including core caused by regular events or influences such as
inflation, and Chapter 22 of the Manual discusses, annual shop sales.
amongst other things, the issue of seasonal
adjustment. The ILO Resolution says very little on the • Smoothing the CPI. The purpose of this is to
subject of special indices apart from advising in eliminate or reduce the effects of price shocks
paragraph 74 that “indices should be presented in or irregular events which affect the CPI.
both seasonally adjusted and unadjusted terms, if 11.5 The second and third processes, which may
seasonally adjusted data are available”. be used separately or together to produce “core” or
“underlying” inflation indices, are the main focus of
Background this chapter.
11.6 However, as well as covering seasonally‐
11.2 The priorities for an NSI are to publish: adjusted and core inflation series, the chapter also
• Long, coherent indices with fixed index covers other special indices, such as indices
reference period. excluding particular products, indices for particular
economic sectors, indices excluding or holding
• 12‐months rate of changes (or 12‐months constant any indirect taxes, and regional indices.
indices).
11.7 These special indices are all related to the
• Month‐on‐month rate of changes (or month‐on‐ extent that they are designed to serve particular
month indices). user needs concerning the measurement of
• Annual average indices/rate of changes consumer price inflation. They can be referred to in
general as “analytical indices”. They decompose or
11.3 But for specialist and analytical purposes it adjust the main CPI in order to shed light on aspects
may be desirable to construct and publish measures of inflation which may not otherwise be apparent.
of “core” or “underlying” inflation which show the
underlying trend in inflation by excluding those
movements which arise from transient factors such
Seasonally‐adjusted consumer price
as seasonality or other short‐term price volatility. indices
Several methods can be used to construct such
measures of “underlying” inflation, some more 11.8 In general, if a time series exhibits
scientific and less discretionary than others. seasonality, as most CPIs do, it is a candidate for
seasonal adjustment. Seasonal adjustment will
11.4 This chapter discusses, among the various
remove recurring seasonal influences from time
types of special index, seasonally adjusted indices
series data. The removal of seasonal and other
and indices which measure core inflation. In this
regular fluctuations from a time series by the
connection it is helpful to distinguish between three
application of seasonal adjustment methods can
distinct processes.
help to provide a clearer picture of the underlying
• Dealing with seasonal products. This is a movement in the data. The presentation of trend
technical problem of how to include products in estimates can further help as the latter can add
a CPI that disappear temporarily from the clarity to the basic picture by the elimination of
market for a period each year, or are always irregular variations which would otherwise obscure
available but at highly fluctuating prices understanding and interpretation. However, few
countries publish a seasonally adjusted version of
their CPI. This raises the question of whether such an
165
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
index should be produced and if not, why not? This be problematic not just for the presentation of 12‐
chapter examines this question but does not month rates of change but also for seasonal adjusted
attempt to describe any detailed technical methods series.
of seasonal adjustment.
11.13 The effect of applying a seasonal adjustment
11.9 It is important at this point to appreciate the to the unadjusted CPI will be to spread out the
difference between the CPI and the inflation rate. effects of such peaks and troughs over the year. The
The CPI is an index number, which shows the level of resulting CPI will not therefore be an accurate record
average prices in a particular month relative to the of the actual price changes taking place each month,
base period which is set to 100. Comparisons of two but will rather show their underlying trend. As
index numbers give a measure of inflation over the discussed below, this is useful for some purposes,
period in between the dates of the two index but it should never be the only version of the CPI
numbers. Thus, a CPI of 145.6 in May 2007 and a that is published. The unadjusted “headline” CPI
figure of 152.7 in May 2008 gives an annual rate of should be published alongside the seasonally‐
inflation of 4.9% ( = (152.7 /145.6 – 1) x 100). adjusted CPI. If the aim of the CPI is to compare the
prices that consumers actually face in two
11.10 CPIs exhibit seasonal patterns caused by, for
consecutive months, regardless of any seasonal
example, the seasonal availability of fresh food such
effects, then the publication of a month‐to‐month
as certain fruits and vegetables and other strongly
unadjusted inflation rate is beyond question. If there
seasonal products. They may also exhibit seasonal
is an apparently erratic movement, the statistical
patterns unrelated to such direct influences.
office should attempt to explain the reason (for
Seasonal variation in demand can also cause
example, “The index for clothing fell last month
seasonal patterns. An example would be ice cream
because of significant reductions in price as a result
sales in summer or warmer clothing during colder
of the end‐of‐season discounts”).
months where demand is related to holidays and to
the weather. Another factor is the pattern of 11.14 The two series meet different needs. The
recurring seasonal sales by retailers in many seasonally‐adjusted CPI will be useful for those
countries. Certain price changes may occur at interested in the underlying or trend rate of
particular times each year, as a result of an annual inflation, while the unadjusted CPI, being a measure
price review for, say, transport fares, electricity of the actual price changes experienced by the
tariffs or university fees. They may also be due to tax consumer, is more appropriate for, say, the deflation
changes which may take effect at a particular time of of monthly sales data or other economic series such
the year. as many components of GDP or, perhaps, for
indexation of state benefits and wages.
11.11 Any of the above events may cause
noticeable peaks and troughs when measuring the
CPI on a monthly basis. The rate of inflation Seasonal adjustment methods
measured on a month‐to‐month basis (m/(m‐1)) may
be misinterpreted if the reason for a particular rise Seasonal adjustment programmes
or fall is not explained, as the volatility of the
unadjusted index may be obscuring the underlying 11.15 The seasonal adjustment of a CPI can be done
trend. On the other hand, the annual rate of either by adjusting the all‐items index (or the
inflation, measured either by the change in the CPI individual group indices) or by adjusting the mean
this month compared with the same month last year prices (or mean price ratios) for those products
(m/(m‐12)) or by the average index for the latest 12 where there are strong seasonal patterns and the
months compared with the average index for the availability of some products at certain times of the
previous 12 months (y/(y‐1)) will normally not be year is very limited. The second of these approaches,
affected by seasonal changes. It is for this reason which has lead to concerns over interpretation due
that for a CPI the question of seasonality is usually to its mixture of actual market prices and adjusted
dealt with by focusing on the 12‐months rate of prices, is discussed in Chapter 9. The first is much
changes. the easier approach because of the ready availability
of standard packages and is normally to be preferred
11.12 In some instances, however, the m/(m‐12)
but can only be undertaken if prices can be collected
index may still include some seasonal effects if the
throughout the year. A standard seasonal
timing of the source of the seasonal fluctuation or
adjustment program, such as X‐12‐ARIMA
the size of the fluctuation happens to vary from year
(developed by the United States Bureau of the
to year. For instance, this will be the case if the
Census), can be applied. However, the use of such
month when planned tax changes take effect is in
packages requires some judgment from the
March one year and in April the next year. This can
166
SPECIAL INDICES
statistician. For example, are the seasonal factors • Uncertainty in the calculation of the seasonal
additive or multiplicative? For how long a period adjustment factors.
should the moving average be calculated? Different
11.19 Noise in the series can be assessed from the
assumptions can give different results. It also
irregular component that is automatically estimated
requires the identification and measurement of non‐
as a by‐product of the calculation of seasonal
systematic movements due to chance or to irregular
factors. Uncertainty in the latter can be investigated
events. It is desirable to eliminate the latter in order
by carrying out an analysis of revisions to the
to identify the general direction of the time‐series,
seasonal factors when they are re‐calculated as
that is, the longer‐term trend, and ‐ just as
additional data become available. But the usefulness
importantly ‐ any changes in the trend. In the case of
of such analysis is limited by the difficulty in
a CPI, the irregular component can be relatively large
distinguishing between revisions occurring as a
due to the influence of, for example, changes in
result of real changes in seasonal patterns and those
excise duty and the dates on which they are applied
resulting from uncertainty in the original calculation.
and also, where applicable, in mortgage interest
payments due to changes in interest rates. This can 11.20 Clearly it is possible to reduce the variance
obscure the seasonal pattern. For this reason it is associated with the calculation of seasonal factors by
technically most appropriate to seasonally adjust the removing specific identifiable effects from the
CPI after it has already had much of the irregular original series. For instance, this can be done by
component taken out. seasonally adjusting the CPI, after taking out the
effect of changes in measurable irregular
11.16 A final point on seasonal adjustment which
components such as excise duty and mortgage
statistical offices need to recognise is that in a
interest payments. The potential difficulties in the
standard X‐12 type of package the final seasonal
calculation of seasonal factors which would have
adjustment factors are not usually available until
arisen as a result of such changes in indirect taxes
there is a time series of at least 24 months and the
and mortgage interest payments are then greatly
factors will be regularly re‐computed. The process of
reduced.
seasonal adjustment will therefore change the
original seasonally‐adjusted series for most if not all 11.21 Perhaps the more important point to note is
of that period. Since the CPI is normally a non‐ that the seasonal adjustment procedure is not
revisable index, it would be difficult to use a entirely reproducible in the sense that different
seasonally adjusted CPI for certain applications such operators would not necessarily come up with the
as indexation of pensions or contracts. same series. This is because it is impossible to have
“final” estimates for the seasonally adjusted series
11.17 To return now to the initial question, should
until 18‐24 months down the road and because the
countries produce a seasonally adjusted version of
seasonal adjustment procedure asks the operator of
their CPI? A seasonally adjusted CPI can be a useful
the adjustment process to choose various options as
macroeconomic policy tool for those analysts and
indicated above. It is for this reason that some
policymakers interested in understanding more
statistical offices which produce a seasonally‐
about the inflationary process. Many statistical
adjusted series label it as an “analytical series”.
software packages today will include a seasonal
adjustment routine as a standard feature. Hence, Moving averages
some of the reasons that were often raised in the
past for not adjusting the CPI for seasonal effects, 11.22 Seasonal adjustment is not, of course, the
such as a shortage of skilled personnel and an only way of smoothing a time series. For example,
absence of computers, are no longer an issue for the price changes resulting from a monthly CPI can
many countries. Nevertheless, the production of a be presented as a 3‐month moving average, or a 12‐
seasonally‐adjusted CPI will not normally be a month moving average. Such presentations are
priority for an NSI and in many cases is left to the discussed in more detail in Chapter 14. Of course,
users to do. most seasonal adjustment procedures are based on
the use of moving averages to form estimates of the
11.18 Perhaps the biggest issues relating to the underlying trend and then seasonal adjustment
usefulness of a seasonally‐adjusted CPI are (a) the factors are calculated relative to the estimated
variance of the seasonal adjusted series and (b) its trend.
vulnerability to revision. These issues can arise as a
result of two factors:
Core inflation and smoothed indices
• “Noise” in the series.
11.23 Many analysts of inflation are interested not
only in the basic CPI, but also in a measure of what is
167
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
called “core inflation”, “underlying inflation” or reduced as a result of technology changes in food
other such terms. Although many countries publish production. An example might be the growing use of
“core inflation” indices, there seems to be no clear poly‐tunnels, which can result in a major extension
definition of the term. The basic idea, however, is of the growing season. Likewise, new instances of
that there are often exceptional movements of volatility may begin to occur and may become long‐
particular prices represented in a CPI which may give lasting. For instance, oil prices have not always been
a distorted picture of the general rate of inflation, in volatile. The problem therefore is how to select
the sense that the movement in the all‐items index is those products which should be excluded from the
quite different from the movement of most prices CPI for the purposes of constructing a core inflation
comprising the index. The aim is to be able to index, and how to monitor their prices with a view to
perceive what the inflation trend would be if the possible removal at a future point from the core
effects of transient or temporary influences are index. It is for this reason that some statistical offices
removed from the index. prefer not to define a “core inflation” index as such,
but instead publish several alternative indices which
11.24 To give a simple example, if there was a rise
exclude different product groups such as CPIs which
in the rate of value‐added tax in a certain month, the
exclude: energy; energy, food, alcohol and tobacco;
index level, all else being equal, would show a rise
energy and unprocessed food; energy and seasonal
for that month. Consequently, the CPI annual (m/(m‐
food. These should not be labelled as “core inflation”
12)) change would also show an increase for that
indices, but they are useful for users who may wish
month and also for the 11 subsequent months until
to treat them as such. This approach allows national
the tax‐rise month dropped out of the calculation. If
statistical offices to publish whichever aggregations
such a one‐off effect was not accounted for in some
approximate to the likely user definitions of “core
way, the resulting inflation figures would not provide
inflation”.
a faithful representation of the underlying rate of
inflation. But as with seasonally‐adjusted CPIs, a core 11.29 Another disadvantage of excluding volatile
inflation index should not be a substitute for the products is that the excluded products may not only
unadjusted CPI, rather it provides an analytical tool be volatile but also exhibit a longer term trend of
to better understand inflation. their own, which may be different from that of the
remainder of the products in the CPI. This could well
11.25 The challenge is to find a measure of inflation
be the case for oil prices, for example, where,
that is free from, or at least less liable to, such
despite the frequent rises and falls, there has been
distortions. Several methods are used for measuring
an upward trend for many years. Excluding oil
core inflation.
permanently from the core index could, in this
The exclusion approach particular case, underestimate the long‐term
inflationary trend. Lastly, using the exclusion method
11.26 The first approach may be described as means that the core CPI is measured with a reduced
adjustment by exclusion. This involves changing the sample compared to the overall CPI. This could
product coverage of the CPI by removing those reduce the statistical reliability of the estimate if the
products for which prices are notoriously volatile excluded items account for a large share of the
(e.g. fresh food, and many energy products). By basket. Such could be the case, for instance, in some
excluding these series from the overall CPI, the developing countries where unprocessed food
modified index should be less variable than the accounts for sometimes more than 50% of total
original index. household expenditure.
11.27 Comparison of the original series with the
The smoothing approach
adjusted core series will show greater or lower
inflation rates depending on (a) the weight of the 11.30 A second method used for estimating core
components removed and (b) the price movements inflation employs a variety of smoothing techniques.
of the components removed relative to the price Typically this involves some form of averaging, either
movements from the remaining components in the at the level of the individual price series or at the
index. aggregate level, in order to show the true trend
more clearly. This method cannot entirely filter out
11.28 However, the exclusion method suffers from
irregular and un‐measurable price shocks or
certain disadvantages. The past volatility of a
seasonal effects, such as those due to varying
particular series may not be a reliable guide to the
weather but can be undertaken in conjunction with a
future volatility of the series. For example, the
CPI where the effects of changes in measurable
volatility of fresh food prices, which is due at least
irregular components, such as excise duty and
partly to random weather events or simply seasonal
mortgage interest payments, have been taken out.
availability, may at some future point in time be
168
SPECIAL INDICES
An annual rolling year index is an objective and been expressed that in trimming the left tail of the
reproducible index number method of changes in prices, such measures might disregard
simultaneously providing a seasonally adjusted and important persistent downward price trends in
smoothed index, which will generally provide a particular sectors such as computers and other hi‐
reasonably accurate picture of the trend in inflation tech goods. It can also be noted that in general,
(see chapter 9). shocks are excluded, while gradual returns to normal
prices are included, so what comes out of this
The adoption of specific adjustments to approach in the end and the interpretation of it is
eliminate particular developments not very transparent.
11.31 A third method of arriving at a measure of
core inflation is by means of specific adjustments to Other special indices
the recorded prices at certain times so as to
eliminate the effect of particular developments on 11.34 In addition to the special types of CPI
the measured aggregate rate. In general, this discussed above, countries compile many other
method does not provide a systematic, transparent special CPIs unrelated to core inflation or
or verifiable inflation indicator and for these reasons seasonality. Such indices are published for special
is not generally recommended. However, adjusting uses. They include the following examples:
for changes in indirect taxes is a special form of
• Indices relating to particular socio‐economic
specific adjustment which is quite commonly used,
groups, which are normally computed by re‐
though it nevertheless results in certain problems. It
weighting the prices collected through the
raises a number of questions: which types of tax
normal CPI price collection exercise to reflect
should be taken into consideration, and should taxes
the different expenditure patterns, but not the
on only final consumer products be excluded, or
different brands and shops visited, by the
taxes at earlier stages of production as well? The
groups in question. These indices have been
general issue of tax‐exclusive or constant‐tax CPIs is
criticised for their lack of specificity. Indices
dealt with later in this chapter.
under this heading include:
Trimmed means o By household size/composition.
11.32 There are a number of methods all based on o Pensioners.
the idea of “trimming”. Essentially, a trimmed index
is one which has had extreme values removed, o Low income groups.
thereby giving more weight to the central values, • Regional indices. These include:
which may be regarded as typical and hence suitable
as a measure of underlying inflation. o Capital cities.
169
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
o Services. Illustrative examples
• Indices that eliminate the impact of changes in
indirect taxation. These include : 11.36 The methods of calculating such indices as
those above vary according to the category.
o Constant‐tax indices. Illustrative examples relating to the compilation of
o Indices which exclude indirect tax. some of these indices are given below. The indices
excluding certain product groups may be considered
o Indices which exclude the direct impact of first, as they are not only usually the easiest to
interest rates, for example on mortgage compile, but they are related – may even correspond
interest payments. exactly to – the first type of core inflation indices
11.35 Indices such as those listed above may be discussed in the previous section.
provided by the statistical office under private
arrangements with users (including government
Indices involving product exclusion
departments) or they may form part of the data 11.37 The method is effectively to give zero weights
published regularly alongside the main CPI. It is a to the product group being excluded. Thus, in the
matter for each country to decide which special simplest case, the groups relating to alcohol and
indices to include with the release of the main CPI. tobacco (COICOP 02.1 and 02.2) may be retained in
Usually resource availability and user needs will the computation but their weights are reduced to
dictate which indices are published. All such indices zero. The remaining weights will of course fail to add
should be treated as supplementary to the main CPI. to 100 and must be re‐scaled so that they bear the
It should also be noted that the publication of same ratios to one another as before the exclusion
several CPI variants could be confusing to the public of alcohol and tobacco.
and potentially undermine credibility in the headline
11.38 A fictitious example, using just 4 product
CPI unless provided with sufficient explanation to
groups as an illustration of the method, is shown in
facilitate comparative analysis. The integrity of the
Table 11.1.
CPI can also be undermined by not publishing special
indices provided as a special service to other users, 11.39 The figures in Table 11.1 show that the
although consideration of issues relating to reliability exclusion of tobacco results in a fall of 0.3
also comes into play. Chapter 14 deals with percentage points in the index, resulting from the
presentational issues. combination of a low weight and a high price
increase.
Table 11.1 Example of exclusion of tobacco from a CPI
(A) ORIGINAL CALCULATION INCLUDING TOBACCO
Product Group CPI Weights Price Index x Weight
Index
1 Tobacco 5 1.15 5.75
2 Food 30 1.06 31.80
3 Other goods 40 1.88 43.20
4 Services total 25 1.11 27.75
Total CPI 100 108.5
(B) CALCULATION EXCLUDING TOBACCO
Product Group New Re‐scaled Price Index x
Weights weights Index Re‐scaled weight
1 Tobacco 0 0 1.15 0.00
2 Food 30 31.58 1.06 33.47
3 Other goods 40 42.11 1.08 45.47
4 Services total 25 26.32 1.11 29.21
Total CPI 95 100 108.2
170
SPECIAL INDICES
Table 11.2 Constant tax CPI
(A) FLAT‐RATE TAX
Month Basic Flat‐rate Observed Price at Index of Index of
Price tax price constant‐ observed constant‐tax
tax prices prices
Dec 3.00 0.60 3.60 3.60 100.0 100.0
Jan 3.15 0.60 3.75 3.75 104.2 104.2
Feb 3.30 0.70 4.00 3.90 111.1 108.3
Mar 3.45 0.70 4.15 4.05 115.3 112.5
Apr 3.60 0.70 4.30 4.20 119.4 116.7
(B) AD VALOREM TAX
Month Basic Tax Observed Price at constant‐ Index of Index of
Price rate price tax observed constant‐tax
(%) prices prices
Dec 3.00 15.0 3.45 3.45 100.0 100.0
Jan 3.15 15.0 3.62 3.62 105.0 105.0
Feb 3.30 17.0 3.86 3.80 111.9 110.0
Mar 3.45 17.0 4.04 3.97 117.0 115.0
Apr 3.60 17.0 4.21 4.14 122.1 120.0
11.40 A CPI for a particular sector, such as catering, g
Pt0 ,tn = ∑
( ) ⋅ w (11.1)
pi ,tn τ i ,tn
is calculated in a similar way. Here, the first task is to
decide which products or product groups comprise i =1 pi ,t0 (τ )
i ,t0
i
171
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
purchaser prices pi ,t n , the product related tax rates 11.48 It should also be noted that the CPI‐CT
assumes that the full effect of tax rate changes are
τ i ,tn as well as the tax rates of the base period τ i ,t0 . reflected in price changes at the moment the tax
rate change happens. So the CPI‐CT does not reflect
11.44 Indices which are compiled along these lines the actual real impact of tax rate changes on price
do not exclude indirect taxes as such but merely levels but merely the direct impact which tax rate
assume no changes in taxes compared with those in changes could have. However, as the CPI‐CT will
effect at the base month or period and derive their draw on the same weights that are used to calculate
value from comparisons with the ordinary CPI. the “normal” CPI, the development of these weights
Differences in the resulting inflation figures show the does reflect the longer‐term impact of tax rate.
contribution of tax changes to the overall CPI
11.49 It should be noted that the CPI‐CT is not
inflation figures. Such an index is not a type of core
without its analytical difficulties. Suppose that a
inflation index as the tax‐exclusive CPI described
country applies a value added tax (VAT) on cigarette
below. It is nevertheless of considerable analytical
packs of 20% of the pre‐tax purchase price, and that
interest, and is relatively easy to construct.
the pre‐tax price of a pack of cigarettes equals $3 in
11.45 It works backwards from observed average the reference period. The amount of tax per pack
prices, stripping out any tax changes and replacing equals $0.6 making the post‐tax purchase price
them with the base period taxes. If all the taxes were $3.60. The pre‐tax price increases by 20% and VAT
charged as a percentage of the pre‐tax price (for remains at 20% and in consequence the amount of
example, value added tax), rather than as fixed tax per pack increases to $0.72. The post‐tax price
amounts (flat‐rate tax), the CPI‐CT and the main CPI becomes $4.32. On the other hand, assume that the
would be the same, as long as the rate of tax country had applied a specific flat sum tax on
remained constant. But flat‐rate taxes will normally cigarettes of $0.6 per pack, making the post‐tax price
cause differences in the two types of CPI even when of a pack of cigarettes again equal to $3.60, but then
the tax is unchanged and may be considered decides to raise the specific tax from $0.6 to $0.72 at
perverse. Table 11.2 illustrates this point. the same time as the 20% price rise in the pre‐tax
price. Again the net result will be a post‐tax price of
11.46 In summary the main use of a CPI‐CT is for
$4.32.
comparing percentage month‐on‐month index
growth rates of the CPI and the CPI‐CT. The 11.50 In the first case, the CPI‐CT would show a
difference between these two growth rates will be price rise of 20 percent for a pack of cigarettes (the
the implicit contribution of tax changes to the overall same as in the normal index, since there was no
CPI inflation. The same applies for the comparison of change in the tax rate), while in the second case the
annual growth rates for the CPI and CPI‐CT but in this CPI‐CT would show a price rise of 16.7 percent for a
case (as in any other long‐term comparison over pack of cigarettes (which follows from applying the
several years), the term ‘constant tax rate index’ has base period tax rate). So, while in both examples the
a slightly different meaning, because for the CPI effect on the amount of tax paid is the same, the
chain index the tax rates are to be updated for each resulting CPI‐CTs differ. This is a consequence of
new link month December (see below for details). applying taxes with different structural
Thus the analytical strength is only fully realised characteristics (for example, VAT as opposed to a
when comparisons are made within a chain. specific tax) to the same product.
11.47 It is also clear from equations (11.1) and 11.51 An index which excludes indirect taxation
(11.2), that the analytical interest of the CPI‐CT only entirely avoids this feature of the CPI‐CT.
materialises if it is compared with the CPI. For
example, where tax rate changes happen in the CPI excluding indirect taxes (CPI‐XT)
observation period, the effect of these changes on 11.52 Another particular class of special CPIs are
prices will be seen by comparing the CPI‐CT with the those which exclude indirect taxes entirely. This class
CPI. For example, if the CPI is 121.3 in a given of index can be regarded as a practical approach to
observation period and CPI‐CT is 118.1, then the 3.2 the issue of dealing with indirect taxes, which
index points difference indicates the effect of tax removes from the ordinary CPI weights the share of
increases since the reference period. From here it expenditures that is accounted for by indirect taxes,
follows that the CPI‐CT equals the CPI if tax rates do and then pro‐rates the remaining weights up to 100
not change between the reference period and the per cent. For example, tobacco, which often carries a
observation period ( τ i , t 0 = τ i ,tn ). high excise tax, has a much reduced weight in a CPI
excluding indirect taxes compared with its weight in
the ordinary CPI.
172
SPECIAL INDICES
173
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 11.3 Example of a tax‐exclusive elementary aggregate price index
Index including taxes Index excluding taxes
(flat rate tax of 2.30)
Base Current Price Base Current Price
price price ratio price Price ratio
Product A $4.00 $4.50 1.13 $1.70 $2.20 1.29
Product B $3.00 $2.50 0.83 $0.70 $0.20 0.29
Geometric mean $3.46 $3.35 $1.09 $0.66
Item index 96.8 60.8
52
In the table the different price level and price changes of the
two products could indicate that they are not close substitutes
(homogenous), for which reason the Dutot should not be the first
choice. A Carli gives an index of around 79.
174
SPECIAL INDICES
Table 11.4 Example of a CPI calculation (excluding indirect taxes and mortgage interest
payments)
CPI: Including taxes CPI: Excluding taxes &
Geometric mean mortgage interest payments
(flat rate tax of $2.30) Geometric mean
Base Current Price Base Current Price
price price relative price price relative
Product 1 $4.00 $4.50 1.13 $1.70 $2.20 1.29
Product 2 $3.00 $2.50 0.83 $0.70 $0.20 0.29
Geometric mean price $3.46 $3.35 $1.09 $0.66
Item index 96.8 60.8
CPI: Excluding taxes &
CPI: Including taxes
mortgage interest payments
Arithmetic means
Arithmetic means
Arithmetic mean price $3.50 $3.50 $1.20 $1.20
Item index 100.0 100.0
11.68 The geometric mean formula implicitly CPIs, it is necessary to ensure that the coverage of
assumes that consumers will switch purchases of the elements of the CPI (weights and prices) in each
particular brands to cheaper alternatives when price of the separate regions is sufficient to calculate CPIs
relatives change. In the example above, this implies of the required accuracy. Each regional CPI is
a greater degree of substitution towards product 2 calculated in the same way as a national CPI, using
when taxes are excluded. The table also shows that regional expenditure weights and regional price
the equivalent arithmetic calculation, using the ratio changes. Of course, it may be possible to calculate a
of arithmetic means, leads to both indices being 100. CPI using national weights and regional prices (or
This illustrates another point, that differences vice versa) but these may only be regarded as
between the two indices using geometric means do approximations to a genuine regional CPI.
not necessarily imply similar differences when
11.70 Aggregating regional CPIs to the national
arithmetic means are used. Similar situations can
level is simple in principle but may not be in practice.
arise with the CPI‐CT.
The method will depend on the extent to which the
Regional Indices national CPI calculation makes use of the basic
regional price data. It is probably best to include a
11.69 The decision to compile regional indices is regional dimension in the definition of the
generally driven by economic policy needs. Although elementary aggregates. Thus, an elementary
national CPIs should in general have a national aggregate might be related to a certain product in a
coverage, it may be the case that certain regions are certain outlet type in a certain region. In the absence
excluded from coverage (either weights or prices, or of such a method, the aggregation will need to be at
both) because the value of consumer purchases in the level of elementary aggregates from each region,
the region is so small, relative to the country as a weighted together to a national total using the
whole, that even significantly different price trends weights for each elementary aggregate in each
(although in the event unlikely in the longer term) region.
would not affect the national CPI to any significant
extent. However, if a country is publishing regional
175
CHAPTER 12
Data Validation
Introduction in operational efficiencies in the compilation of the
CPI. This chapter provides an overview of the
12.1 Chapter 9 of the CPI Manual, paragraphs procedures that can be used. The validation checks,
9.139 to 9.177, discusses data editing, the which are described in this chapter, focus on the
identification of possible outliers and errors and the operational and statistical processes for checking the
verification and correction of prices data. It advises price quotations used in the compilation of the CPI
that if an “outlier” price is found to be correct, then and on the use of analytical methods to check the
it should be included in the index but if it proves to credibility of the resulting indices.
be an error it needs to be corrected by entering the 12.3 The primary goal of editing the CPI prices
correct price if this is possible or by imputing a price, data is to ensure that the resultant indices are a
unless it is considered unrepresentative by some correct representation of the movement in
pre‐defined rule. Where it is not possible to verify consumer prices experienced by the reference
the price it can be accepted or amended according population. It therefore follows that the purpose of
to pre‐defined practice. The Manual also makes the editing should not only be to obtain perfectly
important point that it is not necessary to apply the accurate raw price information but also to end up
same degree of scrutiny to all reported prices as with representative final prices and a credible price
some will carry more weight in the index than index. This emphasis on the credibility of the
others. The ILO resolution points to the fact that “CPI outcome and the potential to influence this should
estimates are subject to errors that may arise from a be the overriding guide in determining how much
variety of sources” and advises compilers that they time to spend on checking the input data. It also
“need to be aware of the possible sources of error, emphasises the importance of the index compiler’s
and to take steps during the design of the index, its task to examine the output and determine whether
compilation and construction processes to minimize or not the results can be considered to be
their impact, for which adequate resources should representative of what has actually happened during
be allocated”. The Resolution is not prescriptive in the period. The power of analysis in checking the
terms of the particular actions which should be credibility of the price index should not be
taken or the processes which should be adopted. In underestimated.
practice, outliers are detected not only through an
examination of prices but also of price movements. 12.4 Data validation and editing processes can
Thus operational procedures are often geared up to also provide an input into the long‐term
identify outliers in price movements as well as in methodological development of the CPI. For
price levels. example, an analysis of the behaviour of the prices
sample over time may show an increase in outliers.
This may indicate that an elementary aggregate is
Background becoming increasingly heterogeneous or that the
existing sample, perhaps the items in the sample or
12.2 Data validation and editing procedures that the outlets at which prices are collected, is becoming
are used to detect and correct errors in the prices less representative and needs to be updated. Using
and weights and other inputs into index construction editing techniques for this type of analysis helps to
are an important issue. Firstly, they can detect data ensure the future credibility of the price samples and
problems that could undermine the statistical the indices. Keeping a record of the results of editing
integrity of the index. Secondly, the procedures may provide additional evidence about the make‐up
themselves, if applied wrongly, can have a of the elementary aggregate and sample as well as
systematic numerical impact on measured inflation providing the necessary audit trial.
which can lead to bias. The adopted procedures also
have operational consequences. Good data 12.5 Because prices in shops may change too
validation and editing procedures can not only quickly for checks on reported prices to be made,
improve the quality of the price index but also result editing needs to be done quickly. In addition the
176
DATA VALIDATION
177
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
after all local data have been submitted, to ensure effort should be expended on editing individual
that all unusual prices or price changes have proper prices.
explanations that can be referred to later in the
• In general, all things being equal, prices from
processing cycle.
elementary aggregates with relatively small
12.13 It should be noted that reports of “no price price samples should receive more attention
change” can also be wrong ‐ but such errors can be from the index compiler. This is because, if the
difficult to detect. One way of detecting them is to weights of the elementary aggregates are
maintain a record of the last time a price change was broadly equal. Each individual price movement
noted. If no price change has been shown for a long from these elementary aggregates will have a
time, the shopkeeper can be asked to verify that the much more significant influence potentially on
price is still valid. The length of time considered to the index calculations than any individual price
be “long” will vary between products and the rate of movement from an elementary aggregate with a
inflation. As a general rule, any price that has not large number of price quotes.
changed for a year is suspect.
• Price samples from elementary aggregates with
Data validation at Regional or National high expenditure weights should also be
Headquarters examined critically as the high expenditure
weight will make all price movements within the
12.14 It is just as important that, as in the field, sample significant to the index calculation.
data validation and editing procedures are
undertaken at Headquarters and are done fast • The highest risk is associated with elementary
enough to allow price collectors to check outliers aggregates with relatively large weights but few
with shopkeepers (within the timescale before prices price quotes and with complex index
change) and within the deadline required for index construction. This situation is associated with
computation. The following represent some of the utilities and other services which account for
more common validation checks that can be applied relatively large expenditures and where there
by staff at Headquarters. may be only one or a handful of suppliers and
prices are based on complex tariffs. Petrol prices
Data editing could be another example.
12.15 Once the price information has been 12.18 There are two main categories of checking
collected and recorded it has to be edited. Data for identifying possible data errors and outliers:
editing is the process of ensuring correct and usable
• Non‐statistical checking.
data for calculation of price indices for elementary
aggregates. Data editing is sometimes referred to as • Statistical checking.
input editing. As indicated earlier there are two
steps in this process. 12.19 Some of the techniques discussed in this
section work best when applied to large quantities of
• The detection of possible errors and outliers. data. The best results will therefore be obtained if
conducted at the location where many prices are
• The verification and correction of data.
available to any one analyst. This will generally be
12.16 The detection of errors in the collection and regional offices or, more probably, head offices.
recording of price information must occur as soon as However, the techniques can be adapted and still be
possible after the information is actually collected. applied to prices held by a small collection centre as
Detection is usually achieved by examining price a way of quickly and efficiently detecting extreme
movements and checking those that exceed some prices. Abnormal prices such as sale prices, or price
pre‐defined limits, as described above, or appear to movements, such as sale recovery prices, may be
be unrealistic based on an analysis of all available excluded from manual and automated procedures
information. for the detection of outliers, in particular the setting
of upper and lower bounds, but should nevertheless
12.17 It should be noted that while price
be checked, for instance by reference to previous
collectors should examine every price they collect,
price history.
subjecting every collected price to the same level of
examination by collection supervisors and index Non‐statistical checking
compilers is not considered necessary and generally
is not feasible. It is recommended that to improve 12.20 In this handbook non‐statistical checking
cost‐effectiveness, some form of significance rating refers to the manual inspection of the data. The data
should be applied to determine how much time and are usually presented in tables, sorted and filtered.
This type of checking can be performed by price
178
DATA VALIDATION
179
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
observed price changes are normally distributed. not be unusual for many categories of items. If C is
Under this assumption, the distance between each set to 2, then 60 per cent of the actual price
of the first and third quartiles (RQ1 and RQ3) and the movements would be flagged as possible errors,
median (RM) will be the same: call this distance ‘DM’. compared with 30 per cent in the unadjusted
Operating under this assumption, the proportion of sample.
price changes that are likely to lie outside specified
12.32 The index compiler should experiment with
upper (LU) and lower (LL) limits can be estimated
different values of C for different product groups or
from a normal distribution table. The limits can be
outlet types to determine appropriate values for
defined as:
local use. The CPI Manual recommends that a
LU = RM + C x DM; and relatively low value of C should be used. C need not
LL = RM ‐ C x DM be an integer.
where C is a user defined value.
12.33 The distribution of prices and price
It will be seen later that a variation of this basic movements is rarely normal, rather in most cases a
approach is recommended to allow for the skewed skewed distribution exists. Thus the underlying
distribution of price changes which can be observed assumption of a normal distribution is invalid and
in practice. the use of symmetrical upper and lower limits will
result in a skewed distribution of prices flagged up as
12.29 If C is defined as equal to one, then
possible errors or outliers. This is operationally
approximately 50 per cent of the observations will lie
inefficient and the examination of differing
between the upper and lower limits. Using the
proportions of “low” and of “high” prices and price
standardized normal distribution, this is equivalent
movements could lead to bias.
to setting the limits at plus or minus 0.7 times the
standard deviation (σ) from the median. The A modified use of median and quartile
following table provides approximate multiples of σ values
for selected values of C and the associated
percentage of the observations that will be flagged 12.34 To use the above method in practice, the
as possible errors and outliers. CPI Manual recommends that three modifications
are made
Table 12.1 Selected values of C and the proportion • Based on the simple price ratios, the distances
of observations flagged from the median represented by price decreases
C σ multiplier Expected are not as large as the distances represented by
proportion of price increases. As an example, consider a case
observations where a product is on special offer at half price.
flagged This is represented by a price decrease of 50 per
1 0.68 50% cent. However, to return to the original price
2 1.37 17% requires a 100 per cent increase. To make the
3 2.07 4% calculation of the distance from the centre the
4 2.75 0.7% same for extreme changes for price decreases as
for the price increases, the price ratios should
6 4 0.14%
be transformed. The transformed distance, Si,
for the ith price observation can be calculated as:
12.30 In practice, there are serious shortcomings
with this method as described. RM
Si = 1 −
12.31 In normal circumstances, the majority of Ri
observations for many products will not show any
price movement. Therefore the values of the if 0 < Ri< RM
quartiles are likely to be very close to the median
Ri
value. As a result of this, using small values for C is Si = −1
likely to cause the majority of price movements to RM
be flagged as possible errors and outliers. To
demonstrate this effect, 16 additional observations if Ri ≥ RM
indicating no price movements have been added to The observations with a price ratio lower than
the sample of 30 observations used in the first RM have now been transformed into the
worked example. These are shown in example 2 in negative of the increase required to return the
Appendix 12.1. A price sample with at least one‐third price ratio to the value of RM. Any observations
of the observations showing no movements would with a price ratio equal to RM will have a
180
DATA VALIDATION
181
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
observations rather than exclude them from the of missing prices it may no longer be an
index. The general rule should be to include verified appropriate outlet for the particular items
prices; the exception should be to modify them. assigned to it, or the varieties of products whose
prices are collected in the shop may need to be
12.41 The tests for outliers are the same as those
reviewed.
for identifying potential errors, as described in the
section on data editing. Outliers can be determined • Where prices are collected by means of a
by comparing the price movement against defined questionnaire sent to the shopkeeper, individual
allowable limits. These may be either pre‐ shopkeepers often follow a regular pattern ‐
determined numerically or pre‐defined based on some return their price surveys promptly, others
statistical tests. The CPI Manual discusses identifying take some time. Staff should be encouraged to
outliers in chapter 9 (paragraphs 9.146 to 9.165). become familiar with these patterns. If the
Guidelines on the preferred methods should be system for recording the return of these surveys
provided by Headquarters. also records the expected return date, then
unexpected non‐returns can be flagged even
12.42 If outliers are to be modified, they are
though the final deadline for return of survey
usually modified to lie on the pre‐defined
forms has not passed. These shopkeepers can
boundaries of acceptable movement or to be
be contacted in advance of the final deadline to
imputed by the movement of a suitable sample of
ensure that the survey form has not been
prices. The CPI Manual urges caution when using
forgotten. Early contact can reduce the number
automatic adjustment of outlier movements.
of prices still missing by the deadline.
Automatic adjustment implies the index compiler
Shopkeepers that provide prices for heavily
has no further information to assist in making an
weighted items can also be monitored and
appropriate decision and this is not always the case.
contacted earlier rather than later.
Experience suggests that the most likely candidates
for adjustment are outliers belonging to small Credibility checking
samples in highly weighted elementary aggregates
but such guidance should not be followed blindly. 12.45 Credibility checking tests the
The index compiler should consider each case on its reasonableness of the input data and the results
individual merits, following agreed guidelines. obtained. Credibility checking of the results should
take place after the checking of the numerical
12.43 It is worth re‐emphasising that price accuracy of the data in the early stages as described
collectors and their supervisors are responsible for above. These early checks are the responsibility of
providing as much information as possible about the the price collectors and their supervisors but also
reasons for extreme price movements or levels and involve outlier detection at Headquarters. These
why they accepted the price quote as valid. In early checks should discover all straightforward
addition to checking for simple accuracy, supervisors errors like incorrect coding, such as wrongly
should also be comparing the price movements for attributing a price as a sale price, and the incorrect
equivalent products obtained by all their collectors. recording of prices.
Missing prices 12.46 Dealing with other potential errors is not so
straightforward. Results that fail a data check, such
12.44 Treatment of missing prices is dealt with in
as exceeding the pre‐defined movement limits
more detail in Chapter 7. This short section discusses
described earlier, may be judged by the index
ways of minimising the occurrence of missing
compiler to be valid as a result of referring to other
observations.
information such as market intelligence. Other
• Maintain the relevance of the sample of items potential errors might only be resolved after
priced. As part of the longer term maintenance checking with the shopkeeper, if time allows.
of price samples, items and locations for which
12.47 If it is still possible with individual price
prices are missing can be examined for common
quotations to re‐survey the price or obtain a
patterns. For instance, if many retailers are
satisfactory explanation from the shopkeeper, then
missing the same item, there may be some
the query can be sent back to the price collector and
supply problem of which no single retailer is
the data can be flagged as being verified and then
aware. This may be an indicator that an item will
subsequently corrected if found to be an error. Even
have to be replaced. If the number of regularly
if it is not possible to check with the shopkeeper
missing items is growing then the sample itself
before the computation deadline, the shopkeeper
might need to be reviewed. If a particular shop
could be questioned during the next regular visit, as
is recorded as having a relatively large number
the answer may assist the statistical office’s
182
DATA VALIDATION
183
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
184
DATA VALIDATION
Appendix 12.1 where the multiplier C has been set equal to 2.
12.1.3 The results are shown in the “Series PR’
Worked Example 1 column in Table 12.1.2.
12.1.1 Worked Example 1 demonstrates the
statistical filtering method discussed in paragraphs Table 12.1.2 Parameters and derived limits
12.27 to 12.33 of this chapter. Table 12.1.1, column
1, shows the price ratios for the illustrative sample. Parameter Series PR Series Si
RQ1 0.94488 ‐0.03907
Table 12.1.1 Price relatives showing movement RM 0.98175 0.00000
from previous period RQ3 1.05035 0.06988
Price Si Flagged Flagged DM 0.05274 0.05447
Ratio using PR using Si C 2 2
0.8138 ‐0.20638 Extreme Extreme LL 0.87628 ‐0.10894
0.8525 ‐0.15161 Extreme Extreme LU 1.08723 0.10894
0.8760 ‐0.12072 Extreme Extreme
0.8990 ‐0.09205 OK OK 12.1.4 As described in paragraph 12.34 of this
0.9086 ‐0.08051 OK OK chapter, the price ratio series can be transformed to
0.9135 ‐0.07471 OK OK provide more equal weighting between negative and
0.9339 ‐0.05124 OK OK positive price movements. The transforms are
0.9414 ‐0.04286 OK OK repeated here as:
0.9553 ‐0.02769 OK OK
0.9608 ‐0.02180 OK OK RM
0.9658 ‐0.01651 OK OK Si = 1 − if 0 < Ri< RM
Ri
0.9668 ‐0.01546 OK OK
0.9702 ‐0.01190 OK OK
0.9724 ‐0.00962 OK OK Ri
0.9817 ‐0.00005 OK OK Si = − 1 if Ri ≥ RM
RM
0.9818 0.00005 OK OK
0.9843 0.00260 OK OK
12.1.5 The transformed observations are shown in
0.9869 0.00525 OK OK
the column labelled ‘Series Si’ in table 12.1.1. The
1.0034 0.02205 OK OK
quartiles, median and calculated limits for the
1.0050 0.02368 OK OK
transformed series are shown in the third column in
1.0061 0.02480 OK OK
table 12.1.2. The increased value for DM for the
1.0301 0.04925 OK OK
transformed sample shows that the transformation
1.0571 0.07675 OK OK has increased the distances for the price decreases
1.0824 0.10252 OK OK while leaving the distances for positive movements
1.0909 0.11118 Extreme Extreme the same.
1.0931 0.11342 Extreme Extreme
1.1300 0.15101 Extreme Extreme 12.1.6 The last two columns in table 12.1.2 show,
1.1550 0.17647 Extreme Extreme respectively, the observations that would be flagged
1.2296 0.25246 Extreme Extreme for further examination (indicated by the word
‘extreme’) for the original price ratios and the
1.2304 0.25327 Extreme Extreme
transformed price movements.
12.1.2 The first and third quartiles (RQ1 and RQ3) and Worked Example 2
the median (RM) can be obtained using the quartile
function in Microsoft Excel. The average distance of 12.1.7 Worked Example 2 demonstrates the same
the quartiles from the median (DM) is defined as: statistical filtering method but with an additional 16
price ratios added to the sample. All the new price
DM = (RQ3 ‐ RQ1)/2 ratios show zero change. The same calculations are
The upper and lower limits are then calculated as: done but on a sample of 46 instead of 30
observations. Table 12.1.3 shows the sample of price
LU = RM + C x DM; and ratios and the transformed price movements, as well
LL = RM ‐ C x DM as the observations flagged for further observations.
185
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
186
DATA VALIDATION
187
CHAPTER 13
Sources of Errors and Bias in a CPI
Introduction 13.3 This chapter lists the potential errors and
biases that can occur in a CPI and also points to ways
13.1 Chapter 11 of the CPI Manual summarises the of minimising their impact. It represents a
types of sampling and non‐sampling errors to which convenient summary of the relevant issues raised in
CPIs are subject, including the issue of bias in earlier chapters and provides references to where
measurement. It also discusses the measurement of advice on solutions can be found.
error and bias including the difficulty of making
quantitative estimates, particularly for non‐sampling Summary of General Measurement
error, and provides guidance on procedures to Problems
minimise errors and bias. It concludes by
emphasising the importance of providing detailed 13.4 The accuracy of a CPI refers not just to the
descriptions of compilation methods and data measurement of statistical precision, as indicated by
sources and the sources and magnitude of sampling the mean squared error of the estimator of the index
and non‐sampling errors to enable users to level or annual percentage change. There are several
understand the limitations of the index. The ILO other indicators of accuracy relating, for example, to
Resolution, emphasising the latter point, states that the relevance of the statistical framework, issues
“In order to ensure public confidence in the index, a arising from measurement and processing errors and
full description of the data collection procedures problems generated by non‐response such as the
and the index methodology should be prepared and non‐availability of prices and the modelling used for
made widely available” and that “The imputing missing prices and for the introduction of
documentation should include ………… a discussion of new or replacement items. Additionally, in practice
the accuracy of the index estimates”. the measurement of accuracy is often constrained by
methodological issues and limited resources. This
Background section provides an overview of the general
measurement problems associated with the
13.2 The literature on CPIs discusses, from several compilation of a CPI and the related issues which
perspectives, the general types of potential error need to be addressed by the compiler.
and bias, to which all price indices are subject. In this
chapter three perspectives are presented. Aggregation
• A summary of the general measurement Sub‐indices for different reference
problems, most of which have been illustrated populations
in preceding chapters of the Handbook.
13.5 Households differ from one another in their
• A review of the arguments presented in various
consumption patterns: they will shop in different
studies that attribute bias in CPIs to
outlets, purchase different varieties of a product and
insufficiently accurate treatment of quality
as a result will often pay different prices even for the
change, consumer substitution and other factors
identical products. The total CPI is an average
– the first two representing two of the most
measure of the price change experienced by the
significant sources of potential bias in a CPI.
reference population which may be quite different
• A description of the main sources of sampling
from the price change experienced by a single
and non‐sampling error that arise in estimating
household in the population or even for specific
a CPI from a set of samples relating to
population subgroups. Empirical studies have shown
commodities (products, varieties, items),
evidence of variation in inflation rates across
outlets, households. These issues exist in all
households based on income, age of household
statistical series based on samples but there are
head, family structures, areas of residence, etc.
some unique features in CPIs which make some
aspects of sampling a challenge. 13.6 When a CPI for a specific sub‐group of the
population is consistently higher or lower for several
188
SOURCES OF ERRORS AND BIAS IN A CPI
periods over time, there is merit in producing and Seasonality
publishing these indices, especially for such uses as
indexation of social security benefits and measuring 13.10 The objective of a CPI using a traditional
changes in real income, etc. But the fact that prices Laspeyres approach is to measure the change
data are collected from retail stores and other between two periods (usually monthly) in the
outlets and not directly from households gives rise expenditure necessary for the acquisition of a fixed
to a major caveat in the construction of such indices annual basket of goods and services, according to
which are intended to represent sub‐populations as expenditure patterns in the base period. However,
defined by socio‐economic characteristics. This is the consumer does not purchase a standard fixed
because variations are likely to exist among basket in all months of the year, but purchases a
households and household groups with respect to basket which changes with the seasons of the year55.
the particular types and qualities of goods and Expenditure patterns may be different because of
services they purchase and the type of shops they the supply‐side (i.e. changing quantities are offered
frequent. It is difficult with a household budget at different seasons such as at harvest time) or the
survey to associate the economic and demographic demand‐side (i.e. consumers may seek differential
characteristics of households with the actual items quantities for seasonal reasons such as buying warm
purchased and prices being paid. Such indices are clothes in winter). The two consumption groups in
normally computed through a re‐weighting of the which the greatest concentration of seasonal goods
prices collected through the normal CPI price is usually to be found are clothing and footwear and
collection exercise, to reflect the different fruits and vegetables. Chapters 9 and 11 describe the
expenditure patterns, but not the different brands complicated issues relating to weight allocation,
purchased and shops visited by the groups in price measurement and index aggregation of
question. seasonal goods.
189
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
encourage conservation of natural resources, putting the above options more succinctly, the
whereas other suppliers may provide bulk‐purchase choice of adopted approach in compiling the CPI is
discounts where the unit price falls in relation to the between a basket of new purchases of durables in
quantities purchased. Whereas for most goods and the base period, payments in the base period for
services there is identity between the average price already existing durables of the household or the
for a unit and the marginal price for the last unit, expenditure on a basket of services supplied by
there is no such identity in these cases. The solution these durables in the base period. It is possible to
that may be adopted is to attempt to collect the data construct different types of price index for each of
on the distribution of consumption by the household these approaches, each being appropriate to
in the base period to generate a series of consumer particular uses of the CPI. Different methods will
profiles which can then be used for calculating require different expenditure data in the base period
average unit prices for different levels of and price data in the current one. Housing usually
consumption. The necessary data for the current represents the consumer durable with the largest
price measurement are: the pattern of consumption expenditure on durables and is one where there
by households in the base period and the differential remains a lack of consensus on which is the
unit prices for each level of consumption. These data appropriate approach to adopt in a CPI. Issues
may be obtained from the suppliers of the services, relating to housing are covered in Chapter 9. In
such as the utility companies. practice, the use of a CPI for many purposes can lead
to compromises in conceptual purity. For example,
13.13 Similarly, differential prices may be offered
some CPIs comprise a mixture of the acquisition
to different consumers based on their economic and
approach and, for housing, either the payments or
demographic characteristics. Differential tariffs by
user cost approach.
income of household head enables cross‐subsiding
of public services and may be found in areas of
health services, childcare, education, etc. Bias in a CPI
13.14 The solution in all these cases is similar to 13.16 This section reviews the categories of error,
the one presented for annual prices and again is either in pricing or in index construction, which can
covered in Chapter 9. lead to bias in the CPI. The emphasis here is on the
Consumer Durables categorisation of errors, along with a consideration
of their likely size, rather than on methods to reduce
13.15 The treatment of consumer durables is or eliminate the errors.
handled extensively in Chapter 9. It poses a
complicated and intriguing conceptual and practical 13.17 The four categories of bias listed by the
price measurement problem. A price index for Boskin Commission in 1996 are: upper‐level
consumer durables should measure the price change substitution bias; elementary aggregate bias; quality
over time of a fixed basket of durables but the change and new goods bias; and new outlet bias.
methodology will depend on the underlying 13.18 It is important to note that an evaluation of
conceptual basis of the index. Under the acquisition a CPI for potential sources of bias is only meaningful
approach the total value of all goods and services against an appropriate benchmark, that is, the
delivered (i.e. acquired) during a given period, conceptual basis of the index. During the course of
whether or not they were wholly paid for or this analysis, a number of dilemmas relating to
consumed during the period, is measured and conflicting conceptual needs implicitly arise. Added
reflected in the index. With the payments approach, to this are the practical problems of measurement.
the index is compiled on the basis of the total No single inflation measure can fully satisfy a diverse
payments made for goods and services during a range of user needs. However, different target
given period, whether or not they were delivered to indices can present compilers with similar issues of
the household. Finally, the user cost (or practical measurement which lead to common
consumption) approach takes account of the total solutions despite different conceptual bases and
cost during a given period of all goods and services analytical frameworks.
consumed during that period. The distinction
between the three approaches is particularly Upper‐level substitution bias
important for purchases financed by some form of 13.19 Upper‐level substitution bias arises when
credit, notably major durable goods and housing. CPIs aiming to measure the cost of living, use a
These are acquired at a certain point of time, used Laspeyres‐type formula which in reality only
over a considerable number of years, and may be provides an upper bound on a cost‐of‐living index
paid for, at least partly, some time after they are under certain assumptions about consumer
acquired, often in a series of instalments. In essence,
190
SOURCES OF ERRORS AND BIAS IN A CPI
191
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
13.29 The quality change problem involves trying 13.33 For services, overall quality trends can also
to measure price changes for products which exhibit be a matter of subjective valuation. New technology
quality changes. The old item is no longer available, has led to unambiguous improvements in the quality
but a replacement or alternative one is on sale and of many consumer durables and other goods. By
priced by the collector. If the effect of quality change contrast, in service sectors such as mail delivery,
on price is, on average, either to reduce or increase public transport and medical care, it can be difficult
the price, or the quality is improving or deteriorating to evaluate changes in quality. Airline travel, for
but this is not reflected in a change in price, then a example, has become safer and faster but perhaps
bias will result if the prices are compared without a less comfortable and reliable in keeping to scheduled
quality adjustment, in other words, if the products timetables in recent decades, and the lack of cross‐
were treated as comparable. sectional variation in these characteristics makes the
use of hedonic or other quality adjustment
13.30 Unlike substitution bias, which can be problematic.
estimated by a comparison of alternative formulae,
quality change bias must be analysed on a product‐ 13.34 Further details on quality change are
by‐product basis. Products and their associated provided in Chapter 7.
index components are likely to experience widely New product bias
varying rates of quality change over time. Moreover,
the methods used for quality adjustment, described 13.35 New product bias, like elementary
in Chapter 7 on Substitution and Quality Change, will aggregate bias, can be divided conceptually into two
also vary. If the adjustment process is inappropriate, components. The first concerns the failure to bring
there will be an error, and if the errors are in a new products into the CPI sample with sufficient
systematic direction, there will be a bias. speed. This can lead to upward bias if those new
products later experience large price reductions that
13.31 A crucial point to recognise is that the are not reflected in the index, for instance after the
direction of overall quality change does not imply novelty value wears off or production costs reduce
the same direction of any quality change bias. Critics as a result of economies of scale. The second
sometimes assume that CPIs make little or no quality component, in the context of a cost‐of‐living index, is
adjustment and that they must therefore the welfare gain that consumers may experience
overestimate price change in view of the many when a new product appears. Clearly, the latter will
demonstrable improvements over time in the quality not be viewed as a bias when the cost of living index
of goods and services. In reality, a great deal of is not stated to be the CPI’s measurement objective.
indirect or implicit quality adjustment takes place.
For any component index, the issue is whether the 13.36 New products can be: products that replace
direct or indirect method chosen for quality predecessor items, for example CDs replacing vinyl
adjustment overestimates or underestimates the records and tapes; product varieties that widen the
relative quality of replacement items in the CPI range of consumer choice, such as imported beers
sample. The resulting bias can be either positive or and ethnic restaurants; or products that represent
negative. wholly new categories of consumption, such as
microwave ovens or mobile telephones. A useful
13.32 Empirical evidence on quality change bias distinction made in the Manual and in this Handbook
has been based largely on extrapolation from is the distinction between evolutionary and
individual studies of particular products. These revolutionary goods and services. Each benefits from
individual studies may involve, for example, different operational procedures to ensure that the
comparisons of hedonic regression indices with the CPI basket remains up‐to‐date and relevant.
corresponding CPI series or estimates of the value of
192
SOURCES OF ERRORS AND BIAS IN A CPI
193
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
194
SOURCES OF ERRORS AND BIAS IN A CPI
• Under‐coverage occurs when elements in the sample. If the price levels and rate at which
target population do not appear in the sampling prices change in the outlets from which prices
frame. The sampling frame of outlets often have not been obtained differ from those which
suffers from under‐coverage, for instance as a have, the results of the price survey will be
result of the time‐lag in enumerating new biased. Non‐response may also be encountered
outlets and including them in a business in an HBS. Total non‐response occurs when
register. In other instances readily available households drawn in the sample refuse to
sampling frames with the necessary information cooperate entirely. Partial non‐response occurs,
on market shares etc are not available. For for instance, when certain households refuse to
example, many statistical offices exclude give information about their expenditure on
purchases made on the internet and from mail certain commodity groups, such as tobacco and
order firms, due to lack of information on size of alcohol.
turnover and the customer database including
13.54 Non‐sampling errors can be reduced by
the range of purchases by households.
applying appropriate quality assurance methods at
• Non‐response errors can arise from the failure to all stages of the CPI compilation, as described in
obtain the required prices data in a timely chapters 6, 7, 10 and 15, including the HBS that
manner from all the outlets selected in the serves as the main source for CPI weights.
195
CHAPTER 14
Publication, Presentation, Analysis and
Interpretation
Introduction documented with a public explanation in order to
protect professional integrity.
14.1 Paragraphs 71‐80 of the 2003 ILO Resolution 14.5 Another principle underlying release practices
give the agreed international standards on is that the CPI statistical press notice should be
dissemination. These should be strictly observed. separate from any related political statements
Not to do so could undermine the integrity of the referring to government policies.
index and the associated statistical system. Chapter
13 of the Manual on Consumer Price Indices 14.6 Additional frameworks have been produced
provides detailed guidelines in support of the which support the above principles and provide
Resolution. mechanisms for enhancing dissemination standards.
NSIs may wish to draw on these. They include the
14.2 In summary, the ILO Resolution states that International Monetary Fund’s General Data
the consumer price index should be produced and Dissemination System (GDDS) and Special Data
presented objectively, without political interference, Dissemination Standard (SDDS)57.
and in such a way as to aid the understanding of
users and meet their needs. Timing of publication
14.3 More specifically, as far as dissemination is
concerned, the Resolution states that publication 14.7 The timing of publication of the index will
should be as soon as possible after the index generally be determined by reference to the price
reference period and in a readily accessible form to collection period. The decision on the timing of
all users at the same time and to a timetable which is publication will involve a trade‐off between
published well in advance. In addition, to facilitate achieving the earliest possible release and the need
transparency the Resolution states that the rules to quality assure and check the data and index
relating to publication of a CPI should be publicly calculation to reduce the risk of errors, and to
available, including details of pre‐release access and provide users with a detailed analysis to assist
how the content of the press notice is determined. interpretation. Other operational considerations will
Guidelines on the detailed presentation of price include managing the publication workload within
indices in the CPI publication, including what and the statistical office. For instance, an executive
how the CPI should be presented in the statistical decision may need to be taken on whether there is
press notice, are also covered. As well as the figures the capacity within the office to publish the CPI and
themselves, background information on other key economic indicators on the same day.
methodology should be provided to aid analysis and Users should be consulted to gauge their
interpretation. Where a CPI figure can be revised, preferences. For instance, it will be necessary to take
revisions should be clearly labelled and follow a into account their capacity to deal with large
published revisions policy. volumes of new data and, conversely, the advantage
to them in receiving a coherent set of new economic
14.4 The underlying principle is that responsibility
data which provide a more comprehensive and up‐
for the statistics and their presentation rests with
to‐date picture of what is happening in the
the National Statistics Institute (NSI). Decisions
economy. Judgements will need to be made by the
about the content of the statistical press notice
should be professional decisions which take into
account the needs of users and the confidentiality of 57
The GDDS provides standards to guide the public dissemination
data suppliers. Publication should be timely and of economic and financial data. The SDDS is designed specifically
follow a fixed timetable which is determined by for those countries who seek access to international capital
markets. Both have been designed to enhance the availability of
operational requirements. Any non‐compliance with timely and comprehensive statistics. The SDDS is designed to
published guidelines and procedures should be improve the functioning of financial markets.
196
PUBLICATION, PRESENTATION, ANALYSIS AND INTERPRETATION
197
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
where the latest data endangers the well‐being of 14.19 Where the pre‐release figures are provisional
the country, for example if an unexpected jump in because they are subject to further quality assurance
inflation would otherwise lead to pressure on the prior to publication this should also be clearly stated.
national currency. More general guidance on the presentation of
provisional figures and subsequent revisions in the
14.16 Pre‐release access should be transparent and
public Press Release is given later in this chapter.
pre‐announced. Transparency can be achieved by
publishing the rules for determining pre‐release and 14.20 In some countries the conditions applying to
by the NSI maintaining an up‐to‐date list of named pre‐release access and the responsibilities it places
individuals (or their job titles) given pre‐release on pre‐release recipients are stated in writing to
access, and making this publicly available. In individuals at the time of granting pre‐release
determining the rules consideration should be given access. An example of a letter authorising pre‐
to: release access is given at Appendix 14.2. The
advantage of this is that there can be no doubt that
• Which person determines pre‐release access
those who have pre‐release access are fully aware of
(normally the director or chief statistician of the
the responsibilities that this brings
NSI).
14.21 Those being granted pre‐release access can
• What factors determine such access (for
be asked to confirm that they accept the conditions
instance, when required for operational or
that apply before receiving their first pre‐release
policy reasons such as setting interest rates or
copy of the figures. Consideration should also be
announcing increases in state benefits or when
given to asking those with pre‐release access to
ministers are required to respond to latest
make a regular declaration, say once a year, that
inflation trends‐ see previous paragraph).
they still need such access and that they only use the
• The conditions that apply when pre‐release information themselves for the purpose intended.
access is granted, i.e. the rules that recipients Alternatively, pre‐release recipients can be sent a
have to abide by. letter reminding them of their responsibilities, the
rules relating to pre‐release circulation and the
• The timing of pre‐release access (normally consequences of not following the rules.
expressed with reference to the number of
hours or working days prior to publication of the 14.22 These procedures reduce the risk of problems
Press Release). occurring and put the national statistics institute in a
stronger position to take corrective action if any
• The statistics covered. The pre‐release data set breaches of the pre‐release arrangements occur.
may be restricted to the headline CPI and key
sub‐indices, for instance where the full Press
Release is not available at the time of pre‐ Political and policy statements by
release because it is still being put together. To ministers and their officials
assist interpretation it may include a short
summary of the main factors determining the 14.23 It has already been mentioned in the
latest index figure. introduction to this chapter that one of the
recognised principles underlying the release of the
14.17 Appendix 14.1 gives an example of a “model”
CPI and other official statistics is that the statistical
pre‐release document used to disseminate figures
press notice should be separate from any related
before publication. It includes a re‐statement in
statements referring to government policies which
summary form of the conditions applying when pre‐
Ministers may wish to make. Release of the latter
release is granted.
prior to the release of the corresponding statistics
14.18 It is strongly recommended that the pre‐ should be discouraged, as such statements should be
release document should have the appropriate seen as a response to the statistics rather than as an
security marking together with an indication of when early warning of the story those statistics tell.
the markings no longer apply (usually on
14.24 Exceptionally, consideration may be given to
publication). For instance: “Market sensitive‐
including in the statistical press notice a factual
PERSONAL‐CONFIDENTIAL‐STATISTICS UNTIL XX
statement on government policy if this helps to put
JANUARY 2009”. Consideration should also be given
the latest CPI figures in context. It is recommended
to the use of “numbered” copies and circulating
that such factual statements are checked for
these in secured double envelopes which have been
accuracy with the appropriate policy official in the
initialled and labelled “FOR THE PERSONAL
relevant Ministry.
ATTENTION OF XXXXX”.
198
PUBLICATION, PRESENTATION, ANALYSIS AND INTERPRETATION
199
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
200
PUBLICATION, PRESENTATION, ANALYSIS AND INTERPRETATION
201
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
method may be regarded as some sort of 14.51 The CPI itself should always be quoted as a
compromise between the monthly and annual level relative to the latest base month (in this
measures, though it is subject to seasonal effects. If, example December 1990 = 100). For instance, the
for example, utility prices tend to be increased in CPI for August 2002 is 176.4.
January each year, then the index for the third
quarter compared with the second quarter will not Calculation of annual and quarterly
reflect this change. averages
14.47 The precise presentation of the CPI data in 14.52 Users may be interested in annual and
the statistical press notice will depend on user needs quarterly average indices in their own right. For
and on the limitations of the data. However, as well example, they may be quoted in contracts or they
as the standard calculations already mentioned may be used as deflators for the National Accounts
there can be a particularly strong case for providing or to set against other social or economic indicators
supplementary presentations to enhance the scope and analytical purposes.
for analysis and facilitate better understanding of 14.53 Annual average indices are defined as the
what the latest figures indicate. In computing the arithmetic mean of the twelve monthly values for
associated calculations, issues may arise, for the year in question. It is common practice to use
example if the calculation spans a change in base published rounded indices for their computation so
period. Issues of interpretation may also arise that they can be replicated by users from published
where, for example, there is a significant element of data:
seasonality in the prices charged by retailers.
1 12
Calculation of percentage change between I12av = ∑ It
12 t =1
any two months
14.48 Typically, a conventional chain‐linked index 14.54 Quarterly indices are defined similarly. It
can be used to calculate changes between any two should be noted that there will not usually be any
months after the base month. For example, if the all‐ year or quarter with an average index of exactly 100,
items CPI for May 1988 is 106.2 and that for August if the indices are always calculated so that a
2002 is 176.4 then the change between these particular month (in the previous section December
months is: 1990) equals 100.
(176.4 / 106.2 − 1)× 100% = 66.1% 14.55 The annual average inflation rate can be
derived from the annual average indices as the
14.49 However, if the base month changed in change in the annual average index from the year
December 1990, then the series based on the before. For example, if for the all‐items CPI for 2001
previous base month, say December 1980, is used up the annual average is 173.3, and if the annual
to December 1990 and the series based on average for 2000 is 170.3, then the percentage
December 1990 is used for the remainder of the change is (173.3 ‐ 170.3) /170.3 x 100 = 1.8%. The
period. For example, if the indices for July 1990 and difference in the mathematics means that this will
December 1990, based on December 1980, are 384.7 not in general exactly equal the average of the
and 394.5 respectively, and that for July 1991, based individual annual percentage changes for January,
on December 1990, is 101.8, then the change from February, through to December, but in practice the
July 1990 to July 1991 is: difference should be small. The calculation based on
the percentage change in the annual average indices
(101.8 / 100 × 394.5 / 384.7 − 1)× 100% = 4.4% has the advantage of transparency over the average
14.50 This can be seen from table 14.1. of the individual annual percentage changes for each
month as it is more easily replicable from published
data. Either average figure will usually be closer to
Table 14.1 Calculation of change in index when
the change between the middle of the year and the
spanning a re‐basing
middle of the previous year than to the changes
between the beginning and end of each of the years.
Month/Year Old series New series
(Dec 1980=100) (Dec 1990 =100) Calculation of monthly average prices
July 1990 384.7 (384.7x100)/394.5=97.52
Dec 1990 394.5 100 14.56 A monthly average price provides users with a
July 1991 101.8 simple statistic which can be readily related to
personal experience. For meaningful results, the
Then ((101.8/97.52)‐1)x100 = 4.4% items included in a particular average should be
202
PUBLICATION, PRESENTATION, ANALYSIS AND INTERPRETATION
reasonably homogeneous across all outlets and over Contributions to changes in the annual
time. For instance, it may not be very meaningful to inflation rate
calculate an annual average price for a category
covering “Household Equipment” which may include 14.60 Contributions to changes provide a powerful
anything from a gas cooker to a washing‐up bowl or analytical tool for understanding movements in the
for “Cultural Services” where the services included CPI. The contribution of a component to a change in
range from the printing and developing of colour the all‐items CPI over a given period of time is
film, to renting a television, to the admission price defined as the change that would have occurred in
for an historic monument. It may also be helpful to the all‐items index if that component had undergone
give the number of price quotes on which the its observed change but all other component indices
average is based and some measure of dispersion, had remained frozen at their values at the start of
e.g. the standard deviation or inter‐quartile range. the period (and all expenditure weights were
unchanged). The effect of each component depends
14.57 The average prices should be calculated in a on both the size of its price change and its weight.
similar way to the item indices. Thus, if an item The formula for calculating the contribution of a
index is built up from regional item indices, with component to the monthly change in the CPI is given
regional weights, then the national average price for below. As the definition of the variables above
that item should be a weighted average of the makes clear, it is important that these calculations
regional average prices. are performed using indices with the same base
Indices for subgroups and special period (i.e. in the example following all index levels
are based on the previous December = 100).
aggregates
14.61 Contribution of component i to the monthly
14.58 In addition to the standard sub‐indices
change in the all‐items CPI =
published alongside the all‐items CPI, which are
compiled using the underlying expenditure ⎛ I ti ⎞ I ti −1 w ti
classification such as COICOP, special indices can be ⎜⎜ i − 1 ⎟⎟ × 100 × a × (1)
computed to suit user requirements. For example, ⎝ I t −1 ⎠ I t −1 1000
some statistical offices include separate indices in
their press notices for goods and for services. These where:
indices are calculated from the same item indices Ia = all‐items CPI
and same item weights used in the calculation of the
standard published sub‐indices. The difference is Iit = index for component i (current base period) in
month t
that the item indices are mapped to these special
indices; aggregation then proceeds in the usual way. Wit = weight (parts per 1000) of component i in all‐
This is illustrated in Table 14.2. items CPI in month t
14.62 The formula for calculating the contributions
Table 14.2 Aggregation of Special Indices
of components to the all‐items CPI 12‐month rate
Items Standard Special Item Item (i.e. the annual rate of change) is given in equation
from Indices Indices weights Index (2). Note that this calculation assumes, for
CPI (e.g. index (as % of illustrative purposes, that the CPI has a single chain‐
1=Items All Items link in January each year, based on December=100.
A1+B1) CPI)
Item A1 Index A Index 1 0.4 102 14.63 Contribution of component i to annual
change in all‐items CPI =
Item A2 Index A Index 2 0.6 101
Item B1 Index B Index 1 0.3 104 w ti −12 (I Li − I ti−12 ) w ti (Iti − 100) × I a (2)
Item B2 Index B Index 2 0.7 103 × × 100 + × L
1000 I ta−12 1000 I ta−12
Standard Index A is calculated as:
where, additionally:
(0.4x102+0.6x101)/(0.4+0.6) =101.40
Iit‐12 = index for component i (previous base period =
Special Index 1(Items A1+B1) is calculated as: 100) in month t
(0.4x102+0.3x104)/(0.4+0.3) =102.86 IiL = index for component i in ‘Link’ month (i.e. month
when the current base period=100).
14.59 Special Index 2 (Items A2+B2) would be 14.64 If there is no change in weights between the
two points in time for which the contribution to the
calculated by the same method.
203
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
12‐month rate is required, then formula (2) overall inflation rate and that contributions can be
simplifies to: negative as well as positive.
Contribution of component i to 12‐month change in 14.66 Figures 14.1 and 14.2 show two illustrative
all‐items CPI = examples of how to apply these formulae in practice.
Figure 14.1 Example of calculation when the weights change
Using formula (2) above, the contribution of housing to the all‐items CPI 12‐month rate for October
2003 can be calculated as shown below.
The published (chained) index values, based on December 1989=100, for housing and the all‐items
CPI are as follows:
Published (chained) index (Dec 1989=100)
Dec 2001 Oct 2002 Dec 2002 Oct 2003
Housing 218.4 232.8 236.7 248.3
All‐items 173.3 177.9 178.4 182.6
In order to work out the contribution of housing to the all‐items CPI 12‐month rate for October
2003, it is necessary to “unchain” the indices so that they are based on the most recent December.
This is done by dividing the current month’s index by the previous December’s figure. For instance,
the housing index for December 2002 (the link month) is calculated as:
236.7
I Li = × 100 = 108.38
218.4
Performing this calculation for each of the dates gives the following set of unchained index values:
Unchained index based on previous December
Dec 2001 Oct 2002 Dec 2002 Oct 2003
Housing 100.00 106.59 108.38 104.90
All items 100.00 102.65 102.94 102.35
The contribution of housing to the 12‐month rate for October 2003 can then be calculated as
follows, given that the weights for housing in 2002 and 2003 are 199 and 203 parts per thousand
respectively:
199 (108.38 − 106.59 ) 203 (104.90 − 100 )
Contribution = × × 100 + + × 102.94 = 1.34%
1000 102.65 1000 102.65
Thus, housing contributed 1.34 percentage points to the all‐items CPI 12‐month rate in October
2003. The way that these contributions to the 12‐month rate are usually used is as follows. For any
given month (e.g. October 2003) the contribution of each group to the 12‐month rate is calculated.
This is also done for the previous month (September 2003 in this case). The October contribution
less the September one is described as the contribution to the change in the all items 12‐month
rate between the two months. Thus taking the case where housing contributed 1.40 points to the
12‐month change to September and 1.34 points to the 12‐month change to October, then it
contributed 1.34 ‐1.40 = ‐0.06 points to the change in the 12‐month rate between September and
October.
204
PUBLICATION, PRESENTATION, ANALYSIS AND INTERPRETATION
Figure 14.2 Example of calculation when the weights do not change
Using formula (3) above, the contribution of transport to the all‐items CPI 12‐month rate for
October 2003 can be calculated as follows, where:
• The weight for transport is 72 parts per thousand.
• The transport index values for October 2002 and October 2003, based on December
1999=100, are 121.5 and 119.7 respectively.
• The all‐items index values for October 2002 and October 2003 respectively are 118.1 and
120.5.
72 (119.7 − 121.5)
Contribution of transport = × × 100 = −0.11%
1000 118.1
Thus transport contributed ‐0.11 percentage points to the percentage point change in the all‐items
rate.
205
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 14.3 The impact of using rounded figures in calculating derived statistics
Index level 12‐month rate
Jul‐01 Aug‐01 Jul‐02 Aug‐02 Jul‐02 Aug‐02
Un‐rounded
Index A 171.44 171.96 174.75 175.34 1.9% 2.0%
rounded indices so that they can be replicated for effects can occur when publishing rounded indices to
transparency. one decimal place, and then calculating percentage
changes from these rounded indices which are then
14.75 There are two approaches which can be used
themselves rounded to one decimal place.
in the computation of derived statistics. One
approach is to calculate derived statistics from un‐ 14.76 This is illustrated by the example in Table
rounded (and unpublished) monthly indices; the 14.3. In this example, published rounded figures
other is to use published rounded indices. The first indicate that the inflation rates for two CPI series ‐ A
approach limits the impact of rounding effects and and B ‐ have both fallen by 0.1 percentage points
ensures that future re‐referencing of indices to a (from 2.0 to 1.9 and from 1.1 to 1.0 respectively).
new base year will not lead to revisions to one‐ However, the picture based on un‐rounded figures
month and 12‐month percentage changes. shows series A to have increased by 0.1 percentage
However, it means that the derived statistics cannot points (from 1.9 to 2.0) and series B to have fallen
always be calculated from the published indices. by 0.3 percentage points (from 1.2 to 0.9).
Calculation from published indices might be
14.77 The choice of which approach to use involves
considered desirable both for transparency and to
a trade‐off between accuracy and transparency. For
allow users to be able to replicate calculations. The
regularly published series it is normally
second approach is transparent in that all derived
recommended that un‐rounded figures are used to
statistics can be linked back to the published indices.
maximise precision. It is important to let users know,
This may be considered particularly important given
by the use of appropriate footnotes, which method
the wide range of uses to which the CPI may be put,
has been adopted.
including the indexation of state benefits and of
private contracts. However, some serious rounding
206
PUBLICATION, PRESENTATION, ANALYSIS AND INTERPRETATION
Appendix 14.1 Pre‐release figures and accompanying letter
MARKET SENSITIVE ‐ PERSONAL‐CONFIDENTIAL‐STATISTICS UNTIL XX JANUARY 2008
To: XXXXXXXXXXXXXXX COPY NUMBER: XX
Copies: XXXX
XXXX
PRE‐RELEASE FIGURES ON CONSUMER PRICE INDEX: DECEMBER 2007
The consumer price index (CPI) for December 2007 will be published at 9:30 am on Tuesday XX January
2008. Attached is a near final draft of the statistical press notice/enclosed are the key indices. [These are
provisional figures/advance estimates and may be subject to revision prior to publication].
You are reminded that the Consumer Price Index is a market sensitive statistic to which you have pre‐
release access. The figures should not be passed on to another person and any attempt to profit from
this information or encouraging others to do so could constitute a market abuse and lead to criminal
prosecution.
Any accidental or wrongful release should be reported immediately and may lead to an enquiry.
Wrongful release includes indications of the content, including descriptions such as “favourable” or
“unfavourable”.
Please prevent inappropriate use by treating this information as strictly confidential.
CONSUMER PRICES DIVISION
XX JANUARY 2008.
Consumer Price Index
Percentage Changes
over one over 12
Month Months
20XX August X.X X.X
September X.X X.X
October X.X X.X
November X.X X.X
December X.X X.X
Advance X.X X.X
Estimate
January X.X X.X
207
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 14.2 Letter confirming pre‐release recipients
CONSUMER PRICE INDEX: INCLUSION IN PRE‐RELEASE CIRCULATION LIST.
This is to confirm that your name has been included in the pre‐release circulation list for the
Consumer Price Index statistical press notice/ the key indices. This/these will be sent to you XXX
hours before publication, that is at XX o’clock on the XXX day before publication. You will be
required to sign on receipt.
The figures are provisional figures/advanced estimates and may be subject to revision prior to
publication].
You are reminded that the Consumer Price Index is a market sensitive statistic to which you have
pre‐release access. The figures should be kept in a secure place and should not be passed on to
another person. Any attempt to profit from this information or encouraging others to do so could
constitute a market abuse and lead to criminal prosecution.
Any accidental or wrongful release should be reported immediately and may lead to an enquiry.
Wrongful release includes indications of the content, including descriptions such as “favourable”
or “unfavourable”.
Pre‐release is granted on account of your current post. You should notify this office if you change
posts as this may have an impact on your entitlement to pre‐release access to the CPI.
Please sign and return the attached form acknowledging that you accept the conditions.
208
PUBLICATION, PRESENTATION, ANALYSIS AND INTERPRETATION
Appendix 14.3 Statistical press notice
The example below is illustrative. The precise content and format of a press notice will depend on user needs.
For example, the percentage change compared with the previous month might attract more attention than
some sub‐indices and this might present a strong argument for its inclusion on the front page.
209
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Note: For more information on quality reports (point 6 of the background notes to the illustrative statistical
press notice) see Chapter 15.
210
CHAPTER 15
Organisation and Management
Introduction Data Dissemination System (GDDS) and Special Data
Dissemination Standard (SDDS).
15.1 Chapter 12 of the CPI manual covers 15.4 The recommended structures and processes
organisation and management issues relating to described in this chapter serve as best practice. For
price collection and the compilation and publication some national statistics institutes, resource and
of the index together with performance capacity constraints will sometimes mean that not all
management, development and training. This of the specific recommendations in this chapter can
chapter of the handbook draws on the ILO manual be followed. In these circumstances, national
but additionally gives more practical guidance statistics institute should follow the principles
including templates for the necessary underlying the recommendations, implement plans
documentation associated with a good organisation which address the key issues whilst reflecting local
and management system. The ILO resolution does circumstances and aspire to the structures and
not specifically address organisation and processes described later in the chapter.
management.
15.2 While this chapter is headed: Organisation Overview of Quality Management
and Management, the main focus is on System
management. It is difficult to prescribe any specific
organisational structure for the CPI in isolation. The 15.5 A Quality Management System for the
latter will be governed by the relevant public service Consumer Price Index should be in place and should
requirement, which will be country specific, and by cover all aspects of CPI data collection and
the overall organisational model adopted within the compilation including the audit of prices, validation
national statistics institute. In particular, it will of the production cycle itself and any annual review
depend on whether the national statistics institute process which focuses on strategic and longer term
is: structured along the lines of the statistical value‐ issues. The latter is strongly encouraged because it is
added chain (for example: sampling, data collection, a major way of learning from past experiences and
coding and editing under separate management to of identifying and taking forward actions which will
compilation and analysis of outputs) to reap the improve the future quality of the CPI.
benefits of specialisation; or by statistical output (for
example, household budget survey, consumer price 15.6 A conscious decision needs to be taken on
index) to reap the benefits of integrated team whether to include in the main quality management
working; or by subject area (for example, national system, periodic review processes such as chain
accounts, prices or economic statistics) to reap the linking and the updating of the CPI basket and
benefits of subject area knowledge and expertise. technical development work such as the
Sometimes the CPI is grouped for management introduction of better sampling techniques and
purposes with the producer price index in order to methods of quality adjustment for replacement
exploit synergies relating to index construction. In goods. Such work should be managed using project
other instances, the CPI is grouped with national management techniques, if not included in the main
accounts because the two share a significant quality management system.
customer base and national accountants use CPI sub‐ 15.7 Figure 15.1 provides an overview of a proto‐
series as deflators. type quality management system for the monthly
15.3 Frameworks for quality reporting and for collection of prices and compilation of the CPI. Each
improving CPIs are covered in detail in Chapter 18. aspect should be seen as inter‐dependent and an
Chapter 14 refers to two frameworks which provide integral part of the whole. The precise outline of
mechanisms for enhancing dissemination standards, such a system will depend on the detailed
namely the International Monetary Fund’s General arrangements and approach to price collection. For
instance, the diagram allows for the possibility that
some prices data is collected directly from the
211
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Figure 15.1 Quality Management System
Audit Local
Collection
Local Price
Collection
LONGER TERM – Annual Strategic Review
Prices Centrally
SHORT TERM – Monthly Review
Collected Calculated
Centrally Indices
Audit Monthly Production Processes
Data Processing &
Validation
Briefing & Checking
Monthly Publication
212
ORGANISATION AND MANAGEMENT
Level 3 ‐ Work Instructions • More efficient production of documentation as
it helps with initial compilation and reduces the
15.12 The work instructions give full details on need to print and circulate paper copies.
exactly how a task should be carried out.
• Better informed staff because they have
Level 4‐ CPI Technical Manual immediate electronic access to latest
documentation including desk instructions with
15.13 The CPI Technical Manual describes the
search facility by subject and author.
procedures used to produce the Consumer Price
Index and the price indices derived from it. It is • Better quality control as authors, with the
aimed at users of the CPI who want to know how the involvement of the document control custodian,
data are collected and analysed and what formulae can readily amend, date stamp and reference
are used in the calculation together with other number updates and, just as importantly, access
methodological detail. to non‐authors is restricted to “read only”.
Documentation control • Better search facility, for example if staff are
looking for cross references to a particular
15.14 All documents in the Quality Management subject such as “chain‐linking” or “weights”.
System should be subject to document control. The
procedures on Documentation Control should 15.16 Where a national statistics institute does not
explain how the document control works to ensure have the necessary IT infrastructure to operate an
that all staff have access to the most up‐to‐date electronic system, it is still important that a
version when carrying out their work. In some NSI document control custodian is appointed with the
offices this is done by storing such documents as the task and authority to keep a record and quality
Quality Manual, Procedures and Work Instructions assure the most up‐to‐date paper documentation.
electronically in a database managed by a document The same principles of good documentation
Figure 15.2 Documentation
Quality Policy and System Description
Manual
Procedures Who and What
Work Instructions How
Reference Documents Reference Documents
e.g. Technical Manual
213
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
management apply whether the documents are • It minimises risks of errors by adding effective
stored electronically or are kept in paper folders. controls including a framework for the initiation,
evaluation and implementation of change.
15.17 A form for a document control template is
given at Appendix 15.1. • It increases public confidence in the CPI, for
instance as an index produced in accordance
Internal Audits with internationally recognised standards.
• It provides a basis for more effective training of
Overview experienced staff and the induction of new staff.
15.18 The auditing function is represented by the 15.22 Internal audits can provide the same level of
left‐hand column of diagram 15.1. It is strongly objectivity and discipline where external
advised that internal audits are carried out regularly accreditation is not an option and can be used on
according to a systematic timetable and that they their own or in conjunction with external audits. It is
are conducted by an internal audit team. Audits recommended that internal audits are undertaken
should cover all aspects of the monthly and annual by people who are at arms length of the processes
processing cycle to ensure that the management being audited. Other options include external audits
systems are fully implemented and provide effective carried out by CPI compilers from another statistical
controls for Quality Assurance. The purpose of each institute – in effect a form of peer group appraisal.
audit should be to verify that operational procedures
and controls comply with the documented The role of an audit team and the
procedures and determine their effectiveness in responsibilities of its members
delivering a consumer price index which is fit for
15.23 It is recommended that an “audit team” is
purpose. Thus an audit should aim not only to
assembled, which at a minimum consists of a Quality
ensure that the index compilers adhere to the
Manager and an internal auditor, each with a
agreed procedures but that the procedures are
distinct, well‐defined and complementary role.
improved where found lacking.
Where this is not possible because of resource
15.19 Audits should be conducted by personnel constraints the two roles can be combined or the
having appropriate training and experience to do so, roles can be undertaken in conjunction with other
ideally by people who are sufficiently detached from responsibilities. In some statistical offices the role of
day‐to‐day operations to take an independent and auditing is undertaken by a methodology branch
objective view unhindered by close involvement in whose responsibilities extend beyond auditing to the
the production and compilation of the consumer development and introduction of better statistical
price index. Too much familiarity can lead to techniques. Whatever arrangement is adopted, it is
unwarranted assumptions about the important that sufficient numbers of staff, qualified
appropriateness of the procedures and the extent to in auditing, are available in order to carry out these
which they are followed. audit functions.
15.20 What follows is a description of the standard 15.24 The Quality Manager should be responsible
processes associated with performing an internal for:
Audit and producing an Audit Report, with
• Producing an Audit Schedule, managing it and
illustrative examples of the associated
ensuring it is updated as necessary.
documentation.
• Agreeing and specifying the objectives and
15.21 To achieve external accreditation, such as ISO
scope of the audit.
9001(2000), additionally requires the organisation to
undergo an external audit twice a year by the • Managing implementation of the audits
appropriate accreditation body. The advantages of specified in the Schedule.
obtaining external accreditation are that:
• Ensuring the auditor is properly trained.
• It gives a discipline to reviewing and specifying
production procedures and to properly • Ensuring that, where possible, the auditor is
documenting and quality controlling them. independent of the function being audited.
• It can provide the efficiencies associated with a • Ensuring Review Reports are written.
ready made standard for documentation and • Ensuring audit action points are implemented.
quality control and the added trust associated
with a well‐tested system. 15.25 The Quality Manager should prepare a
schedule of audits covering all aspects of the CPI
214
ORGANISATION AND MANAGEMENT
• Updating documentation where necessary. • Conduct structured interviews.
• Taking responsibility for monitoring and • Ask to be shown documentary evidence
following‐up actions from the audits (spreadsheets, signatures on spreadsheets) to
themselves. support staff responses to questions.
Objectives of an Audit • Identify issues as they arise and advise the index
compiler.
15.28 The objectives of an audit need to be clearly
defined and agreed before it starts. The precise After the audit
objectives will depend on local circumstances, but • Produce a report, with recommendations
may be stated in general terms as follows:
• Provide compilers with the opportunity to
• To assess compliance with documented comment and then agree on an action plan.
procedures.
• Try and resolve any disagreements and where
• To provide assurance to senior management appropriate include the index compilers
that the agreed Quality Management System is comments in the report.
being implemented, and is effective and
relevant. • Note opportunities for improvement to current
procedures as well as non‐compliance with
• To identify improvements required and any them.
necessary corrective actions and preventive
measures. This includes taking action to prevent Audit report
problems occurring in future.
15.31 It is recommended that similar issues
• To ensure activities are in accordance with the identified in several areas are grouped under the
Quality Management System. This includes same heading and documented as one audit
identifying whether there are any activities observation. Clear and succinct headings should be
being undertaken which affect the quality of the used for each issue identified and a short description
index but which are not included in documented given of what was found. Observations can either be
procedures. made in order of the most significant ones first or
they can follow a logical sequence, generally the
• To ensure the procedures are adequate. order in which the work activities underlying the
Auditing procedures and techniques compilation of the index are carried out.
15.29 The success of an audit relies not only on 15.32 For each audit observation, suitable remedial
clear objective setting and well trained auditors but action should be recommended together with a
statement of who is responsible for carrying out the
215
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
remedial action and the date when the action should 15.39 Targets for the delivery of a CPI may cover
be completed. both quality (data and statistical methodology) and
timeliness and may encompass both the tracking of
Risk assessment the data collection and compilation processes and
15.33 In order to prioritise follow‐up actions it is the quality of the final output ‐ the CPI. The national
recommended practice to attribute an overall score statistics institute will need to decide which are the
to the risk associated with the issues arising from the most relevant for its CPI. Chapter 12 deals with data
audit. This can also be done individually for each validation and Chapter 13 deals with measuring the
issue identified and is a decision to be made by the quality of the CPI itself. Possible targets for
NSI. monitoring monthly performance in terms of the
process of compiling the CPI and maintaining its
15.34 The results of an audit may be categorised as relevance, that is the intermediary steps in compiling
Low Risk if the issues which have been identified are a CPI and the subject of this chapter, may include:
unlikely to arise and if they did would result in major
difficulties for the published CPI, such as errors in • Timeliness ‐ process delivery times meeting the
the published index. A High Risk categorisation will agreed schedule. For example, whether the
be justified if there is a significant chance that unless prices data was entered on to the computer and
addressed the issue could lead to an error in the edited to the agreed timetable or whether index
index, that is, the error will be of high impact and compilation took place on time so as not to
there is a significant chance of it happening. The potentially compromise publication.
results of all other audits should be deemed to be of • Accuracy – This might include the proportion of
Medium Risk. prices which are found to be wrong, the number
15.35 If judged appropriate, the Quality Manager of prices collected compared with target
may issue an instruction for a follow‐up audit. The sample, errors in the compilation of elementary
proposed date for this audit should be added to the aggregates.
Schedule for Follow‐up Audits and Progress Checks,
• Delivery – This might include the delivery of
and the actual date of the audit should be added to
planned reviews of specific sub‐indices and
the schedule of internal audits. Whenever possible,
methodological reviews.
the follow‐up audit should be carried out by the
same auditor who undertook the initial audit. 15.40 Clearly, an agreed monthly schedule of tasks
associated with the compilation of the CPI is
15.36 Appendix 15.3 gives an example of a template
required in order to be able to measure performance
for an audit report and the recording of follow‐up
as well as being required for the management of the
actions.
processes.
216
ORGANISATION AND MANAGEMENT
Figure 15.3 Fishbone Diagram: an example
PLANNING TEAM
Time Management
Poor time management
Poor communication
Too much workload
Problem: Error percentage
above target value
Not willing to assist Poor product knowledge
Lack of communication skills Methodology inconsistent
RESPONDENTS METHODOLOGY
217
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Table 15.1 Example of a table to formulate action plans
How will you How will you know that the
What do you (Who)
implement the improvement goal has been (When) (When )
want to achieve? Responsible
actions? List your achieved? List your Start date End date
List your goals person
methods measurement methods
agrees that further iterations will no longer provide management. An example of a training evaluation
them with useful information. form for providing feedback on specific courses is
given at Appendix 15.5. It is important to clearly
15.54 The results are then analysed the results of
state the training objectives beforehand and ideally
the fishbone firstly by looking for those items that
these should be agreed with the line manager.
appear in more than one category. These become
the 'most likely causes". 15.60 Two particular aspects of training which are
particularly relevant to the CPI are the training of
15.55 For those items identified as the "most likely
price collectors and the training of those involved in
causes", the team then agrees a listing in priority
using the prices data to compile a CPI.
order, with the first item being the most probable"
cause. Price collector training
15.56 Specific action plans are then developed for 15.61 One possible approach is to split the training
the most likely causes to the problems to be into two phases:
addressed, using the what, when, who, where and
how technique. Additional skills required and any • Shadowing – where all new price collectors
development needs are also identified. See Table accompany an experienced collector working on
15.1. the CPI before they go through formal in‐depth
training. This, in part, is to test their aptitude for
Long‐term annual Reviews the job.
15.57 The longer‐term element of a quality review • Formal training – consisting of the following
system takes a higher level strategic look at modules:
objectives and should be conducted through the
annual planning cycle, where such a cycle exists, and • Desk training:
should address both the quality of outputs assessed o Introduction to the CPI: what is the CPI
against user needs and the processes by which and why it is important. Why
quality is achieved. Ideally, the latter should be laid confidentiality is important.
down in an annual Quality Management Action Plan
annexed to the main CPI work programme (see The price collection schedule.
Chapter 18). Dealing with the retailer, how to
make initial contact and arrange
Training visits, the carrying of identification
15.58 The procedure of putting together badges.
documentation examined as part of a Quality Representative product selection.
Management System, together with feedback from
the process of auditing, including the monthly and The importance of item
annual review processes, should help to identify descriptions and how to choose
training needs. These should be incorporated into replacement items.
personal development plans for individuals and in a Non‐standard situations and
higher‐level Training Plan for the team responsible prices, for example sales, missing
for the CPI. A template for a personal development goods, negotiating prices in
plan is given at Appendix 15.4. markets.
15.59 It is recommended practice to put in place
• Training in the field:
pre‐ and post‐training evaluation procedures to
monitor the quality of training and its value against
business needs and as an input into quality
218
ORGANISATION AND MANAGEMENT
219
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
220
ORGANISATION AND MANAGEMENT
Appendix 15.1: Documentation Control Template
CPI Technical Board has Mr Smith,
Calculating CPI food Change in
agreed that in the future Consumer
XX/XX/XX item weights (non‐ 2.1 process with
weights should be taken Price
seasonal) effect from...
from National Accounts. Statistician
Methodology changes in
Calculating and
pricing structures for mobile Mr Smith,
updating price index Change in
telephones – new Consumer
XX/XX/XX for tele‐ 2.5 process with
methodology agreed by CPI Price
communications effect from...
Technical Board. Reflects Statistician
services
changing market
Additional
checks to be
Last audit indicated current Ms Brown,
Desk instructions for carried out
checks inadequate resulting CPI
XX/XX/XX checking and editing 3.1 based on
in incorrect prices entering Operations
of prices month‐on‐
the CPI. Manager
month price
change.
221
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 15.2: Audit Schedule Template
SCHEDULED AUDIT LAST AUDIT
RISK
PROCESS AND/ DATE AUDIT ASSESSMENT FOLLOW UP
LAST DATE OF TARGET
OR DOCUMENT Ref AUDIT ALLOCATED (High, REQUIRED?
AUDITOR AUDIT DATE
TO BE AUDITED DUE TO Medium or (Y/N)
Low)
Calculating CPI
food item
2.1 January Mr Graham Ms Jones XX/XX/XX HIGH N N/A
weights (non‐
seasonal)
Calculating and
updating price
index for 2.5 January Mr Graham Ms Jones XX/XX/XX MEDIUM Y April
telecommunicati
ons services.
Desk instructions
for checking and 3.1 January Mr Graham Ms Jones XX/XX/XX LOW Y April
editing of prices.
222
ORGANISATION AND MANAGEMENT
Appendix 15.3: Audit Report Template
AUDIT REPORT – CONSUMER PRICE INDEX
Process Audited: Calculating CPI food item weights (non‐seasonal)
Owner: CPI Statistician (name)
OBSERVATIONS/FINDINGS
1. The CPI Technical Board has agreed that, in the future, weights should be taken
from National Accounts – this will lead to various changes to the processes.
2. The process document needs updating in order to reflect changes in methods
3. It was agreed that the initial detailed instructions should be written at the same
time as carrying out the process.
ACTION REQUIRED
DUE: TAKEN:
1. Changes made to the process documentation to
XX/XX/XX XX/XX/XX
reflect new methods.
2. Follow‐up Audit XX/XX/XX XX/XX/XX
RECCOMMENDATIONS FOR FUTURE QUALITY IMPROVEMENTS
1.
2.
3.
AUDITOR: Mr Graham AUDIT DATE: XX/XX/XX
FOLLOW UP AUDIT – DETAILS OF ACTION TAKEN
1.
2.
3.
AUDITOR: AUDIT DATE:
223
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 15.4: Personal Development Plan
Name: Ms Jones
Start Date: XX/XX/XX End Date: XX/XX/XX
Grade: Principal Statistician Line Manager: Mr Graham
Organisational Unit: Consumer Prices Post Held: Statistician
INSTRUCTIONS
Competency: Areas of competence where you have identified that some development is needed in order to enable
you to reach your full potential in your current position, or prepare you for promotion.
Development Need: Why you feel that it is necessary for you to develop the specified competency.
Method of Learning: How you will develop the identified competency i.e. e‐learning, internal or external training,
shadowing?
Position Holder Evaluation: Consider what knowledge and skills you have gained from the development and what
impact they have had on your outputs and the outputs of your team.
Line Manager Evaluation: Consider what has been the impact of the new knowledge and skills on the outputs of the
job holder and the team.
PERSONAL DEVELOPMENT PLAN
Competency Development Method of Position Holder Evaluation Line Manager Evaluation
Need Learning
1 Managing Need to develop Microsoft The course enabled me to The job holder has
data improvements to Excel ‐ learn new methods for implemented two valuable
the monthly external validating data and the validation checks which
validation checks training formula and functions improve the quality of CPI
on CPI data – needed in Excel to carry data.
Microsoft Excel. out this task. I have since
introduced 2 additional
validation checks to the
monthly process.
2 Knowledge Need to Shadowing I found the shadowing of The job holder now has a
about sources understand price price collector price collectors useful to better/complete
collection in 2 locations understand the particular understanding of the price
processes and challenges in price data collection process enabling
how they feed into collection. This has them to carry out their
data delivered for enabled me to better current role effectively and
CPI compilation. understand the data used also to be available to
for compilation of the CPI. cover price collection as
part of the disaster
recovery plan.
224
ORGANISATION AND MANAGEMENT
Appendix 15.5: Training Evaluation Form
Name of Trainee:
Learning Activity:
Date of Activity:
i.
ii.
iii.
Yes No Partially
3. Which elements of the course did you find particularly useful? Please list below and provide a
short explanation.
4. Which elements of the course, if any, did you find least useful? Please list below and provide
a short explanation.
5. Did you learn new skills and knowledge from the course?
Yes No
Yes No
7. Will you be able to apply the learning when you return to the workplace?
Yes No
Yes No
9. Please rate the materials used throughout the classroom based sessions
225
CHAPTER 16
User Consultation
Introduction 16.5 Three principles underlie effective
consultation:
16.1 Understanding users’ needs is essential in • Clarity over who is being consulted, about what
ensuring the relevance of the consumer price index issues and for what purpose.
and its proper use. It is therefore important that
mechanisms are in place to obtain users’ views on a • Simplicity in presentation of the issues, in
regular basis both to facilitate effective stakeholder particular well structured documentation which
engagement in the planning process, for instance in effectively summarises the key points.
identifying priority areas for methodological 1. Transparency in the process, including records
improvement or the development of additional being kept of all the documentation including
indices, and to provide users with advice on the user views and subsequent discussions and
strengths and weaknesses of using different indices actions plus a record of the decision and the
for different purposes. User consultation is arguments supporting it.
particularly important in connection with the CPI
given the fact that there is a particularly broad and 16.6 It is important that consultation
disparate body of users and there are many different arrangements relating to the CPI are put in the
uses of the index. public domain in order to protect the integrity of the
index and the processes associated with its
16.2 Paragraph 82 of the 2003 ILO Resolution on development and production. Consultation with
CPIs states the responsibilities that NSIs have to users and complete openness in the mechanisms
consult users, and Chapter 13 of the CPI Manual has used to consult will help maintain confidence in the
a short section at the end on user consultation, CPI.
including the role of Advisory Committees.
16.7 User consultation on the CPI may be part of a
16.3 Neither the ILO Resolution nor the Manual more general consultation process relating to setting
addresses the more general but equally important priorities for the overall statistical work programme
issue of stakeholder engagement and management of the NSI ‐ covering all outputs and the submission
and the need to develop appropriate mechanisms to of bids to the finance ministry for resources ‐ or may
manage the diverse range of customers and complex be specific to the CPI. The latter may be necessitated
relationships. This is considered at the end of this by its special status. It forms a major part of this
chapter after considering the specific issue of user chapter and is addressed in more detail below after
consultation. a brief discussion on the practicalities of conducting
a general user consultation process as part of the
The general principles of planning process.
consultation
User consultation for planning future
16.4 Engaging users should be an integral part of work programmes and submitting
the statistical process. When the CPI work
programme is under review and developments in CPI
budget bids
outputs and methodologies are being contemplated,
16.8 Producers of official statistics have an
consideration should always be given to whether
obligation to produce a statistical programme that
there are issues on which users need to be informed
takes into account the needs of users and the
or consulted. Where there are such issues,
available resources and reflects priorities. This
allowance for consultation needs to be built into the
necessitates a process of user consultation which
future work plan, including the resource
should cover all official statistics, especially in
requirements and a time allowance for the period of
respect of a statistic as important as the CPI.
consultation.
Consultation arrangements will depend in part on
226
USER CONSULTATION
227
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
• Scope. This refers to the breadth and coverage has scope to decide on which issues warrant
of the CPI in terms of population and consideration and to initiate work. The
expenditure. For example, whether the alternative is an ad hoc committee which is only
expenditure of visitors from abroad should be convened when a particular issue arises on
included. which advice is required. The arguments are
finely balanced and depend, to some extent, on
• Definition. This establishes what the CPI should
the governance arrangements relating to the CPI
measure. Essentially it prescribes the conceptual
and other official statistics. In particular, the
basis. For example, whether it is a cost‐of‐living
arguments will be influenced by whether there
index or a measure of pure price inflation.
is a formal committee structure in place for
• Methodology. This refers to the statistical statistical assessment of the NSI’s outputs and
techniques used to collect data and construct for putting together future work plans and the
the CPI in the best way which is compatible with extent to which a standing CPI advisory
its scope and definition. The focus of committee fits into this structure.
methodology is on implementation.
• Whether the remit extends to scope and
16.14 Decisions on methodology, as defined above, definition or is restricted to methodological
are the responsibility of the national statistician. It is issues. For instance, whether the committee
also the responsibility of the national statistician to should pronounce on whether the current scope
offer advice and make recommendations on the is the appropriate one for fitness of purpose
scope and definition of the index regardless of the when taking into account the uses of the index.
decision making processes. The latter are conceptual
• Who convenes an advisory committee and
issues determined by the use of the index58. But the
decides on membership and who should it report
obligation on Government and its relationship with
to. If the committee’s remit is restricted to
Parliament arising from the use of the CPI for the
methodological issues then it should be the
indexation of state benefits and pensions, and, for
national statistician, although the latter should
example, tax allowances, imposes a particular
consider undertaking consultation before
obligation on the National Statistician to consult and,
decisions are made, for instance on
in some instances, defer the decision to government
membership, if this increases the acceptability
or parliament. Transparency of processes is
of the committee in the eyes of users. It is less
particularly important in this situation, especially the
clear cut where authority lies if the remit
publication of the advice offered by the national
extends to scope and definition. As indicated
statistician.
earlier the national statistician may defer
16.15 A flow diagram illustrating such an approach decisions on scope and definition to the
is given at Appendix 16.1. It is, of course, just one of government or parliament (see also the section
a number of possible models. Flow diagrams can be on the role of the statistical office and the
useful in articulating the decision making process national statistician).
and in providing greater transparency.
• The make‐up of the committee. In particular,
whether it should be restricted to price index
Practical options for consultation‐ experts from academia and NSIs or should
advisory committees include the main users. The answer to this
depends on whether the view is taken that the
16.16 Clearly there are many possible variations to advisory committee is essentially a group of
the model shown at Appendix 16.1 and practices can experts whose function is to look at detailed
vary significantly between different statistical methodological issues or whether it has a
offices. Issues to be decided include: broader remit to investigate issues relating to
scope and definition. If the latter, a common
• Whether the CPI advisory committee should be a practice is to appoint a broad‐based committee
standing committee, that is, one which meets and nominate a technical sub‐committee to give
regularly and is not time‐limited and which looks advice on the more technical issues which arise.
at the continuous development of the CPI and
• The inclusion of non‐government users and
experts helps to reinforce the “independence”
58
Paragraphs 1 to 7 of the 2003 ILO Resolution on CPIs introduce of the committee, and the inclusion of non‐
the different definitions of a CPI associated with different technical experts, such as employers and trade
conceptual bases for different purposes. A discussion on how the
scope of the CPI is dependent on the uses for which it is intended union representatives, provides not only a
is given in paragraphs 8 to 14 of the Resolution. helpful user perspective but also helps users to
228
USER CONSULTATION
understand the practical constraints of index of official statistics and the public’s confidence in
production and the inevitable compromises that them. The requirement of the NSI and the National
sometimes need to be made. It can add to the Statistician to fulfil these commitments would argue
perceived integrity of the index. for, all things being equal, the National Statistician
convening an advisory committee and determining
• Who decides on whether to accept the
its terms of reference and for the advisory
recommendation, or should the conclusions be
committee to report its findings to the National
binding? The answer to this depends to a large
Statistician.
extent on the answers to the first three bullet
points listed above. But professional and 16.22 Correspondingly, operational arrangements
methodological issues should be under the would normally lie with the NSI which would provide
remit of the national statistician. a chairperson and a secretariat. The national
statistician should consider chairing an advisory
Advisory committees ‐ operational board if the issues under consideration are
particularly important or sensitive.
arrangements and terms of
reference Stakeholder engagement and
management
16.17 It is important that a CPI advisory committee
16.23 Users are a sub‐set of the community of
is furnished with sufficient resources to enable it to
stakeholders in the sense that every member of the
do its job satisfactorily and that it is seen to be
community has an interest in good quality official
independent. Clear terms of reference should be
statistics even if they do not use them. For example,
drawn up which address the issues raised in the
a reliable CPI is essential for effective economic
previous section. Arrangements should be
management ‐ an unreliable index can lead to bad
transparent and the publication of background
policy decisions and an eventual drop in the
papers and discussion, as well as the terms of
standard of living of each citizen.
reference and final recommendations, should be the
norm. Permission should be sought before evidence 16.24 Carefully nurtured relationships, based upon
submitted to the advisory committee is published. regular and open dialogue, will not only assist
This is best done by exception ‐ that is making a forward work planning and the management of user
general announcement that the presumption will be expectations but also will help to reinforce the
that any evidence submitted to the committee can legitimacy of the CPI and to raise the profile and
be made publicly available. influence of the NSI and increase trust in and
support for it. Stakeholder management implies a
16.18 It is strongly advised that a date is given for
more proactive relationship than stakeholder
the delivery of any reports which are commissioned.
engagement and is more appropriate for managing
16.19 An example of a model terms of reference is relationships with influential stakeholders who have
given at Appendix 16.2. a major interest in the CPI. Most will be major users
16.20 It is also important that advisory committee such as the relevant government departments,
reports are properly structured with a clear which determine economic policy, and the national
bank which has operational responsibilities relating
statement of the issue, an evaluation of the options
and the arguments supporting the final to the running of the economy, for example for
setting interest rates.
recommendation. A possible structure for an
advisory committee report is given at Appendix 16.3 16.25 Common components of a stakeholder
although the precise make‐up will depend on the management strategy include:
terms of reference. Advisory committee reports
should be published. • Identifying the different types or groups of
stakeholders.
229
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Figure 16.1 Stakeholder Engagement and Management Matrix
Keep Satisfied
High
Manage Closely
Members of Parliament
-Parliament
-Policy Advisers -Government
-United Nations (ILO, IMF -NSI Governing Board
etc)
-Government Departments
-National Audit Office
Influence
-Regional Administrations
-Central Bank
Monitor (minimum effort) Keep informed
-Public -Professional Statistics
Society
-Statistics User Forum
-Professional User Groups
-Research community
Low
-Academia
Low Interest High
230
USER CONSULTATION
Appendix 16.1 Decision Making
Note: In basic question 2, gilts refer to Government index linked securities.
Basic Question 1: Issue
relates to scope or
definition of CPI?
No
Yes
National Statistician
convenes expert group
and consults key users
National Statistician
convenes expert
group and consults Group makes
key users recommendations to
National Statistician
on proposals
No
Wider Consultation by
National Statistician
National Statistician
presents proposals to
Government or
Parliament and
makes Do users raise
recommendations on problems that
implementation require the
recommendations
to be significantly Yes
changed?
No
Government or
Parliament decides National Statistician
on course of action decides on the
change to be made
*or overrules the intentions of parliament on the indexation of state benefits
231
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 16.2 Example of terms of reference for a CPI Advisory Committee
In this example the CPI Advisory Committee is a standing committee and scope and definition are deemed to
lie outside its remit. There is no “general” as opposed to “expert” user representation, the former being part
of a process of user consultation, the latter as part of a process of drawing on user experience. The terms of
reference may be as follows:
“To advise the National Statistician and make recommendations on issues submitted to it by the National
Statistician which relate to the methodological development of official consumer price indices produced by
the Government Statistical Service taking into account considerations relating to the accepted scope and
definition of the indices produced.”
In this context, the following definitions apply:
1. Scope refers to breadth and coverage in terms of population and expenditure.
2. Definition establishes what an index should measure. Essentially it prescribes the conceptual basis,
for example, whether it should be a cost‐of‐living index or a pure price index.
3. Methodology refers to statistical techniques used to collect data and construct the index in the best
way that is compatible with its scope and definition. Thus the focus of methodology is on
implementation.
The Committee will report to the National Statistician.
In framing its recommendations the Committee will take into account best international practices and the
current ILO Resolution on Consumer Price Indices.
The Consumer Price Indices Advisory Committee (CPIAC) will also support the National Statistician in fulfilling
the latter’s obligation to produce a high level statistical programme that reflects the needs of users. The
CPIAC will do this by:
1. Reviewing progress made on the introduction of methodological developments including on the
implementation of earlier Committee recommendations that have been accepted by the National
Statistician.
2. Quality assuring, prior to implementation, methodological developments relating to routine index
maintenance, such as the changes that form part of a periodic exercise conducted to ensure that the
consumer prices ‘shopping basket’ – the selection of items priced each month to measure inflation –
is up to date and representative of consumer spending patterns and that the approach to price
measurement remains relevant.
3. Providing an input into future research agendas with a view to maintaining the relevance of
published price indices and in particular drawing the national statistician’s attention to any concerns
it has about the methodology used in index construction.
It will meet as directed by the National Statistician and at least annually to fulfil its quality assurance role with
respect to the annual updating of the consumer prices “shopping basket”. It will submit an annual report
within three months of the end of the financial year to which it relates. Background papers, minutes of
meetings and recommendations submitted to the National Statistician will be published to coincide with the
release of the corresponding annual report.
In general the views of the Committee will be expressed through its annual report and the minutes, but the
Committee can produce and publish papers of its own on the topics it is asked to investigate by the National
Statistician. It may also publish individual reports and recommendations on specific topics passed to it for
consideration.
Membership of the Committee will consist of experts, including expert users, and will normally consist of no
more than eight people and at least six. Members will be chosen for their expertise and will not have a
representational role. Membership will depend on the topics under discussion but the core membership will
consist of expert producers and users. In addition there will be three academic statisticians/economists, who
will be chosen according to the skills relevant for the topics under discussion.
The Committee will be serviced by a small secretariat from within the National Statistics Institute.
These arrangements are designed to deliver statistical integrity and consumer price indices that are
accessible and fit for purpose.
232
USER CONSULTATION
Appendix 16.3 Example of a possible structure for an advisory committee
report
1. TERMS OF REFERENCE
2. Membership
a. Names, job titles, affiliations and dates of appointment
3. Introduction and summary of findings
4. The issue(s)
a. User requirements
b. International guidelines
c. Methodological options
d. Experience of other national statistics institutes
5. Recommendations with argument
6. Appendices
a. Background to current methodology
i. History
ii. Reasoning
b. International guidelines and practices (additional detail)
c. Technical details of proposed methodological recommendation(s)
d. More detail of options rejected by the committee
e. List of people who submitted evidence together with evidence submitted
f. Estimated numerical impact of recommendation on index
233
CHAPTER 17
CPI/ICP Integration and Harmonisation
Introduction 17.4 The second core strategy – the linking
approach ‐ is more appropriate for many developing
17.1 This chapter goes beyond the CPI itself to a countries. Pairs of countries are identified for
close relative: the International Comparison bilateral comparisons. The pairs of countries can
Programme (ICP). Many countries – in fact the then be used for multilateral comparisons by using
majority – participate in the ICP: 146 were included chains constructed on the basis of the bilateral links.
in the 2005 round. The aim of the ICP is to make Again this approach should, at least in principle,
international comparisons of economic aggregates make better use of CPI data as countries are paired
such as Gross Domestic Product (GDP) per person, according to the degree of similarity in their retail
using specially calculated exchange rates known as markets. But neither method is fully developed.
Purchasing Power Parities (PPPs). PPPs eliminate the Further exploratory work is needed before either
effects of different price levels and consumption strategy can be followed.
patterns across countries. They result in more 17.5 There are major benefits available to
meaningful comparisons than if ordinary exchange individual countries that participate in the ICP.
rates are used. Participation can provide the catalyst for
17.2 Appendix 4 of the CPI Manual addresses the improvements to the national price statistics
topic of integrating CPI work with the International programme, household budget surveys (HBS),
Comparison Programme (ICP). Section 5 of the national accounts, government finance statistics,
Appendix, in particular, lists the benefits of CPI/ICP and to more general aspects of statistical
integration including reduced costs and the prospect infrastructure. For these reasons the ICP is regarded
of better methodologies leading to better data and as an important catalyst for sustainable and long‐
improved statistical estimates, particularly of term national statistical capacity‐building. This
purchasing power parities. Section 5 also discusses chapter does not discuss the ICP as such, but
two core strategies for CPI/ICP integration: the use describes ways in which national CPIs can benefit
of a price‐determining characteristics approach to from ICP participation, and, to some extent, the
determine what to price rather than pricing very value of well‐constructed CPIs to the ICP. The two
specific and narrowly defined items in each country systems can in fact be of mutual benefit.
and the linking approach, which unlike the price‐
determining characteristics approach does not Spatial versus temporal price indices
require all countries to use the same product
characteristics classification. 17.6 A CPI is a temporal price index; it measures
price changes within a country or region, over time.
17.3 The characteristics approach makes greater
There is a base period, such as a particular month or
use of national CPI prices data but requires a price to
year, which is set to 100. Price levels in subsequent
be attributed to each price‐determining
periods are expressed as index numbers relating to
characteristic. Even with a standard product
the base of 100. So an index of 110 indicates a price
characteristics classification, these can be difficult to
rise of 10% compared with the base period. The
compute, particularly for complex goods, such as
geographic coverage of the index is the same for
computers, which have a number of price‐
each index observation.
determining characteristics whose values are inter‐
dependent. For example, it can be difficult to 17.7 The ICP is a spatial index. It can be compared
compute individual values for processor speed and with a CPI in which geographical locations are
for size of memory in a PC, where new models may substituted for time periods. Thus, a “base country”
include increases in both. Techniques such as can be selected and its average price level set to 100.
hedonics can be used, but they require large data Comparable price levels in other countries are
sources and considerable statistical expertise. calculated, and differences in average prices are
converted to index numbers relating to the base
234
CPI/ICP INTEGRATION AND HARMONISATION
235
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
price the same product. Often countries are unable this, it is necessary to have consumption weights,
to do this and the item priced may deviate in a small usually drawn from HBS data. In many cases,
way from the specification. Where the difference detailed expenditure data may not be available for
relates to the units or quantities of measurement all the regions within a country. For ICP purposes,
the quality adjustment is relatively straightforward the HBS data must be able to provide reliable
as long as the required meta‐data are correctly estimates of national average weights, rather than,
recorded. For more subjective quality differences say, weights representative of consumption in urban
judgments are needed in order to adjust the price areas. For many products the consumption weights
observations. The ICP quality adjustment task (to (after adjustment for scope differences as
assess the value of the difference in specifications) is mentioned below) will be the same for the ICP and
essentially the same task as is needed for CPI quality CPI. But there are some important differences. The
adjustment. ICP definitions must follow national accounts
definitions. Major differences can be found in.
17.15 The ICP process starts with establishing a
product list for each ICP region and sub‐region using • The treatment of own‐account consumption
Structured Product Descriptions (SPDs) (see (e.g. households growing and consuming their
references later on). Countries propose products for own food). This is usually excluded from CPIs but
inclusion in their respective region’s list. For a given must be included in the ICP.
region many products are likely to be similar, but
• The treatment of owner‐occupied housing. The
others may be quite different. The ICP methodology
ICP uses imputed rents, whereas CPIs use a
requires the preparation of a common list of
variety of methods, sometimes even excluding
products which is representative of the countries in
owner‐occupied housing entirely.
the region. The aim is to maximize the overall
representativity of the common list. Countries then
select a subset of the regional list which they agree Integration of CPI and ICP datasets
to price. In selecting the subset, countries aim to
maximize the representativity of the national list. 17.18 The CPI is a vital tool for the management of
The country lists are then reviewed to ensure there an economy, and as such it is usually given a high
are sufficient products being priced in each basic profile and may be relatively well‐resourced. At the
heading59 by each country, to produce reliable data. very least, it can be generally assumed that a
Where necessary countries may be asked to add national CPI will remain a permanent part of a
some less representative products to ensure there statistical office’s work programme. On the other
are sufficient matches across countries60. hand, earlier ICP rounds have been conducted at
infrequent and irregular intervals. Consequently, the
17.16 Seen from the point of view of the
mechanism and the resources needed for effective
participating country, at the end of the process there
ICP participation have had to be re‐started for each
will be an ICP product list which contains (a) as many
new round. This gives rise to organizational
as possible of the products which are representative
problems as generally the expertise gained from
in the country concerned and (b) a number of
earlier ICP work is lost by the time of the next round.
products which may be found and priced in the
country, even though they are not representative. 17.19 Because of the value of the ICP data, and to
The country will also have omitted some additional maximise the organisational benefits for countries,
products that were on the regional list because they there are advantages in establishing an ongoing ICP
were not able to be priced. The omitted set of programme with phased data collection similar to
products should be as small as possible relative to (a) the OECD and European PPP programmes. With this
and (b). How such a composition may be achieved in background, it would clearly be an advantage if the
practice, and how this can help to integrate the two ICP is linked as closely as possible to the regular CPI
systems is discussed later. system. This can be done by maximizing the quantity
of data usable in both systems. In this way, ICP
17.17 Once the prices data are compiled, the next
participation can become less of a burden for
stage in the compilation of a CPI is the aggregation
participating countries, and the quality of both ICP
of the detailed price changes to higher levels. To do
and CPI data can improve. In addition, there are
many potential benefits for the CPI and other parts
59
A basic heading is the smallest aggregate for which expenditure of a statistical office’s work, especially national
data are available. accounts and the HBS. The ICP can be used to
60
improve measures of poverty and income inequality,
http://siteresources.worldbank.org/ICPINT/Resources/Ch5_Produ and to allow meaningful comparisons of economic
ct_Lists_Apr06.doc
data with other countries within and between world
236
CPI/ICP INTEGRATION AND HARMONISATION
regions. The potential benefits of integration are purpose. International standards recommend the
discussed below. use of this classification in the national accounts,
HBSs, in establishing CPI weights, and in other areas.
Integration benefits from ICP 17.24 Some countries that have not yet adopted
participation COICOP for their national CPI and other statistics
have found that the ICP provides the incentive for
17.20 First, it should be borne in mind that a benefit making the change. In any case, the System of
for the ICP can be a benefit also for NSIs. The ICP and National Accounts (SNA 2008), which most countries
CPI have overlapping products. To the extent that have implemented, requires adoption of this
the two data collection operations are integrated classification. This can be done much more easily if it
and duplication is avoided, staff will be at least is applied consistently across the national statistical
partially relieved of the extra workload. system; for CPI weights, for HBS data, for ICP, for
national accounts and for government finance
17.21 It has been a long‐standing requirement of statistics.
ICP participation that countries should be able to
show post‐hoc examples of benefits which the ICP Improved quality of weights
has brought to the CPI and other aspects of their
17.25 The ICP demands high quality weights, in
national statistical systems. However, in previous
respect of both freshness of data and accuracy. The
rounds of the ICP this aspect was to a large extent
ICP Operational Manual has the following remark
ignored, probably due to the overriding needs for
concerning the up‐to‐dateness of weights: “If the
the provision of data for the ICP itself and the lack of
[most recent HBS] is more than 10 years old, you will
funds for the necessary follow‐up work. The 2005
need to make it up‐to‐date by adjustments to survey
round of the ICP enabled some follow‐up work to be
data reflecting changes in demography . . . . . or
done.
expenditure patterns”. The ILO Resolution on CPI
17.22 What aspects of a CPI can benefit from the states that “weights should be reviewed and if
experience of ICP participation? The answer to this appropriate revised as often as accurate and reliable
question will depend to some extent on national data are available for this to be done, but at least
circumstances and in particular on the state of once every five years.”
development of the CPI itself. A list of areas of
17.26 Although the cost of conducting an HBS is
potential improvements is given below, and is
high, having reasonably up‐to‐date results is more
discussed in the following paragraphs.
likely to give accurate weights than making ad hoc
• More efficient use of classifications. adjustments based on incomplete information about
changing patterns of expenditure. Countries should
• Improved quality of weights. also bear in mind that with a Laspeyres‐type CPI they
• Wider geographic coverage. can reduce upward bias to the CPI if the weights are
updated more frequently. This is because the results
• Improved outlet coverage. of substitution to lower‐priced products are more
• Improved outlet selection. quickly reflected in the CPI.
• Aligned CPI/ICP product lists. Wider geographic coverage
• Improved product specifications. 17.27 The ICP and the CPI have different needs for
geographic coverage. The ICP attempts to measure
• Better quality adjustment. national average price levels at a particular point in
• More efficient price collection. time, whereas the CPI measures price trends over
time. If time trends in different parts of a country are
• More efficient data processing. fairly similar, a limited geographic coverage, if
• Improved documentation. correctly designed, can provide a fair estimate of
national price trends in the CPI. But price levels may
• Better staff training. differ according to region and between urban and
• Reduced costs. rural locations. Furthermore, these relationships
may differ across commodity groups61. During times
More efficient use of classifications of major structural change, significant changes in
17.23 The ICP product list is based on COICOP, the 61
international classification designed to classify For example domestic fresh produce may be cheaper in rural
areas than in the cities but imported goods are likely to be
individual consumption expenditure according to cheaper in port cities than in distant regions or in rural areas.
237
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
regional and urban/rural price differentials can also the CPI and ICP lists to be maximised, thereby
result. The ICP thus demands coverage which reducing the burden of extra price collection on NSIs
reflects the average level of prices in the entire in future ICP surveys.
country. In some countries this implies extending the
17.31 Increasing product list alignment can be
geographic coverage of price collection. At the same
achieved in several ways. Firstly, most countries
time, some countries are concerned that their CPI
have products in their CPI list that have not been
coverage fails to reflect national trends, in particular
reviewed for a long time. By reviewing both lists, old
regarding differences in trends between urban and
products in the CPI list can be replaced by more
rural locations. Furthermore, some countries
relevant products from the ICP list. Secondly, it may
produce, or would like to produce, regional CPIs, and
be found that the CPI list is unnecessarily long: close
for this reason a wide geographical coverage is
comparison with the ICP list may suggest possible
needed. Such countries can benefit from ICP
redundant products. Thirdly, a review of the
participation as it shows how regional coverage can
products in the ICP list, especially relatively new
be extended, thus enabling a permanent extension
products such as mobile telephones some years
of coverage for the CPI.
back, may alert CPI statisticians to the need to add
Improved outlet coverage such products to their list. Fourthly, it may be found
that the specifications of a product may differ only
17.28 There are similar arguments here to those slightly between the two lists. As the ICP
regarding geographic coverage discussed above. If specification is often more precise than the CPI
time trends across different outlets are fairly similar, specification, the latter can be replaced by the
limited outlet coverage can provide a fair estimate of former ‐ provided of course that the ICP product is
national price trends in the CPI. However, this representative of national consumption.
condition is unlikely to hold during times of
significant structural change The ICP measures 17.32 Adjustments along all of the four lines
national average prices, thus requiring coverage of described above will automatically lead to an
all main types of outlet. ICP experience can be used increased overlap between the CPI and ICP product
to review CPI outlet‐types. The results of such lists. Experience shows that the degree of overlap of
reviews often indicate that an extension of outlet the CPI and ICP lists can be increased by 25% or
types for the CPI is desirable and achievable. For more. Such an increase considerably eases the
example, it may happen that supermarkets are less burden of work for subsequent ICP rounds without
well covered than market stalls, which could jeopardising the quality of the CPI – in fact CPI
upwardly bias the CPI if the increasing market share quality can be improved as a result of having more
of supermarket sales is not correctly reflected. up‐to‐date and detailed product specifications. The
outcome of such a review is illustrated in Figure
Improved outlet selection 17.1, which uses actual data from a country that
17.29 ICP methods can stimulate countries into re‐ participated in the 2005 ICP round.
thinking how to select individual outlets for 17.33 From the figure it can be seen that in the
sampling, to enable the appropriate coverage of ICP 2005 ICP round it was necessary for this country to
products. To the extent that ICP products are price 500 products over and above the 360 already
partially incorporated in the CPI, the same outlets priced for the CPI – a “burden ratio” of 1.4
can be added to the CPI sample. For example, (500/360). For the next ICP round (assuming no
comparison of the ICP product list with the CPI changes to the ICP list) it would be necessary for only
product list may have led to the conclusion that DVD an additional 400 products, out of a total of 440 in
players, say, are sufficiently widespread to warrant the CPI, to be priced – a burden ratio of only 0.9
inclusion in the CPI. But if the outlets which sell DVD (400/440). It may be noted that part of the increase
players are not well covered in the CPI, there may be in the overlap rate is related to an increase in the
a need to extend the coverage so as to include more size of the CPI product list. But in this case the ICP
of them. The same outlets can be used for ICP and work coincided with a planned review of the CPI
CPI price collection. basket, which was in need of renovation and
extension.
Aligned product lists
17.30 Product list alignment is potentially one of
Improved product specifications
the most important areas to benefit from ICP‐CPI 17.34 The 2005 round of the ICP used a structured
integration. It opens up the prospect of updating the approach to product specification. The “Structured
CPI list to make it more relevant to present day Product Description” (SPD) was introduced to help
purchasing habits. It also allows the overlap between ensure that the products priced across countries
238
CPI/ICP INTEGRATION AND HARMONISATION
Figure 17.1 Effect of improved CPI/ICP alignment
500 200 160 400 300 140
239
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
• Instructions on what to do when outlets are the day’s work. The ICP manuals provide a useful
closed or products are unavailable. model for countries wishing to develop improved
internal CPI documentation.
More efficient data processing
Better staff training
17.39 Integrated ICP‐CPI software for future
processing of both ICP and CPI data, including 17.41 As a consequence of their 2005 ICP
modules for data input, verification and editing, experience, some countries have extended their CPI
calculation, and output can enhance data quality. coverage in terms of regions, products and outlets.
But the practical problems of development, To support these extensions they have strengthened
deployment and maintenance of computer the staffing of the prices division and introduced a
programmes and the resources required should not permanent regime of staff training for head office
be under‐estimated. and regional staff, including price collectors. Training
sessions used materials developed for the 2005 ICP
Improved documentation round and built on the training given to staff working
17.40 The 2005 ICP round achieved major advances on the ICP.
in planning and management. Among its successes
Cost Reduction for both ICP and CPI data
were a new set of manuals covering the ICP work at
all levels from the theoretical to detailed collection operations
organizational issues. While these were written for 17.42 Costs can be cut by integrating the two data
the purposes of the ICP, national CPI practitioners at collections. Savings can be made in areas such as the
all levels have found much of the material of benefit use of staff and equipment, using a shared outlet
to their work on the CPI. For example, the ICP sampling frame, training, data editing and validation.
Operational Manual includes sections written Shared outlets offer savings in terms of transport,
especially for local price collectors, including such time and taking advantage of the benefits of good
detail as the advance planning for the daily route, contacts with outlets. As savings are achieved,
tasks to do at the start and end of the day, and a list statistical offices are encouraged to further integrate
of the items needed to be taken by the collector for CPI/ICP work into other areas of statistical activity.
240
CHAPTER 18
Quality Reporting and Improving the CPI:
Frameworks, Checklists and Work
Programmes
Introduction for OECD Statistical Activities and Eurostat’s Quality
Framework for European Statistics. There is also the
18.1 Paragraphs 81 to 85 of the 2003 ILO IMF Data Quality Assessment Framework. This
Resolution make reference to the need to respect provides a flexible structure for the qualitative
both the “UN Fundamental Principles of Official assessment of a CPI which can be used in a variety of
Statistics” and the “ILO Guidelines concerning contexts, including self‐assessments performed by
dissemination practices for labour statistics”. Both national statistical offices, central banks, and other
address issues of integrity, engagement with users data producing agencies.
and the principles underlying production and 18.4 The focus of this chapter is on the use of
dissemination. The UN principles are given at these and similar frameworks to provide users with
Appendix 18.1. The ILO guidelines, which were the metadata that they are entitled to on the
endorsed at the Sixteenth International Conference characteristics, quality, access and integrity of the
of Labour Statisticians, 1998, can be accessed on the CPI and, correspondingly, the information that an
ILO website www.ilo.org. The guidelines cover four NSI needs to know and collect to monitor whether it
headings relating to: is fulfilling its international obligations and meeting
the needs of users and to enable it to put in place a
• Access to data, metadata and statistical
programme of improvements. A framework can be
products (access).
augmented by a checklist of compulsory and/or
• The terms and conditions under which statistics desirable characteristics of a CPI. The use of a
are produced and released (integrity). checklist can also be used as an input into a
metadata framework. They should be seen as
• Publication issues including timing, information
complementary.
on coverage, reference periods, and the main
aggregates to be released (data).
A model framework: quality reports
• Regularly updated documentation providing
metadata on definitions, methodology, sources, 18.5 Using one of the internationally recognised
sampling error and other quality indicators, plus frameworks for reporting on the CPI is
advance warning of revisions and estimates of recommended because it has a number of
the impact of discontinuities (quality). advantages:
18.2 The primary focus of the guidelines is labour • It has the authority associated with an
market statistics but they are also relevant to internationally agreed framework and benefits
Consumer Price Indices. from the experience of different countries.
18.3 This chapter of the Handbook provides
• It allows comparisons on a like‐for‐like basis
guidance on reporting mechanisms which provide
with CPIs produced by other countries.
the metadata which are necessary to fulfil the above
obligations of producers of CPIs. In particular, it • It is readily available.
looks at the use of quality frameworks and
• It fulfils the reporting obligations to
checklists. Two such examples of the former are the
international organisations.
International Monetary Fund’s General Data
Dissemination System (GDDS) and Special Data • It provides the basis for reporting to users.
Dissemination Standards (SDDS). Other examples
include the OECD Quality Framework and Guidelines
241
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
• It provides a benchmark for future 18.8 In order to provide a benchmark for future
developments, particularly when carried out in developments, it is important that where standard
conjunction with a checklist. definitions, methodologies and procedures are not
followed, a development plan should be included in
18.6 Such frameworks are generic in nature and
the report or as a supplementary document. This
not specific to CPIs, apart from the Eurostat quality
should include the procedures to be followed to
reporting framework for the European Harmonised
achieve compliance. Thus, these frameworks provide
Index of Consumer Prices (HICP). The latter is based
both guidance to the user on the appropriate uses of
on the IMF’s Special Data Dissemination Standards
the CPI and benchmark information to producers for
(SDDS) specifically adapted for the HICP. Appendix
the formulation of a work programme for CPI
18.2 gives an example of a model “quality” report
maintenance and development.
document based on the reporting framework for the
HICP. Quality is defined as “fitness for use” in terms
of user needs and extends beyond the statistical A model framework: checklists
accuracy of the index to its definition and coverage,
effective dissemination and the transparency of the 18.9 For internal purposes a checklist might also
statistical system. See Appendix 1 of the CPI Manual be useful, which would generally be more specific
for further information on the EU HICP. than a document design for public dissemination and
would constitute part of a more detailed evaluation
18.7 The general principles underlying the and corresponding development programme for
publication of “quality” reports on the CPI, as with all operational use. Appendix 18.3 gives an example of
official statistics, are that: a model checklist based on compliance with the
• The reports should be easy to access and use by 2003 ILO Resolution on CPIs. Similar checklists could
all interested parties. be used for compliance with national standards or
aspirations and with plans to develop internationally
• The contents should be sufficiently detailed to harmonised consumer price indices.
allow users to assess fitness for particular
purposes. Qualitative (and where possible
quantitative) measures of quality should be
Planning: work programmes
included to help users to understand better the
strengths and the limitations of the CPI and 18.10 Quality reports and checklists should be fully
associated series and the corresponding integrated into the planning mechanisms for a
implications for interpretation and appropriate corresponding CPI statistical work programme. The
use. latter is an essential part of the process for the
continued development of a CPI and for fulfilling the
• Quality measures and detailed technical delivery function of the NSI. It is, itself, part of an
information should be supplemented by over‐arching system of programming, planning and
guidance on interpretation to help users assess reporting.
fitness for purpose.
18.11 The general principles underlying a
• Clear statements should be given on the degree programming, planning and reporting system are
of compliance with agreed definitions, methods that:
and practice ‐ including both those determined
nationally and those laid down in the 2003 ILO • There should be clear and transparent
Resolution on CPIs ‐ and any known reasons for governance arrangements relating to:
deviations. o The allocation of responsibilities for
monitoring and reporting on the production
• Where possible, the presentation of information
and dissemination of the CPI and on its
on quality will be tailored to meet the needs of
development.
different types of users, with more
comprehensive information being prepared for o The setting of protocols relating to the
expert users. This may indicate separate quality scope and definition of the CPI and the
reports directed at different user groups. methodological detail which supports the
latter.
• Producers should systematically review at
regular intervals the documentation relating to o Putting in place and managing the day‐to‐
the CPI and update it to reflect up‐to‐date day operational arrangements.
methods and processes.
242
QUALITY REPORTING AND IMPROVING THE CPI
Figure 18.1 Programming, Planning and Reporting process
Plan
Define objectives.
Agree outputs.
Allocate resources and
responsibilities.
Produce detailed project
plan.
Select Control
Programming - Select Monitoring. What are
Projects you doing to ensure that
Project prioritisation tool. the projects will deliver
the outputs and benefits
projected?
Evaluate
Based on your
evaluation, did the
project deliver what you
expected?
Process Dynamic Lessons learned
Information Flow
243
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
244
QUALITY REPORTING AND IMPROVING THE CPI
245
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 18.1 Fundamental Principles of Official Statistics, Endorsed by the
United Nations Statistical Commission, 1994
Principle 1. Official statistics provide an indispensable element in the information system of a society, serving
the government, the economy and the public with data about the economic, demographic, social and
environmental situation. To this end, official statistics that meet the test of practical utility are to be
compiled and made available on an impartial basis by official statistical agencies to honour citizens’
entitlements to public information.
Principle 2. To retain trust in official statistics, the statistical agencies need to decide according to strictly
professional considerations, including scientific principles and professional ethics, on the methods and
procedures for the collection, processing, storage and presentation of statistical data.
Principle 3. To facilitate a correct interpretation of the data, the statistical agencies are to present
information according to scientific standards on the sources, methods and procedures of the statistics.
Principle 4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of
statistics.
Principle 5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys
or administrative records. Statistical agencies are to choose the sources with regard to quality, timeliness,
costs and the burden on respondents.
Principle 6. Individual data collected by statistical agencies for statistical compilation, whether they refer to
natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.
Principle 7. The laws, regulations and measures under which the statistical systems operate are to be made
public.
Principle 8. Co‐ordination among statistical agencies within countries is essential to achieve consistency in
the statistical system.
Principle 9. The use by statistical agencies in each country of statistical concepts, classifications and methods
promotes the consistency and efficiency of statistical systems at all official levels.
Principle 10. Bilateral and multilateral co‐operation in statistics contributes to the improvement of systems
of official statistics in all countries.
The UN Fundamental Principles of Official Statistics with more details and explanations are available online
on www.unstats.un.org/unsd/default.htm.
246
QUALITY REPORTING AND IMPROVING THE CPI
Appendix 18.2 Model Quality Report Document for the CPI
SUB‐HEADINGS: SPECIFIC ILLUSTRATIVE RANGE OF ISSUES WHICH
TOPICS FOR INCLUSION MIGHT BE COVERED
Coverage Population (e.g. treatment of
institutional households, rich/poor
households)
Basket of goods and services (any
exclusions e.g. owner‐occupied housing,
informal markets)
Expenditure (e.g. final household
DATA
monetary consumption, treatment of
foreign visitors, expenditure of resident
population abroad, own‐account
production)
Periodicity Weekly (all or some prices only)
Monthly (all or some prices only)
Quarterly (all or some prices only)
Point in time or spread over month
Timeliness Time‐lag between price collection and index
publication
Public access Pre‐announcement of publication date.
ACCESS
Simultaneous release to all.
Dissemination format Electronic/paper.
Pre‐release access Protocols on any pre‐release arrangements.
Transparency Protocols on compilation and dissemination
of CPI are published and readily accessible.
INTEGRITY
Protocols comply with UN Fundamental
Principles.
Revisions Statement of revisions policy, revisions
clearly marked.
Advance notice given of methodological
changes, numerical impact given.
Publication of information Dissemination of documentation on
QUALITY
needed by users to asses quality methodology.
Confidence intervals calculated and
disseminated with other information on
quality/accuracy of CPI.
247
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Appendix 18.3 Model Checklist based on compliance with the 2003 ILO
Resolution on Consumer Price Indices
ACTION PLAN
where
COMPLIANCE appropriate
STATEMENT (including
(with supporting deliverables and
Y N evidence) timetable)
a. Alternative price indices available for specific
1. USES
purposes (if extra expense justified).
b. Alternative price indices appropriately defined
and named.
c. Where only one index, definition determined
by main use.
a. Target population relevant to use
2. SCOPE
b. Population excluded is explicitly stated
c. If geographical scope is restricted due to
resources, this is explicitly stated.
d. Resident or domestic consumption.
e. Index covers all types of consumer goods and
services relevant to the reference population.
f. No omission of illegal or socially undesirable
goods or services.
g. Goods and services purchased for business
purposes or expenditures on assets excluded
from the index
a. Classification system used – COICOP or a
3. BASKET / WEIGHTS
reconcilable version.
b. Classification system used for index
compilation also used for household expenditure
statistics.
c. Classification provides the framework for the
allocation of expenditure weights.
d Weights derived from Household Budget
Surveys and National Accounts estimates of
household consumption expenditure.
e. If weight reference period differs from the
price reference period weights are price updated.
f. Weights revised at least once every 5 years.
g. Continuous CPI series created by chaining
when new basket replaces the old.
h. New models/varieties of existing product
introduced once significant.
i. Seasonal products included in the basket (fixed
or variable weights used).
248
QUALITY REPORTING AND IMPROVING THE CPI
a. Probability sampling used for item sampling if
4. SAMPLING
resources available.
b. If resources not available for probability
sampling, judgemental sampling by statistician.
c. Sampling frames comprehensive and up‐to‐
date
d. Sample of outlets and of items reviewed
periodically and updated where necessary to
maintain representativity.
a. The CPI is calculated starting with elementary
5. INDEX
CALCULATION
aggregate indices.
b. Higher level indices are calculated by
aggregating the elementary aggregates using
expenditure weights.
a. Geometric Mean used for elementary
6. ELEMENTARTY
AGGREGATES
aggregate indices.
b. Ratio of Averages only used for elementary
aggregates which are homogeneous and where
there is little substitution.
c. Average of Relatives not used.
b. Elementary index computed using chained
form of the formula.
a. Laspeyres‐type index (current prices, historic
7. UPPER LEVEL
INDICES
weights) used.
b. Other index types for specific purposes?
a. Standard methods for collecting and
8. PRICE OBSERVATIONS
processing price information used.
b. Prices collected systematically and accurately
at regular intervals.
c. Price collectors trained
d. Price collectors supervised
e. Manual available for price collectors
f. Same product priced in each period
249
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
a. Index relates to either monthly average prices
9. COLLECTION
or prices for a specific period of time within the
month.
b. For perishable goods prices are collected at the
same times of day and not just before closing
time.
c. Price collection representative of all
geographical areas within the scope of the index.
d. Prices collected in all type of outlets that are
relevant, including open‐air markets, informal
markets, and internet sellers.
e. Specifications provided detailing the variety
and size of the items for which prices are to be
collected.
f. Prices collected are actual transaction prices,
including indirect taxes and non‐conditional
discounts.
g. Exceptional prices charged for stale, shop‐
soiled, damaged or otherwise imperfect goods
are excluded unless permanent and significant
feature of retail market.
h. Genuine sale prices included.
i. Subsidised and controlled prices and
unrestricted prices covered.
j. Alternative means of collection used for
different items where appropriate, i.e. outlet
visits, brochures, telephone, scanner data,
internet.
k. Collected price information reviewed for
comparability and consistency with previous
observations, the presence of replacements,
unusual or large price changes and to ensure
price conversions of goods priced in multiple
units are correct.
l. Consistent procedures established for dealing
with missing price observations.
m. Estimated prices included for non‐seasonal
products that are temporarily unavailable.
a. Items replaced when they disappear
10. REPLACEMENTS
permanently.
b. Replacement made within the first three
months of the item becoming unavailable.
c. Replacement selected from available criteria
based on: the most similar item; the most
popular variety; the most likely to be available in
the future (state which).
d. Outlets replaced if price cannot be obtained.
250
QUALITY REPORTING AND IMPROVING THE CPI
e. Criteria in place for the selection of a
replacement outlet and includes outlet type and
location.
a. Relevant characteristics of items are recorded
11. QUALITY
CHANGES
on an ongoing basis to ensure changes in quality
are identified.
b. An adjustment is made to the price when a
quality change is detected – the methods used to
adjust are based on international guidelines.
a. Possible sources of errors actively identified
12. ACCURACY
and quantified.
b. Impact of possible error minimised in the
design, construction and compilation of the
index.
a. CPI compiled and released monthly.
13. DISSEMINATION
b. Timetable for publication pre‐announced.
c. CPI released as quickly as possible‐ within X
weeks of the reference period.
d. CPI released to all users at the same time
e. CPI release accompanied by comments,
interpretation and a short methodological
explanation.
f. Rules relating to release made publicly
available – including who has pre‐release access
(and details of this arrangement).
g. Corrections made and published if necessary
i.e. if estimates have been seriously distorted by
errors in compilation.
h. Planned revisions pre‐announced and meta‐
data provided.
i. Publication should show the index level from
the index reference period.
j. Indices for the major expenditure groups
compiled and released.
k. National CPI results reported to the
International Labour Office shortly after release.
a. The compiling agency has the professional
14.CONSULTATIONS AND
INTEGRITY
independence, competence and resources to
support the CPI programme.
b. The agency responsible consults users on
issues of importance to the CPI, i.e.
methodological changes.
c. A full description of the data collection
procedures and methodology is widely available.
d. Users informed in advance of changes to
scope, weights or methodology.
251
Appendix 18.4 Project Prioritisation Tool
Project/ Data Score
Ref Description Frequency Customers Cost Income Impact Legal Savings Strategy Capacity Comments Effective Efficient
Output Sources (%)
252
Headline Fuller How often is List the Who is the What does the What income Impact on key Extent of Savings How well How well does
description description of the output surveys customer / user product / service does it generate? stakeholders legislative, elsewhere does this fit this use NSI
the output, produced or (acronym and and what do they cost per year? and customers regulatory or as a result with future core skills and
product, does the no.) and other use it for? / users? How contractual of this NSI develop/ build
service or activity take sources used Identify internal important / obligation? project strategy future
activity? place? to compile the customers as well valuable do capacity for
outputs as final external they think it dealing with Including
customer. is? new comments
challenges? on scope R/A/G R/A/G
M ‐ Monthly 0 minimal(<$10k) minimal(<$10k) minimal not at all none poorly poorly for making
Q ‐ Quarterly 1 low($10k‐$25k) low($10k‐$25k) low little low not v well not v well savings
A ‐Annual 2 med($25k‐$50k) med($25k‐$50k) medium medium medium medium medium
or list if none 3 high($50k‐$100k) high($50k‐100k) high strong high well well
of the above 4 v high($100k+) v high($100k+) v high non‐negotiable v high v well v well
MAX
2 Indices for
To inform pay
regional Expanded
Regional settlements
administration Regional
Price Q price 4 0 1 0 0 0 0 17% for RED RED
s and administration
Indices indices government
introducing at
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
workers
chain link
Separate sub‐
3 indices for
goods and
Separate
services Ministry of Required to
Price
published CPI prices enact
indices for M Finance and 0 4 4 4 4 4 4 89% GREEN GREEN
monthly from data legislation on
goods & Central Bank
a re‐analysis price control
services
and re‐
weighting of
prices data.
Score: The ‘score’ column represents the sum of the value times the weight for each of the components divided by the maximum possible score (i.e. sum of all weights times 4). It can be used as a quick
indication of which projects may provide the greatest/least value for money and where further discussions need to be focussed.
Effective: This column is used to get an indication of whether the project would be (or is) effective, i.e. whether its value to the user outweighs its cost. The column will display RED (not effective), AMBER
(effective to some extent) or GREEN (highly effective), and is reached by the following process (the workings are not shown in the above table):
1. Calculations are made to reach a value for ‘net cost’ (cost minus income) and ‘overall weighted value’ (sum of impact x weight, legal x weight, strategy x weight and capacity x weight, divided by
10).
2. ‘Net cost’ is then assessed as High (>=2) given a score of 3, Medium (1, 0, ‐1) given a score of 2 or Low (<=‐2) given a score of 1 and the overall weighted value as High (>=9) given a score of 3,
Medium (5, 6, 7, 8) given a score of 2 or Low (<= 4) given a score of 1.
3. ‘Effectiveness’ is calculated as the result of overall weighted score (3, 2 or 1) minus net cost score (3, 2 or 1).
4. If the result of this formula is >=2 the effective rating is GREEN, if the result is 1, 0, or ‐1 the effective rating is AMBER, if the result is <=‐2 the effective rating is RED.
Efficient: This column is used to get an indication of whether the project would be efficient, i.e. whether the cost is outweighed by the potential savings or income. The process for calculating this column is
in essence the same as for the ‘effective’ column. Efficiency is calculated as the result of the savings score minus the net cost score. At step 2 ‘savings’ assessed as Very High are given a score of 4, High a
score of 3, Medium a score of 2 , Low a score of 1 and Minimal a score of 0. If the result of the calculation is >=2 the efficient rating is GREEN, if the result ‐1, 0 or 1 the efficient rating is AMBER, if the result is
<=‐2 the efficient rating is RED.
QUALITY REPORTING AND IMPROVING THE CPI
Appendix 18.5 Managing Risks in a Project
The management of risk is a major aspect of project management. If a project is to be a success, risks that
the project objectives might not be met must be controlled and contained by actions designed to reduce the
chances of problems arising and to alleviate their potential impact.
There are several stages in the management of risk:
Identification: Risks can be identified through a number of mechanisms but an initial ‘horizon scan’ for
risks is a useful tool. Once identified risks should be documented in a ‘risk log’. For instance, this
might include the risks that a new CPI computer programme might not deliver the software required
for planned methodological improvements to the index, or might not facilitate more efficient editing
of prices data and a more‐timely index.
Evaluation: A risk should be assessed as to its likelihood (the probability of the risk becoming a reality)
and impact (how critical is the result of a particular “risk” outcome actually happening). It is useful to
score likelihood and impact to calculate an overall ‘exposure’, see Figure 18.5.1. This can help when
deciding on what action to take in the next stage. For instance, if it is assessed during the course of a
project that new CPI software is likely to lead to a less timely CPI which will not meet statutory
requirements, this could warrant a top score of 30 and special action to be taken.
Respond: A decision must be made on what action to take, this can be to discount the risk (typically if the
‘exposure’ is low), to introduce countermeasures to stop the risk, or take action to control the
impact of the risk if it occurs.
Monitoring: Risks that have been identified must be monitored to watch for signs that the risk is
changing its status and to ensure that agreed responses or corrective actions are being applied. For
instance, if it has been agreed that the module in the new CPI programme should be re‐designed to
facilitate more efficient automated data editing for increased speed, then checks need to be carried
out that this has happened and that the re‐design has achieved the desired objective.
All of the above stages are ongoing throughout the course of a project.
Figure 18.5.1 Risk Scoring Method
Exposure
Unlikely 2 2 4 6 8 12
Rare 1 1 2 3 4 6
1 2 3 4 6
Insignificant
Minor
Moderate
Major
Disastrous
Impact
253
Glossary of Main Terms
Note: Words in boldface have their own entry in the glossary
Acquisition approach In CPI construction relates to the time period in which products are acquired by a
household (as opposed to when they are wholly or partially consumed),
regardless of when the products are paid for. (See also Payment approach and
Uses approach).
Ad valorem tax A tax expressed as a percentage of the value of a product, as opposed to a flat‐
rate tax.
Aggregate A set of transactions (or their total value) such as the total purchases of
households on goods and services in a certain period.
Aggregation The process by which CPIs for lower‐level aggregates are averaged, or otherwise
combined, to obtain CPIs for higher‐level aggregates.
All‐items index The highest level of aggregation of a CPI which covers all the items within the
scope of the CPI.
Average of relatives (AR) See Carli price index.
Axiomatic (test) The axiomatic or test approach to index number theory attempts to determine
approach the most appropriate formula by reference to specific axioms or tests that the
index should ideally satisfy.
Bargaining A selling procedure where the transaction price is negotiated personally between
buyer and seller. The final price and quantity are not known until the purchase
has been made. Also known as “haggling”.
Base period The period with which all other periods are compared. See also “price reference
period”, “weights reference period” and “index reference period”.
Basket A specified set of quantities of goods and services. In the CPI context, the set may
comprise the actual quantities acquired or used by households in some period, or
may be made up of hypothetical quantities.
Bias A systematic tendency for the calculated CPI to diverge from some ideal or
preferred index, resulting from the method of data collection or processing, the
index formulae used or some other aspect of index construction such as the
treatment of changes in quality.
Black market See “Parallel market”.
Branded/unbranded Goods which carry a label indicating the maker or distributor. The term is
goods sometimes used to denote products whose maker or distributor is nationally or
internationally well known. Goods which are not “branded” are called
“unbranded”.
Carli index An elementary price index defined as a simple, or un‐weighted, arithmetic
average of the sample price relatives.
Carry forward A situation in which a missing price in some period is imputed as being equal to
the last price observed for that item.
254
GLOSSARY
Central collection Collection of prices directly from NSI Headquarters. These usually concern
products whose prices are determined by national organisations or government
(e.g. fuel and transport charges) or by head offices of large retail chains.
Central product An international classification of goods and services based on the physical
classification (CPC) characteristics of goods or on the nature of the services rendered.
Chain index An index number series for a long sequence of periods obtained by linking
together index numbers spanning shorter sequences of periods. (See also
“chaining”).
Chaining/chain linking The construction of a continuous price series by multiplying together price indices
that have been constructed using different weight reference periods. The
resulting index is referred to as a “chain index”.
Characteristics The tangible or intangible attributes of a good or service which serve to identify it
and enable it to be classified. Some characteristics will help determine price and
are commonly referred to as price‐determining characteristics.
Class mean imputation Imputation of a missing price by reference to the average price change for the
price of a comparable item, usually from a similar outlet.
Classification of products A procedure in which individual items of goods and services are organized into
categories based on characteristics inherent to the items.
Cobb‐Douglas function A model of consumer behaviour in which consumers are assumed to vary the
quantities of products consumed in inverse proportion to the changes in relative
prices.
COGI (Cost‐of‐goods An index in which a fixed basket of goods and services is priced each period,
index) calculated as the cost of the basket in the comparison period divided by its cost in
the reference period.
COICOP The international Classification of Individual Consumption according to Purpose.
It is required for use in national accounts complying with the System of National
Accounts (SNA 2008) and is widely used for CPIs, HBSs, and in the ICP.
Consumer Price Index A monthly or quarterly price index compiled and published by a National
(CPI) Statistical Institute that measures changes in the prices of consumption goods
and services acquired or used by households.
Consumers Individual persons or groups of persons living together as households.
Consumption of own Goods or services that are consumed by the same household that produces
production them.
Commodity Flow A technique based on analysis of various national accounts aggregates designed
method to produce reliable estimates of expenditure weights where direct estimates are
lacking or unreliable. See product balance.
Compensation Index A CPI designed to be used primarily as a measure of inflation faced by employees,
for informing wage negotiations or for indexation of wages.
Constant‐tax index A type of CPI which excludes price changes which are directly due to changes in
indirect taxation.
Core inflation A type of CPI which shows the underlying trend in inflation by excluding those
movements which arise from transient factors such as seasonality or other short‐
term price volatility. Also known as “underlying” inflation.
255
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Cost of living index An index which measures the change between two periods in the minimum
(COLI) expenditures that would be incurred by a utility‐maximising consumer whose
preferences or tastes remain unchanged, in order to maintain a given level of
utility, standard of living or welfare.
Coverage The set of goods and services of which the prices are actually included in a CPI.
(See also “Scope”).
Current value The actual value of some aggregate in the period in question: the quantities
purchased in the period multiplied by the prices in the same period.
CPI Manual The international Consumer Price Index Manual published by the ILO in 2004
(ISBN 92‐2‐113699‐X).
Cross‐border shopping Describes the situation where individual consumers travel to a neighbouring
country in order to benefit from lower prices or the availability of different
products from those obtainable in their domestic market.
Customer The usual description in the Handbook of a consumer as an individual purchaser.
Data editing The process of adjusting or eliminating price quotations input to the CPI
calculation process. (See also data validation).
Data validation The process of checking to ensure that all inputs to the CPI calculation process
(especially prices), are correct. (See also data editing).
Deflation The division of the current value of some aggregate by a price index (“deflator”)
in order to revalue its quantities at the prices of the reference period. The word is
also used to describe the situation when the CPI trend is downwards (as
compared with “inflation” when the trend is upwards).
Democratic index A form of CPI in which each household is given equal weight in the calculation of
the index, irrespective of the size of its expenditures. (see also “plutocratic
index”).
Disaster Recovery Plan A contingency plan designed to ensure that the CPI can continue to be calculated
and published when major unforeseen events occur.
Discount A deduction from the list or advertised price of a product that is available to
specific customers under specific conditions. Discounts include quantity discounts
(e.g. additional volumes for the same price) as well as price discounts.
Domain An alternative name for the scope of an index.
Domestic coverage The use of weights covering all consumption expenditure on the national
territory, regardless of the nationality or normal residence of the consumer. (See
also National population coverage).
Durable good A good that can be used repeatedly or continuously for purposes of consumption
over a long period of time, typically several years.
Dutot index An elementary price index defined as the ratio of the un‐weighted arithmetic
average of the prices in the two periods compared.
Elasticity of substitution A measure of the extent to which one product is substituted for another in
response to relative price changes. (See also Substitutability). A zero elasticity of
substitution means that the quantities purchased will not change with price.
256
GLOSSARY
Elementary aggregate The smallest aggregate for which expenditure data are normally available and
used for CPI calculation to compute an elementary price index. The consumption
expenditures associated with the elementary aggregates are used to weight the
elementary price indices to obtain higher level indices.
Elementary price index A price index for an elementary aggregate.
Evolutionary product A product which is “new” in the sense that its features and “quality” differ from
its predecessor. Examples of evolutionary products would be new models of
household appliances such as refrigerators.
Expenditure weights See “Weights”.
Explicit quality A direct estimate of how much of the change in the price of a product is
adjustment attributable to changes in its physical or economic characteristics. (See also
“Implicit quality adjustment”).
Fashion change which is Changed specifications of products such as clothing or cars which may be
not quality change considered as unchanged (and hence comparable) if they have no material effect
on purpose or utility to the consumer, e.g. a change from one fashionable colour
to another.
Fisher price index The geometric average of the Laspeyres price index and the Paasche price index.
It is a symmetric and superlative index.
Fixed basket index A CPI which uses the same basket in each time period, normally consisting of the
total quantities consumed by households over a period of a year.
Fixed weight index A series of weighted arithmetic averages of price relatives that all use the same
(constant) weights.
Goods Physical objects for which a demand exists, over which ownership rights can be
established, and for which ownership can be transferred between units by
engaging in transactions on the market. (See also “services”)
Haggling See “bargaining”.
Hedonic method A regression model in which market prices of different products are expressed as
a function of their characteristics. The resulting estimates may be used to predict
the price of a new product for which the mix of characteristics differs from those
of any product already on the market, and can thus be used to estimate the
effects of quality change on prices.
Heterogeneity See “Homogeneity”.
HICP The European Union Harmonized Index of Consumer Prices, developed to ensure
comparability of CPIs among members of the EU.
Higher level index An aggregate price index as distinct from an elementary price index.
Homogeneity The similarity of products in a group in terms of price (both levels and trends),
purpose, price‐determining characteristics and substitutability. The opposite is
“heterogeneity”.
Household Either an individual person living alone or a group of persons living together who
make common provision for food and other essentials for living.
Household Budget A sample survey of households in which households are asked to provide data on
Survey (HBS) the amounts they spend on consumption products over a given period or periods
of time.
257
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Household consumption Expenditure on final consumption goods and services incurred by individual
expenditure households on their own behalf. It excludes expenditure by governments and
non‐profit institutions on goods and services provided to households as free
social transfers in kind.
ICP The International Comparison Program for purchasing power parities, designed
to produce PPPs enabling international economic comparisons to be made
without the distorting effect of differing price levels and consumption patterns.
ILO Resolution In this Handbook, it refers to the ILO’s Resolution concerning Consumer Price
Indices adopted by the 17th International Conference of Labour Statisticians,
2003.
Implicit quality Inferring indirectly the change in the quality of a product whose characteristics
adjustment change over time by estimating (or assuming) the pure price change that has
occurred. (See also “Explicit quality adjustment”).
Imputed price The price assigned to an item for which the price is missing in a particular period.
The same term is also applied to the price of an item that is not sold on the
market, such as a good produced for own consumption.
Imputed rent The rent which an owner‐occupied dwelling may be expected to fetch if it were
let on the open market. Used as an estimate of the shelter costs of owner‐
occupiers.
Imputed transaction A situation where, in order to impute a price, the transaction itself has to be an
imaginary one. An example is the valuation of own account consumption.
Indexation The periodic adjustment of the money values of some regular scheduled
payments (such as wages, pensions, rents etc.) based on the movement of the CPI
or other price index.
Index reference period The period for which the value of the CPI is set at 100.
Institutional unit A national accounts concept defined as an economic entity that is capable, in its
own right, of owning assets, incurring liabilities and engaging in economic
activities and transactions with other entities.
Item An individual, specified good or service in the sample of products selected for
pricing.
Jevons price index An elementary price index defined as the un‐weighted geometric average of the
sample price relatives.
Laspeyres price index A basket index in which the basket is composed of the actual quantities of goods
and services in the earlier of two periods to be compared (the price reference
period).
Linking Connecting together two consecutive sequences of price observations or price
indices which overlap in one or more periods.
Lowe price index A CPI that measures the proportionate change in the total value of a specified
basket of goods and services between the two periods being compared. The
basket does not necessarily have to consist of the actual quantities in some
period.
Lower level index An elementary price index as distinct from an aggregate or higher level index.
Matched pairs See “matched products or models”.
258
GLOSSARY
Matched products or The practice of pricing exactly the same item in two or more consecutive periods.
models It is designed to ensure that the observed price changes are not affected by
quality change. The change in price between two perfectly matched products is
described as a “pure price change”.
Matched samples A variation of the “matched pairs” concept in tariff pricing, when a full tariff list
or an element of the tariff structure is treated as a “product specification” and re‐
priced in subsequent periods.
Micro‐index/micro‐class An index/class relating to groupings below the sub‐index level
National population The use of weights covering the expenditure of national residents of a country,
coverage regardless of where the expenditure is made (see also “Domestic population
coverage”).
National Statistics Title used in this Handbook to describe the statistical agency which is responsible
Institute (NSI) for compiling the official CPI (and usually other national statistics) of a country.
New product A “new product” may be of a revolutionary or evolutionary type. It may also be a
product which has been introduced into a country’s CPI because it has become
widely consumed.
Non‐probability A sample design based on the non‐random, selection of a sample of outlets or
sampling products on the basis of the knowledge or judgment of the person responsible.
(Also known as “purposive” or “judgmental” sampling).
Option costing A method of quality adjustment, used particularly for durables such as motor
cars, in which the prices of specific options (such as air‐conditioning) are used to
enable price comparisons of hypothetical models which have identical
specifications.
Outlet The interface between a supplier of products and the consumer. It may be a
shop, a market stall, a catalogue, a website etc. Also referred to as a “retail
outlet”, although it can include wholesale outlets which also sell directly to the
consumer.
Outlier A term used to describe any extreme value in a set of data, such as a price or
price relative that requires further investigation or has been verified as correct.
Overall mean imputation Imputation of a missing price by reference to the average price change for the
prices available in the elementary aggregate to which the missing item belongs.
Overlap pricing A method of quality adjustment based on the difference in price between the old
product and its replacement when both can be priced simultaneously. Also
referred to as the “bridge overlap” method.
Own account Goods produced by households for their own consumption.
production/consumption
Owner‐occupied housing Dwellings owned by the households who live in them.
Paasche price index A basket index in which the basket is composed of the actual quantities of goods
and services in the later of the two periods compared. The later period serves as
the weight reference period and the earlier period as the price reference period.
Parallel market Unofficial (and often illegal) markets where goods can be obtained at lower than
usual prices, or where goods in short supply can be obtained at higher than usual
prices. Also known as “Black market”.
259
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Payments approach In CPI construction relates to the time period in which actual payments for a
product are made by the consumer. (See also “Acquisition approach” and “Uses
approach”).
Plutocratic index A form of CPI in which the weights are based on total aggregated expenditure
values rather than average household expenditure proportions. (See also
“Democratic index”).
Price reference period The period for which the prices appear in the denominator of the price relatives.
(See also “Base period”).
Price relative The ratio of the price of an item in one period to the price of that same item in
some other period.
Price updating A procedure whereby the quantities in an earlier period are re‐valued at the
prices of a later period, using components of the CPI.
Probability proportional A sampling procedure whereby each unit in the universe of those units has a
to size sampling (PPS) probability of selection proportional to the size of some known variable, such as
the value of the sales of an outlet.
Probability sampling The random selection of a sample of units, such as outlets or products, in such a
way that each unit in the universe has a known (non‐zero) probability of
selection.
Product A generic term used to mean a good or a service. Individual sampled products
selected for pricing are often described as “items”.
Product balance SNA 2008 uses the term product balance rather than commodity flow and
defines it as follows, “the product balance for any product recognizes that the
sum of output at basic prices plus imports plus trade and transport margins plus
taxes on products less subsidies on products is equal to the sum of intermediate
consumption, final consumption and capital formation, all expressed at
purchasers’ prices, plus exports.”
Purchaser’s price The amount payable by the purchaser to acquire a good or service. It includes any
charges incurred in order to take delivery at the time and place required by the
purchaser.
Purchasing power Changes in the internal purchasing power of a national currency are the inverse of
(internal) changes in the price level: when prices rise, the amount which can be purchased
with a given sum of money falls, and vice versa.
Purchasing power Artificial exchange rates which eliminate the distorting effect of differing price
parities (PPPs) levels and consumption patterns when making international economic
comparisons.
Pure price change The change in the price of a product of which the characteristics are unchanged;
or the change in the price after adjusting for any change in quality.
Quality adjustment An adjustment to the change in price of a product of which the characteristics
change over time, designed to remove the contribution of the change in
characteristics to the observed price change. The adjustment is needed when the
price of a replacement product has to be compared with the price of a replaced
product.
Quantity weights A term sometimes used to describe the quantities in the basket. However,
expenditures rather than quantities act as weights for price relatives.
260
GLOSSARY
Quota sampling Sample selection using judgmental procedures with respect to known
characteristics such as product group or outlet type. The sample is drawn so as to
contain the same proportions as in the total population of products or outlets.
Ratio of averages (RA) See “Dutot price index”.
Ratio of geometric See “Jevons price index”.
means (GM)
Rebasing Rebasing may have different meanings in different contexts:
• Changing the weights used for a series of indices;
• Changing the price reference period used for a series of indices;
• Changing the index reference period for a series of indices, for instance
from 1990=100 to 2005=100.
Rebasing can involve the simultaneous changing of both weights and the price
reference period.
Reference population The set of households included within the scope of the CPI.
Remuneration in kind The regular provision of goods or services in exchange for labour services
rendered, often operating as part of a contract of employment).
Rental equivalence The estimation of the imputed rents which would be payable by owner‐occupiers
on the basis of the market rents payable for accommodation of the same type
and location.
Replacement product A product chosen to replace a product for which prices have been collected
previously, either because the previous product has disappeared altogether or
because it accounts for a diminishing share of the sales of the outlet, or of the
expenditures within the elementary aggregate.
Representative product A product, or category of products, that accounts for a significant proportion of
the total expenditures within an elementary aggregate, and/or for which the
average price change is expected to be close to the average for all products
within the aggregate.
Representativity bias Bias in a basket index which results from the use of quantities that are not
representative of the two periods being compared, i.e. that systematically diverge
from the average quantities consumed in the two periods.
Respondent In the context of retail outlets, the proprietor, manager or shop assistant is
sometimes referred to as the respondent. In the context of Household Budget
Surveys, the respondent is the household or household member who completes
the forms.
Revolutionary product An entirely new product that is expected to satisfy some need in a new way and
is unlikely to fit neatly into an existing CPI item category, such as video recorders
or mobile telephones (when they were first produced). See also “Evolutionary
product”.
Re‐weighting Replacing the weights used in an index by a new set of weights.
Rolling year index An index based on average prices and quantities in moving 12‐month periods
compared with base year averages. Used for dealing with seasonal products.
Rothwell index A type of variable weights method used for dealing with seasonal products. Also
known as the Bean and Stine Type C index.
261
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
Sampled product An individual product that is included in the sample selected for pricing within an
elementary aggregate. Also referred to as an “item”.
Sample rotation A method of keeping the sample up to date by limiting the length of time that
outlets and/or products are included in the price surveys by dropping a
proportion of them (or possibly all of them) after a certain period of time and
selecting a new sample of outlets and/or products.
Sampling frame A list of the units in the universe from which a sample of units can be selected.
The list may contain information about the units, which may be used for PPS
sampling. In a CPI, the units are normally outlet types, outlets and products.
Scanner data Detailed data on sales of consumer goods obtained by scanning the barcodes for
individual items at electronic points of sale in retail outlets. Increasingly used for
hedonic analysis.
Scope The set of products and household types for which a CPI is intended to measure
the price changes. (See also “Coverage”).
Seasonal adjustment The removal of seasonal influences from a CPI, done in order to provide a clearer
picture of the underlying trend in the index. Not to be confused with the
treatment of seasonal products. (See also “Core inflation”.)
Seasonal products Products that are either not available on the market during certain seasons or
periods of the year, or are available throughout the year but with regular and
significant fluctuations in the quantities available or in demand and prices that
are linked to the season or time of the year. (See also “Seasonal adjustment”).
Services Services (as opposed to goods) are products which do not have a physical
presence. A pair of scissors is a good, but the use of them in cutting hair is a
service. Services may be characterised by: perishability – they cannot be stored
up; lack of separability – it is impossible to separate the production and
consumption of a service; uniqueness – the quality of services can differ each
time they are delivered.
Simple random sampling Type of sample selection when outlets or products are sampled with equal
(SRS) probability. It can be used when all the outlets or products are homogeneous.
Specification A description or list of the characteristics that can be used to identify an
individual sample product (item) to be priced. A “tight” specification is a fairly
precise description, intended to narrow the range of items from which a price
collector might choose. A “loose” specification is a generic description of a range
of items that allows the collector some discretion as to which particular item or
model to select for pricing.
Splicing The imputation of an overlap price for a product no longer available in the
current period, by reference to the price changes between the previous and
current periods of similar items. Also known as “linking”, “overall mean
imputation” and “non‐class mean imputation”.
Stratification The division of heterogeneous populations of outlets, regions or products into a
number of relatively homogeneous populations (strata). The group of strata can
then be used as a frame for stratified random sampling.
Structured Product A tool used first in the ICP to standardize the specifications of products for which
Description (SPD) prices are sought, adapted for use with CPI price collection.
Sub‐aggregates Indices produced at levels of aggregation in between the elementary aggregate
level and the all‐items index level.
262
GLOSSARY
Sub‐index A general name used in the context of a CPI which relates to a relatively detailed
product or other stratum level.
Subsistence index A special CPI whose weights relate to the expenditure patterns of the poorest
households.
Substitutability The extent to which consumers substitute one product for another in response to
relative price changes. (See also “Elasticity of substitution”).
Substitute An item or product of which the characteristics are similar to those of another
item or product and that can be used to meet the same kinds of consumer needs
or wants.
Substitution bias The bias which results when a basket index is used to estimate a cost‐of‐living
index, because a basket index cannot take account of the effects on the cost of
living of the substitutions made by consumers in response to changes in relative
prices. Sometimes also used more generally in the context of all CPIs.
Substitution effect The effect of substitution on the value of a CPI.
Superlative index A type of index which approximates to a cost‐of‐living index. They are generally
symmetric indices, i.e. they treat both the base and current periods symmetrically
by attaching equal importance to the price and expenditure data in both periods.
Systematic (interval) Type of sample selection when outlets or products are chosen with equal
sampling probability, but sampling units are selected at equal “distances” from each other
in the sampling frame, with only the first unit being randomly selected. It can be
used when the outlets or products are homogeneous.
System of National A coherent, consistent and integrated set of macroeconomic accounts, balance
Accounts sheets and tables based on a set of internationally agreed concepts, definitions,
classifications and accounting rules.
Tariff A list of pre‐established prices and conditions for the purchase of certain goods or
services which has been fixed by the supplier to exert influence on consumption
patterns by means of differentiated prices and conditions according to the
characteristics of consumers or to the level, the structure or timing of the
consumption.
Tax‐exclusive index A type of CPI which excludes indirect taxes entirely, by removing from both the
prices and the weights, those elements which are accounted for by taxes. This
type of an index is sometimes also referred to as a net‐price index.
Törnqvist price index A symmetric index defined as the weighted geometric average of the price
relatives in which the weights are simple arithmetic averages of the expenditure
shares in the two periods.
Trimmed index A CPI which has had extreme prices or price changes removed, thereby giving
more weight to the central values, which may be regarded as more typical and
hence suitable as a measure of underlying inflation.
Tukey algorithm A method of filtering data as part of a data validation and editing process.
Unit value (average The total value of the purchases/sales of a set of homogeneous products, divided
value) by the sum of the quantities. It is thus a quantity‐weighted average of the
different prices at which the products are transacted. A price index based on unit
values is known as a “unit value index”.
263
PRACTICAL GUIDE TO PRODUCING CONSUMER PRICE INDICES
User cost The cost incurred over a period of time by the owner of a fixed asset or consumer
durable as a consequence of using it to provide a flow of capital or consumption
services. It consists mainly of the depreciation of the asset or durable, plus the
capital cost or interest.
Uses approach An approach to CPIs in which the consumption in some period is identified with
the consumption of goods and services actually used up by a household to satisfy
their needs and wants. (See also “Acquisition approach” and “Payments
approach”).
Variable weights A technique used in dealing with seasonal products, under which weights vary
from month to month according to seasonal consumption patterns. One
particular type is known as the Rothwell index.
Weights (or expenditure The weight of a product in a CPI is the proportion of total household expenditure
weights) which is spent on that product during the weight reference period. It may be
defined so as to measure the un‐weighted average of all households’ expenditure
proportions or so as to measure the average proportion of total household
expenditure on that product. (See “Democratic index”, “Plutocratic index” and
“quantity weights”).
Weight reference period The period for which the expenditure shares serve as the weights for a CPI.
Young Index This is a weighted version of the Carli index.
264