Full download test bank at ebook textbookfull.

com

Research Evaluation for Busy

Students and Practitioners Helen

CLICK LINK TO DOWLOAD

https://textbookfull.com/product/research-
evaluation-for-busy-students-and-
practitioners-helen-kara/

textbookfull
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Research methods in clinical psychology an introduction

for students and practitioners 3rd Edition Barker

https://textbookfull.com/product/research-methods-in-clinical-
psychology-an-introduction-for-students-and-practitioners-3rd-
edition-barker/

River Ice Processes and Ice Flood Forecasting A Guide

for Practitioners and Students Karl-Erich Lindenschmidt

https://textbookfull.com/product/river-ice-processes-and-ice-
flood-forecasting-a-guide-for-practitioners-and-students-karl-
erich-lindenschmidt/

Introducing Bronfenbrenner A Guide for Practitioners

and Students in Early Years Education 1st Edition
Nóirín Hayes

https://textbookfull.com/product/introducing-bronfenbrenner-a-
guide-for-practitioners-and-students-in-early-years-
education-1st-edition-noirin-hayes/

A Practitioners Manual on Monitoring and Evaluation of

Development Projects 1st Edition Dharmendra Chandurkar

https://textbookfull.com/product/a-practitioners-manual-on-
monitoring-and-evaluation-of-development-projects-1st-edition-
dharmendra-chandurkar/
Quantitative Research Methods for Linguists A questions
and answers approach for students Tim Grant

https://textbookfull.com/product/quantitative-research-methods-
for-linguists-a-questions-and-answers-approach-for-students-tim-
grant/

Philosophy for Busy People Alain Stephen

https://textbookfull.com/product/philosophy-for-busy-people-
alain-stephen/

Dissertation Research and Writing for Built Environment

Students 4th Edition Shamil G. Naoum

https://textbookfull.com/product/dissertation-research-and-
writing-for-built-environment-students-4th-edition-shamil-g-
naoum/

Japanese for Busy People 3rd Edition Ajalt

https://textbookfull.com/product/japanese-for-busy-people-3rd-
edition-ajalt/

Graduate Research, Fourth Edition: A Guide for Students

in the Sciences Densmore

https://textbookfull.com/product/graduate-research-fourth-
edition-a-guide-for-students-in-the-sciences-densmore/
RESEARCH AND
EVALUATION FOR
BUSY STUDENTS AND
PRACTITIONERS
A TIME-SAVING GUIDE

Helen Kara

SECOND EDITION
RESEARCH AND EVALUATION FOR BUSY
STUDENTS AND PRACTITIONERS
A time-saving guide
(Second edition)
Helen Kara
First edition published in Great Britain in 2012, Second edition published in 2017 by

Policy Press North America office:

University of Bristol Policy Press
1-9 Old Park Hill c/o The University of Chicago Press
Bristol 1427 East 60th Street
BS2 8BB Chicago, IL 60637, USA
UK t: +1 773 702 7700
t: +44 (0)117 954 5940 f: +1 773-702-9756
[email protected] [email protected]
www.policypress.co.uk www.press.uchicago.edu

© Policy Press 2017

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication Data

A catalog record for this book has been requested

ISBN 978-1-4473-3841-3 paperback

ISBN 978-1-4473-3842-0 ePub
ISBN 978-1-4473-3843-7 Mobi
ISBN 978-1-4473-3875-8 epdf

The right of Helen Kara to be identified as author of this work has been asserted by her in accordance with
the Copyright, Designs and Patents Act 1988.

All rights reserved: no part of this publication may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise without the
prior permission of Policy Press.

The statements and opinions contained within this publication are solely those of the author and not of the
University of Bristol or Policy Press. The University of Bristol and Policy Press disclaim responsibility for
any injury to persons or property resulting from any material published in this publication.

Policy Press works to counter discrimination on grounds of gender, race,

disability, age and sexuality.

Cover design by Qube Design Associates, Bristol

Front cover image: istock
Printed and bound in Great Britain by by CPI Group (UK) Ltd,
Croydon, CR0 4YY
Policy Press uses environmentally responsible print partners
To my parents, Julie and Mark Miller, who taught

me to read, write and think, gave me a love of

learning, and encouraged me even when my

choices were different from their own.

 Contents

List of figures, tables and boxes ix

About the author x

Acknowledgements xi

Foreword by Patrick Sturgis xii

one Introduction 1
Introduction 1
Being a researcher or evaluator 3
Why do practitioners do research? 5
Insider and outsider research 8
Doing research or evaluation 9
Managing and commissioning research or evaluation 12
Terminology 13
Structure of this book 15
Exercise 16
Note 16
Further reading 16

two Overview of research 19

Introduction 19
Quantitative or qualitative? 19
Solo or collaborative? 21
Involving service users in research and evaluation 22
Highly time-consuming methods 25
Research and evaluation ethics 29
Looking after yourself 35
Exercises 38
Note 38
Further reading 38

three Methodologies, approaches, and theories 41

Introduction 41
Methodologies, methods, and approaches 41
Positivist methodology 43
Realist methodologies 44
Constructionist methodologies 45
Interpretivist methodologies 46
Transformative methodologies 46
Ontology and epistemology 47
Action research 49
Evaluation research 50
Mixed methods research 51
Arts-based research 52

v
 Research and evaluation for busy students and practitioners

Digitally mediated research 54

The role of theory in research and evaluation 55
Theory, research, practice 58
Exercise 59
Further reading 59

four Topics and proposals 61

Introduction 61
Choosing a research or evaluation topic 61
Refining your topic 63
From question to data 64
How much data? 66
Qualitative or quantitative methods? 67
Sampling techniques 68
Probability samples 69
Non-probability sampling 70
What is evidence? 71
Bidding and tendering for research or evaluation 72
Research funders 74
Exercises 75
Notes 75
Further reading 76

five Managing your research or evaluation project 77

Introduction 77
Planning 78
Organisation 80
Time management 81
Support from employers 84
Creative time management 88
Reward yourself 92
Look after yourself 92
What works, and what doesn’t work 96
Exercises 97
Notes 98
Further reading 98

six Background research 99

Introduction 99
Document review or literature review? 100
Record-keeping 103
Critical and strategic reading 105
Finding academic journal articles 109
How to conduct a document review 112
How to conduct a literature review 113
Using libraries 114
Making notes 116
Knowing when to stop 117
Exercises 118

vi
 Contents

Notes 118
Further reading 119

seven Secondary data 121

Introduction 121
Online secondary data sources 124
Secondary qualitative data 125
Archival data 126
Open data 127
Application programming interfaces 128
Large-scale surveys 129
International surveys 131
Working with secondary data 131
Exercises 132
Notes 132
Further reading 132

eight Primary data 133

Introduction 133
Collecting quantitative data 134
Questionnaire tips 139
Collecting qualitative data 140
Interview tips 143
Focus groups: pros and cons 144
Focus group tips 145
Documents as data 146
Observation 147
Visual data 150
Case studies 152
Creative methods 153
Collecting data online 154
Exercise 156
Note 156
Further reading 156

nine Data analysis 159

Introduction 159
Data preparation 160
Quantitative data coding 164
Qualitative data coding 165
Quantitative data analysis 170
Qualitative data analysis 178
A real-life example of qualitative data analysis 180
Geographic information systems 181
Data synthesis 183
Exercises 184
Note 184
Further reading 184

vii
 Research and evaluation for busy students and practitioners

ten Writing for research and evaluation 187

Introduction 187
Myths about writing 187
The writing process 190
Structure 196
Plagiarism 197
Citation 200
Findings versus recommendations 201
Editing 202
Polishing your writing 203
Exercises 204
Note 204
Further reading 204

eleven Disseminating research and evaluation 205

Introduction 205
Summarising research or evaluation 206
Barriers to disseminating research and evaluation 206
Presenting findings in person 208
Data visualisation 212
Dissemination methods 213
Disseminating workplace research 215
Disseminating academic research 216
Dissemination ethics 217
Closing the dissemination loop 218
Research and evaluation into practice 219
Exercise 220
Further reading 220

twelve Conclusion 221

Appendix One: Job titles of interviewees 225

Appendix Two: Sample record-keeping grid 226

Appendix Three: Exchanging information electronically 227

Appendix Four: Ethical codes from different disciplines and countries 233

Glossary 235

Bibliography 247

Index 253

viii
 List of figures, tables and boxes

Figures
3.1 The action research cycle 49
3.2 The evaluation cycle 51
3.3 Research, theory, and practice: a cyclical relationship 58
5.1 Gantt chart for a research project 79
5.2 Time audit example 83
6.1 Example of a literature grid 104
9.1 Metadata record for photographic data 161
9.2 Data coded using a coding frame 166
9.3 Data coded using emergent coding 167
9.4 (a) Frequency distribution table 172
9.4 (b) Bar graph 172
9.4 (c) Pie chart 172
9.5 Scattergraph showing strong positive relationship 174
9.6 Bivariate frequency table 175

Tables
1.1 Comparing insider and outsider research 9
3.1 Methodologies, ontologies, epistemologies and methods 48
5.1 What works, and what doesn’t work 97
7.1 Pros and cons of using secondary data 122
8.1 Pros and cons of questionnaires 139
8.2 Pros and cons of interviews 143
8.3 Pros and cons of focus groups 145
8.4 Pros and cons of using documents as data 147
8.5 Pros and cons of observational data 149
8.6 Pros and cons of visual data 152
8.7 Ranking matrix 154
8.8 Pros and cons of online data collection 155
9.1 Pros and cons of two kinds of coding 168
9.2 Parametric tests and non-parametric equivalents 177
10.1 Regularly expressed myths about writing, with my responses 190
10.2 Structures for research reports, dissertations and theses 196
11.1 Pros and cons of different presentation methods 212
11.2 Pros and cons of different dissemination methods 214

Boxes
6.1 Process of written communication 106
9.1 Managing outlying codes in practice 169

ix
 Research and evaluation for busy students and practitioners

About the author

Dr Helen Kara has been an independent social researcher and writer since
1999. Her background is in social care and the third sector, and she works with
universities, non-profit organisations, and social care and health partnerships.
While working full time, she obtained both her MSc in Social Research Methods
and her PhD. She is a Director of We Research It Ltd and a member of the
UK and Ireland Social Research Association (SRA), currently serving on their
Board as the SRA’s lead for research ethics. Helen is a Fellow of the Academy of
Social Sciences, and a Visiting Fellow at both the National Centre for Research
Methods and the Graduate School of Staffordshire University. She teaches research
methods to students and practitioners, and loves to demystify the processes of
writing and research.

x
 Acknowledgements
There may be only one name on the spine, but it takes more than a single person
to make a book. In fact, it takes more than just people. For example, this book
would never have been written without Montezuma’s organic chilli chocolate to
keep my brain on fire, and the Co-op’s Fairtrade wines to damp it down when
necessary.
My family have to be supportive, because it’s in the job descriptions for Mother,
Father, Sister, Brother-in-Law and Nephew: that’s Julie Miller, Mark Miller, Ros
Hodge, Carl Hodge, and Jamie Round, respectively. Talking of families, my out-
laws – the Holmeses, Mengells and Parrish-Joneses – aren’t a bad bunch either.
A number of people have helped this book along in ways that surpass the
bounds of comradeship. These are: Debi Alper, Jenny Beattie, Pam Carter, Zoë
Clarke, Su Connan, Anne Cummins, Mike Cummins, Nick Dixon, Ros Edwards,
Leigh Forbes, Anjali Gupta, Radhika Holmström, Helen Hunt, Denyse Kirkby,
Rob Macmillan, Jenny Maltby, Rachel McAllister, Jenny Parfitt-Bowman, Lucy
Pickering, Janet Salmons, Wayne Thexton and Katy Vigurs.
Writing can be a solitary process, but my friends on Twitter alleviated my
solitude, usually by taking the mick. The enchanting illustrations are by Carol
Burns; I would encourage you to see more of her excellent artwork at http://
artisbycarolburns.blogspot.co.uk/. Martin Holmes of the University of Victoria,
Canada, gave me great advice on referencing when we met in Paris. The technical
aspects of this book would have been woeful without the help of Nik Holmes
from Creative Technology, to whom I owe particular thanks for the first draft
of Appendix 3. John Kaye from Jisc gave invaluable input on two sections in
Chapters Seven and Nine. Jenny Parfitt-Bowman of Staffordshire University,
UK, kindly reviewed the section on quantitative data analysis and helped to
clarify several points.
At Policy Press, Victoria Pittman, Catherine Gray, and Dave Worth were kind,
supportive, and on the ball. Hawk-eyed and knowledgeable copy editor, Judith
Oppenheimer, saved me from many blunders in the first edition. I owe a big debt
of gratitude to five anonymous referees of the proposal for the first edition of this
book, and seven anonymous referees for the second edition; without your helpful
and constructive suggestions, this would have been a much feebler tome. Cole
Davis and Woody Caan also gave some helpful feedback on the first edition. I
owe another big debt of gratitude to the 20 anonymous people who gave their
time and expertise in interviews for the first edition; your input strengthened
my work enormously. Thanks, also, to a couple of Research Companions who
provided tips for Chapter Seven in the second edition.
The biggest thanks of all, for his constant love and support, go to my beloved
Nik.
All the mistakes are, of course, mine. So are most of the good parts.

xi
 Research and evaluation for busy students and practitioners

Foreword
For many of us in the university sector, the term ‘researcher’ conjures up an
image of a highly trained academic, working full time on a well-defined and
well-resourced project. Our archetypal researcher is also probably working in a
team of similar individuals, has regular meetings with project leaders, and acquires
additional skills through bespoke training, as and when they are required.
In reality of course, this type of researcher is something of a rare and rarefied
breed. The majority of social-science research and evaluation in the United
Kingdom, as in other parts of the world, is undoubtedly undertaken by very
different kinds of researchers. These are people for whom research is probably but
one component – and often a small one at that – of their work responsibilities.
Often they are based in public and third-sector organisations with little or no
research culture.
It is also unfortunately likely that they will have few resources at their disposal,
and perhaps little in the way of formal training in research methods and project
management. On top of this, some will be pursuing formal qualifications
alongside their working responsibilities. No wonder many people in these kinds
of environments find research and evaluation somewhat daunting.
It is for this key group of social scientists that Research and evaluation for busy
students and practitioners by Helen Kara is ideally suited. It provides excellent
overviews of the different perspectives, frameworks, and theoretical orientations of
research practice, as well as a compendium of key methodologies and techniques,
covering quantitative and qualitative approaches as well as their integration.
Readers will find clear descriptions of core aspects of research practice such
as: undertaking literature reviews; drawing samples; questionnaire design;
interviewing; focus groups; and observation. But this is not a book that aims, as
so many methods texts do, to provide a menu of techniques alongside a ‘how to’
set of instructions for their application. Rather, and this is its real strength, Research
and evaluation for busy students and practitioners provides practical guidance on the
broader set of skills and capabilities that are needed to deliver high quality research.
This is not so much about technique as selecting appropriate and feasible research
questions, deciding between primary or secondary research, managing your time
effectively, complying with standards and procedures of ethical research practice,
and managing your research alongside that other important thing: your life.
Containing a rich blend of guidance on theory and technique alongside helpful
reflections, practical tips, and imaginative exercises, this second edition will serve
as an invaluable guide for practitioner researchers for years to come.

Patrick Sturgis, Professor of Research Methodology, University of Southampton, and

Director of the ESRC National Centre for Research Methods (NCRM)

xii
 ONE

Introduction

Introduction
This book is for both students and practitioners in public services. By ‘public
services’ I mean services run by society for society, such as health, social care,
criminal justice, and education services from pre-school to university. Some
public services are paid for by the state from our taxes, and others are run as
charities, businesses, or social enterprises. ‘Practitioners’ are people who work
in these services, whether paid or unpaid. They may also be users of the services
in which they work and/or of other public services. They may also be studying
for a formal qualification.
In the current climate, research is becoming an increasingly common
requirement of public service jobs. This may be workplace research, such as
evaluation of a service or intervention, a service user satisfaction survey, a skills
audit, or training needs analysis. Or it may be academic research such as a diploma
or a Master’s degree for the purpose of professional development or to support
career progression. The differences between workplace and academic research

1
 Research and evaluation for busy students and practitioners

are not as pronounced as you may think, because good quality research demands
most of the same approaches and techniques, regardless of context.
There is also less difference than might at first appear between academics,
practitioners, students and service user/carer researchers. People move much
more freely between educational and practice roles these days. Any academic or
practitioner may also be a service user and/or a carer, and/or be studying for a
qualification. Work across disciplinary boundaries is also more common, whether
by multi-disciplinary teams or by a single researcher who has, and wants to use,
knowledge from more than one discipline. I am using the term ‘practitioner’ in
this book as a general term for anyone doing research while working in a public
service, whether that work is paid or voluntary, informal or formal, and whether
their research is under the auspices of the organisation or an academic institution.
I have worked as an independent researcher in public services since the late
1990s. After a while, practitioners began to ask my advice about their own research
projects. I decided to write this book because I have seen at first hand some of
the difficulties practitioners encounter when they are faced with the need to do
research. As part of the preparation for writing the first edition, I conducted 20
in-depth interviews with public service practitioners. These practitioners had
a wide range of roles (see Appendix 1) and they all had experience of doing
research on top of a main job. Some also had experience of doing research as
service users, or of supporting service users or other practitioners through the
research process. I asked them how they managed to do research on top of their
main jobs, and what advice they would give others who found themselves in a
similar position. The insights they gave me were invaluable. Quotes from their
interviews can be found throughout this book indented, like this:

I think, obviously naively, where’s the book that will tell me what to
do? It doesn’t exist because it’s more complex than that. There’s no
one book that will say, ‘If you want to research this, do this.’

As this interviewee suggests, this book is not an instruction manual that will
lead you step-by-step through your research project. That’s because there is no
such book. Writing a comprehensive instruction manual for research would be
as impossible as writing an instruction manual for a city. You have to find your
own way through your research, just as you would have to find your own way
around an unfamiliar city, and this is your guidebook. Within these pages you
will find much useful advice about places to go – and places to avoid. Then it’s
up to you whether you go to the recommended places, or the dangerous places;
the risk is yours. Think about it, though. Careful use of a guidebook can help
to increase enjoyment and reduce stress when you are in an unfamiliar place.
Similarly, this book is designed to help maximise enjoyment and minimise stress
as you navigate through your research project.
The book is about research and evaluation because evaluation is a particular
type of research that public service practitioners are often required to carry out.

2
 Introduction

However, I’m not going to write ‘research and evaluation’ all the time, because
that would become very tiresome for my readers. In this book, ‘research’ generally
includes ‘evaluation’.

Being a researcher or evaluator

While you are conducting research or evaluation, you are a researcher (Davies
2007:6). It doesn’t matter whether you are a novice or experienced; whether
your project is one of many in your life, or the only piece of research you ever
carry out. While you are doing your project, you have the identity of ‘researcher’
to add to all your other identities: friend, colleague, sibling, parent, service user,
and so on. You already know what those identities mean and how they interact.
For example, you might be more likely to talk about your emotional problems
to a family member than to a colleague. You will probably respond lovingly if
a child wakes you in the night because they had a bad dream, but you might be
less sympathetic if you were woken for the same reason by a friend.
Some common qualities of researchers are:

Determined Creative Thoughtful

Intelligent Tenacious Organised
Reflective Empathic Meticulous
Conscientious Thorough Analytical
Self-aware Assertive Honest

From a personal point of view, I want to do it to the best of my ability.

I have had comments like, ‘Just do enough to pass.’ I think, ‘Yes, I
will,’ but I can’t. It’s not who I am.

Researchers also need to be skilled in reading, writing, thinking, generating ideas,

making connections, dealing with people, negotiating, balancing competing
needs, and managing setbacks.
This may look terribly daunting. I include it here because I think it’s easier
to find your way if you know where you’re going. All identities are learned.
You will have had experience of this during your life: perhaps you learned to
be a university student, or a parent, or a sportsperson. So you can learn to be a
researcher – and, like all identities, you learn it on the job.
All identities overlap and interact with each other. Nobody is ‘just a shop
assistant’ or ‘only a student’. We all manage multiple identities all the time.
Nevertheless, adding a new identity to the portfolio can be a stressful process,
even if it’s one you would like to acquire. That process will be even more stressful
if it’s not an identity you want. You may feel resistant to the idea of taking on the
‘researcher’ identity, particularly if you loathe the idea of doing research and are
having it forced upon you. Perhaps you have a Masters’ dissertation or thesis to

3
 Research and evaluation for busy students and practitioners

complete, or your manager has delegated some evaluation research to you. But
even if you hate the very idea and are determined only to ever do one piece of
research in your life, it’s worth doing it as well as you can. This is because you
will inevitably use research in your work over the coming years and decades, and
not all research is good research. Nothing equips you for judging the quality of
others’ research as well as doing a piece of research yourself.
One key research skill is communication. Fortunately, public service practitioners
usually have good communication skills. However, embarking on research can
leave you feeling like a bumbling amateur. I think this is because you’re working
from an unfamiliar identity – that of researcher – and you’re putting other people
into unfamiliar identities of their own, such as research participant, before you
have acquired the skills to help this along.

For people who haven’t got a lot of experience in doing research, it

gets very difficult when you’re learning on the job, in front of your
colleagues, in a whole new sphere of skills that you haven’t actually
got, so you feel well outside of your comfort zone even though
you’re a very experienced practitioner. People feel very exposed and
unsure, and that gets very difficult for a lot of practitioners who are
very experienced to understand that they’ve got to go through that
learning process.

Also, research inevitably includes a lot of ‘first contacts’ which are notoriously
difficult for us humans who are predisposed to jump to conclusions (Kahneman
2011: 83–4). This can be doubly difficult when a first contact is made
electronically, for example by email or on social media, where there is more
scope for misinterpretation due to lack of contextual cues. Whatever your first
impression of someone you meet during the research process – whether favourable
or unfavourable – try to keep an open mind about that person, and remember
how little you actually know about them. Where possible, take the time to find
out more. Kahneman points out that we often operate as if what we see is all
there is (2011: 85), and so we are willing to reach a conclusion about someone’s
personality and potential on the basis of a very brief contact. In fact, people are
astonishingly complex, and even a small investment of time and attention can
often pay dividends.
Other key research skills, such as project management, negotiation, and time
management, will be covered later in the book. Although each of these is a skill
that can be learned, and they are all essential for researchers, it is unusual to find
these skills being covered in a book of this kind. Most research methods textbooks
are written as if research exists in a bubble, separate from everything else. My
intention is not to criticise research methods textbooks, or the wider body of
research methods literature, much of which is very useful. I have drawn on a
considerable amount of that literature, and I certainly couldn’t have written this
book without learning a great deal from other authors and scholars. However,

4
 Introduction

my aim in this volume is to acknowledge that research is part of life in all its
messy complexity, and to demonstrate ways of managing the research process in
its wider context without compromising research quality.

Why do practitioners do research?

We do research all the time, don’t we? If you want to buy a product
for work, say we need some marketing materials produced, I’m not
going to go with the first company, I’ll go with the company that
gives the best product at the best price. That’s research.

Why are you doing research? Because you’ve been told to? To earn money? To
please your manager? To improve your CV or help you get a qualification? To
increase knowledge, improve practice, or influence policy? For some other reason?
Or for two or more of these reasons, combined?
One reason may be that the requirement for research to be conducted within
public services appears to be increasing. I have observed this in recent years, as an
independent researcher working with practitioners in a range of public services,
and ten of the 20 interviewees for this book – from across the sectors – also said
that this was their experience.
Due to the worldwide economic downturn, there is less money to outsource
research, so it has become part of the main job for more people. There are also
positive reasons for the increase. One interviewee from the health sector was
eloquent about this:

Some reasons I can think of are:

1. Health professionals such as nurses access education at degree level

and above now which has changed in the last few years. This has
increased the body of literature regarding nursing, midwifery and
health visitors generated by those professionals.
2. Education at these levels creates more academically confident
professionals who understand, can contribute to and can critique
and apply research. They understand the benefit of research and how
it can benefit patient care in a way that was not there previously.
3. Extended roles for nurses, midwives and health visitors have
been developed over recent years, such as prescribing, which was
traditionally the stronghold of the medical profession. The ability
to manage complex issues and to make clear decisions requires
higher-level skills.
4. The NHS is required to make the best use of resources and deliver
the best possible outcomes for patients – evidence base is needed
to inform this.

5
 Research and evaluation for busy students and practitioners

5. Patients are able to access information such as through the we

band health professionals need to be able to demonstrate that the
treatments they offer are the best available in terms of safety and
efficacy – this can be shown through research evidence.

Another interviewee, from the criminal justice system, said an increase hadn’t
happened yet in the UK, but they could see it coming:

This whole research thing, it’s something that is ripe for expansion.
In America there’s research and development departments in some
police forces. There is in Australia; Australians in some respects are
quite far ahead of the British in terms of establishing a police research
culture. In Britain, there’s one or two research and development type
departments but they’re few and far between. I think if the police had
the capacity to do their own research, the database that they have, if
they had the ability to use academic analysis on their material, they
would, I’m certain, be able to reduce crime quite significantly because
they have the data that people would just love to get access to, but
because it’s confidential, you can’t.

As these interview extracts indicate, practitioners in public services are ideally

placed to conduct research which will improve those services. This applies whether
research done by practitioners is conducted entirely in the workplace or is for
an academic qualification. In some countries, and in some disciplines, research
skills are a requirement for career progression at certain levels. For example, in
New Zealand, social workers have to demonstrate that they can do a piece of
evaluation or research before they can progress to senior clinician status. However,
as the interviews for this book showed, public service workers are often required
to do research with little or no training or support.

I know people who think research is just about going in and talking
to people, and don’t understand even that you shouldn’t ask leading
questions. There is something to having done your background
research before you’ve done the research. It’s not true that anyone can
do it. I don’t think you necessarily need formal qualifications but to
have read something about it, or done a training course, you would
probably reap the rewards.

Formal training is seen as helpful but difficult to access for a number of reasons:
budget restrictions, lack of local courses, lack of courses in the subject area, and
insufficient understanding of training needs.

6
 Introduction

I don’t think there’s enough training courses available in this area. I

don’t think people understand, really, that things like training audits,
skills audits, evaluation, all fall under research.

One interviewee had some advice for those starting a job involving research
when they had no experience of research.

You need to set aside a day of your induction, sit with your line
manager or ideally your head of department, and get them to show
you what evaluations they’ve done, how they do it, and what they
expect of you. So you’re really seeing either a benchmark you’re
working towards, or the style of working practice.

This book will explain the fundamental principles and practices of research in
public services. It won’t provide everything you need – no book can do that –
but it will give you a good grounding in the subject, as well as lots of practical
advice about how to manage the research process in tandem with the rest of your
life, and tips to save you time and stress. Here’s an example:

Develop a strategy for naming and storing document files on

TIP your computer that will make them easy to find and retrieve.

Some of the tips in the book, like this one, may at first sight seem to add extra
work to your load. Think, though: putting in a little time up front will save you
from having to sort out a tangled mess later on.
Backing up your work also benefits from a strategic approach. There are many
ways to back up. For example, you can:

• Transfer your work manually to a memory stick or an external hard drive (and,
if you wish, from there to a different computer).
• Set up a free email address via one of the many providers such as Yahoo! Mail
or Gmail and send your work to that address.
• Use a free cloud-based service such as Dropbox, iCloud or Google Drive (see
appendix 3 for more information about working ‘in the cloud’).
• Use a paid back-up service such as SOS Online Backup or Carbonite.

Any of these processes will take only a few minutes to set up initially, and just a
few seconds to use each day. You should of course keep your back-up somewhere
away from your main data store; there’s not much point backing up onto a
memory stick which you then put in the same bag as your laptop. Also, you need
to consider the wisdom of using cloud-based services for people’s personal data
which should be kept confidential. But between all the options, if you take a little

7
 Research and evaluation for busy students and practitioners

time to think it through, you should be able to set up an easy-to-use system that
will protect you against time-consuming and stressful data loss.

While you’re working on a computer, get into the habit of pressing

Ctrl+S (on a PC) or Apple+S (on a Mac) every few minutes.
TIP
This will save your latest work against computer crashes, power
cuts, or accidents.

Insider and outsider research

‘Insider’ and ‘outsider’ are opposites, but presenting them in this dichotomous
way conceals the fact that they’re actually two ends of a spectrum. If you’re a
practitioner conducting research within the service you help to run, or a service
user researching the service you use, you’re definitely an insider. However, it can
be argued that anyone doing research in their own service is in a sense becoming
an outsider, if they are taking on a researcher’s identity in a setting where others
are not researchers. But when are you definitely an outsider? When you’ve
stopped working in, or using, a service? If so, do you become an outsider straight
away, or only after a period of time? Or are you only, really, an outsider if you’ve
never worked in, or used, that service? In which case, can anyone ever conduct
outsider research on primary health care or school education? Conversely, how
much research does an outsider have to do within a particular organisation before
they become an insider?
There are pros and cons to both insider and outsider research (Robson and
McCartan 2016: 399–400). A comparison is set out in Table 1.1 overleaf (and
I am indebted to Robson and McCartan for many of the points made here).
As you can see, neither insider nor outsider research is ‘best’. The answer to
the question of which approach should be used depends on the nature of the
research to be conducted.
There is more information about insider research in the ‘further reading’ section
at the end of this chapter.

8
 Introduction

Table 1.1: Comparing insider and outsider research

Insider research Outsider research

Insider researchers have expert knowledge of their Outsider researchers have to spend time learning
service which will help to inform the research about the service
Insider researchers know where to go for help and Outsider researchers have to spend time
information within their organisation getting to know the people and politics of the
organisation
Insider researchers may be attached to a Outsider researchers bring a fresh, independent
particular view of their service view to the research
Insider researchers may have more credibility with Outsider researchers have to build relationships
their colleagues and service users than an external and develop trust with service staff, volunteers
stranger (although the reverse can also be true) and users
Insider researchers may have difficulty in Outsider researchers usually find it easier to
challenging the practice of their colleagues, or challenge practice where necessary
their own practice, even where research findings
demonstrate that change is needed
Insider researchers are likely to have full access to Outsider researchers may not have full access to
confidential information confidential information
Organisations may be more willing to facilitate Outsider researchers may find it more difficult to
insider researchers’ access to potential gain access to potential participants
participants
Some participants will give fuller and more honest Some participants will give fuller and more honest
information to insider researchers who they information to outsider researchers who they
already know and trust believe will maintain their anonymity
Insider researchers are well placed to oversee Outsider researchers have to let go of the research
implementation of the research findings or when it is finished, and leave the organisation to
recommendations implement its findings or recommendations
Research done by an insider, with the full Research done by an outsider can become the
involvement of service users and colleagues, can focus of organisational discontent, at worst
greatly improve the working practices of the leading to the outsider researcher becoming a
organisation scapegoat

Doing research or evaluation

For a long time, writers on research and evaluation have acknowledged that
participating in such research – say by completing a questionnaire, or taking
part in an interview – may not be someone’s top priority. Interviewees were
also aware of this.

It’s OK doing research but if you can’t get anyone to take part in it and
you’ve got very low sample numbers it almost becomes pointless. It’s
never anyone’s priority, it’s only the priority of the person conducting
it, never the recipient’s.

I’d go further. I think it’s essential to acknowledge that there are times when
conducting research or evaluation isn’t even a researcher’s top priority. There are

9
 Research and evaluation for busy students and practitioners

times when looking after a sick child, a night out with friends, visiting a bereaved
relative, or going on holiday with your partner will come first. And quite right too.
There is an art in balancing research or evaluation with the rest of your life.
Everyone will differ in the extent to which they prioritise research. I prioritise
it highly, because I love research and evaluation, and have made this my career
since the late 1990s. For some people, research is a full-time job, or full-time
study, and most people in either of these situations do not have difficulty finding
time for their research. But for many people it is a part-time occupation, and
for some people very part-time. For example, someone who has been forced to
do a service evaluation – by a manager whom they don’t like, with no training
or support, when they are sure they already know what the findings will be – is
not likely to prioritise their research very highly at all.
Whether research is a high or low priority for you, and whatever your other
commitments, this book will help you find ways to fit research or evaluation into
your life. And it is important to balance research with the rest of your life, or there
can be long-lasting consequences, as these interviewees found after neglecting
friends and family in favour of research.

I did not spend the time I should have with them and I know I lost
lots of contact with relatives and friends which I miss now that I am
semi-retired and have the time.

I do feel that I probably lost a little bit of my social circle because I

wasn’t as accessible, I wasn’t the party animal I had been. Now I’m
trying to make up for it!

Some friendships have been sacrificed, and relationships have been

sacrificed. Some people need you to make that time and if you’re
not there to make that time, well, I’m OK with that. I think from
an impact point of view, I’ve perhaps been more absent than other
friends, but the people who have persevered, we’ve perhaps got a
deeper relationship.

Although, as the last interviewee hinted, there can also be positive consequences
for family and friends from spending time on your research.

I think if it had an impact, it was a positive one. My son has a great

intellect but really struggles with his dyslexia, he struggles with getting
things on paper. He learned perseverance. I think some of my women
friends would say it’s a bit inspiring to see one of your friends doing
that, and they’ve gone on and retrained and changed what they do.

Several interviewees suggested that you should think through your plans at the
earliest possible opportunity, and talk them over with people you trust.

10
 Introduction

People should seek others out who have done what they are proposing
and sit down and talk to them about the ups and downs.

Get some good sounding-board people – they don’t have to be

researchers, but people you can talk to about the project, people you
feel comfortable talking to about things.

Just as identities overlap and interact, so do the component parts of the research
process. This book is written, like most books on research or evaluation, as if
there are separate and discrete parts of the process: background reading, data
collection, data analysis, writing up. Here is some of the reality:

• Reading in various forms is likely to occur throughout the process.

• Notes from your reading may be coded and analysed in the same way as
primary data.
• Documents may be categorised as background reading, or secondary data,
or both.
• Writing is an essential part of data analysis in particular, whether it be qualitative
or quantitative, and of the whole research process in general. (Rapley 2011: 286)

These are just a few examples of the ways in which research processes interact
and overlap. It is necessary to separate them for the purposes of discussion and
teaching, but in reality, they are inextricable parts of a whole.
Nevertheless, there are transition points in research projects, and it is these
transition points which some researchers find hardest to handle.

I finished gathering the raw data, and it sat there and looked at me
for about three weeks before I actually did anything with it. Then I
did a few graphs and tables…

Even after years of experience, I still procrastinate when faced with the blank page
on which I need to start writing a research report. But these days I know that it
is the constants of reading, writing, and thinking, that will carry me through the
bumpier stages of the research process. So I read some of the notes I’ve written,
and think about what I want to say, until I’m ready to start. Other people use
different techniques, such as this interviewee:

I’ve got a studying CD which involves quite a bit of Frank Zappa. It

makes me laugh, it makes me relax, then I can get started. I need to
find a way to relax. Music is my medicine.

Whatever works for you is fine. But if you don’t yet know what works for you,
I recommend experimenting with reading, writing, and thinking.

11
 Research and evaluation for busy students and practitioners

Reading, writing, and thinking permeate the research process (Hart 1998: 6),
and thinking is the most important of the three. Your thinking, like everyone
else’s, develops and moves on day by day. The fact that your brain is your most
useful research tool is particularly helpful for busy practitioners, because you can
think about your research or evaluation when you don’t have time to do any
of your other research work. You can make progress with your thinking in the
shower, on the bus, at the supermarket – any time you have to be doing something
which doesn’t require much of your brainpower. As a researcher, your brain is
your greatest asset, both for conducting the research itself, and for working out
the best ways to manage the research process in the context of the rest of your life.
The other thing it’s important to say at this early stage is that there is no such
thing as perfect research or evaluation. Yes, the standards are high, and with good
reason. Research should be rigorous, ethical, and robust; researchers should be
thorough and conscientious. If you don’t yet have a good understanding of the
reasons for this, you will by the time you’ve finished reading this book. But
research and evaluation are never perfect, and can never be perfect, because they
are conducted by, with, and for people just like us with all our conflicts and
inadequacies. Planning and carrying out research or evaluation to a high standard,
in the real world, is an enormous challenge – and can also be a source of great joy.

Managing and commissioning research or evaluation

It is difficult to manage or commission research or evaluation if you don’t know
how to do research yourself. (It’s not that easy when you do know how.)

I had to produce a commissioning document, interview all the

organisations that applied, and make a decision based on their
application and interview, on which could really deliver what we were
asking them to deliver. I don’t think we appointed the best contractor,
but we went on who scored the most points, and now, 18 months
in and looking back, I’m not quite sure what I’d do differently but I
know something needs to be different.

You will need at least a broad idea about research methods and the research
process if you are to manage or commission research effectively. You also need a
clear idea of what the research or evaluation is that you want to delegate or buy,
and why you want that work to be done.
Useful questions to ask yourself include:

• What do you want the research or evaluation to achieve?

• What are the resources you can invest? (These may include funding, time,
training and so on.)
• How will you know whether you’re getting good value for your investment?

12
 Introduction

• What steps will you need to take to maximise the likelihood of the research
achieving your aims?

One key step is to figure out which methodology, approach, and/or methods
you think would be most appropriate, to enable you to match your resources to
the research you want to manage or commission. The information in this book
should help you do that. I don’t recommend being entirely prescriptive about
the research design, as a researcher may come up with good ideas you haven’t
considered. This may be because they’re more experienced in doing research or
evaluation than you are, or simply because they’re looking at the research question
and design from a different angle. But it does make sense to be clear about what
resources are available and why you are making those resources available, and to
share that information with researchers at the earliest opportunity.
Another key step is to ensure that the person you charge with the responsibility
for doing the research or evaluation actually has the necessary skills and abilities.
Books like this may help, but nobody can truly learn to conduct research from
a reference book.

Terminology
Some academic language is quite impenetrable. Here’s an example: ‘Unlike
psychoanalysis, psychoanalytic competence (which confines every desire
and statement to a genetic axis or overcoding structure, and makes infinite,
monotonous tracings of the stages on that axis or the constituents of that
structure), schizoanalysis rejects any idea of pretraced destiny, whatever name is
given to it -- divine, anagogic, historical, economic, structural, hereditary, or
syntagmatic…’. This is quite an extreme example, being taken out of context
from a well-respected book by two French poststructuralists.a I do not include it
here to criticize the book or its authors, who have made a unique and valuable
contribution to social theory. However, many people do find their work, and
that of other academics, quite difficult to read.

Academic language, oh, ow, it made my eyes bleed, and that was really
time-consuming. I still can’t remember what ‘hermeneutic’ means.

Sometimes, I’m sure some people would shoot me for saying this,
when I’ve started to unpack some of it, some of it isn’t complex
in itself but it’s the language that’s used to make it complex. Like
phenomenological analysis. You’re analysing a phenomenon as
experienced by someone else, that is fairly straightforward but the
language has made it complex.

Luckily there is a fair amount of academic work which is written in more

accessible language. However, each profession has its own jargon and other

13
 Research and evaluation for busy students and practitioners

complex terminology, and this can also be a barrier, for example to educators or
service user researchers. The profession of research, including evaluation, is no
exception, and the language of research practice can be quite opaque.

Some of the people I teach are expected to do small-scale research

studies. Ninety per cent of them are practitioners within the service.
They find the language quite difficult, it’s a language they’re not used
to. Qualitative and quantitative research is not something that trips
off the tongue.

The language of research can put people off. I’m in practice and it’s
too academic for me.

In this book, I have tried to be clear about the meanings of the terms I use,
and to use those terms consistently. But other writers will use some of the same
terms with slightly different meanings, such as ‘document analysis’ where I have
used ‘document review’. And other writers will use different terms to mean the
same thing, such as ‘subject’ or ‘respondent’ to mean a person who takes part in
research, where I have used ‘participant’. Also, I have not tried to include every
possible research term in this book, just those that are necessary for understanding
the points I want to make. I have prepared a glossary of research terms which
you will find at the end of this book and online. But as you read other people’s
work on research methods, you will inevitably come across new research terms
and different definitions.
If you’re doing research or evaluation you are likely to have to get to grips with
some unfamiliar and challenging concepts. After all, you’re doing it to learn. If
you find complex abstract thought exciting and appealing, you’ll find plenty to
amuse you. If, on the other hand, you prefer simpler explanations, there is so
much literature available that you should be able to find readable commentaries
on the topics you need to understand.

TIP You don’t have to read the unreadable.

As a researcher, it is as important to learn to identify and skip documents that

won’t help you, as to learn to identify and read those that will (Langdridge and
Hagger-Johnson 2009: 19). Information about how to do this is in Chapter Six.
It has to be said that the language of public services can also be difficult and
off-putting. There are some good glossaries and jargon-busters online if you are
struggling with that terminology.

14
 Introduction

Write definitions of new words on Post-It Notes and stick them

TIP
where you will see them frequently, for example on the inside
of your front door, next to the kettle, or on the bathroom mirror.

There is a useful resource for help with research vocabulary in the ‘further reading’
section at the end of this chapter.

Structure of this book

Chapter Two gives an overview of research. It introduces quantitative and
qualitative research, and discusses the role of service users in research. The chapter
also introduces research ethics.
Chapter Three engages with some of the more complex concepts in the field:
methodologies and some more specialised approaches to research (action research,
evaluation research, arts-based research and digitally mediated research). The
chapter discusses how these link with methods, the role of theory, and outlines
the links between research, theory, and practice.
Chapter Four looks at how to choose a research topic and refine it into a
research question. Research methods are introduced, and advice is given on how
to write a research proposal or plan.
Chapter Five discusses ways to manage the research process in the context of
the rest of your life. Planning, organisation, and time management are discussed
in detail, and there is a lot of advice from practitioners who are experienced in
research. The pros and cons of receiving support for research from employers
are outlined. The need to reward yourself, and look after yourself, as you do
research, is emphasised. There is a summary of what works – and what doesn’t
work – in managing research.
Chapter Six describes the similarities and differences between document reviews
for workplace research and literature reviews for academic research. Information
is given about how to conduct document reviews and literature reviews, together
with advice on record-keeping, critical and strategic reading, finding open access
materials, using libraries, and making notes.
Chapter Seven explains the advantages and disadvantages of working with
secondary data, and shows how to find secondary data on the web. Chapter Eight
covers a range of ways of collecting quantitative and qualitative primary data.
Chapter Nine outlines some ways of preparing, coding, analysing and synthesising
quantitative and qualitative data.
Chapter Ten begins by identifying and dispelling some myths about the writing
process. Then there is a full discussion of how to write research, with advice
on how to structure, edit and polish your writing. The chapter explains how to
avoid plagiarism and how to cite other people’s work.

15
 Research and evaluation for busy students and practitioners

Chapter Eleven looks at how to disseminate research, why dissemination

is important, and the potential barriers to this process. The advantages and
disadvantages of different methods of dissemination are outlined. The similarities
and differences of workplace and academic research are discussed, as are the ethics
of dissemination.
Chapters One to Eleven end with some suggested exercises and recommendations
for further reading.
Chapter Twelve concludes the book with a summary of the key points made
in the earlier chapters.
There is also a glossary of research terms and bibliography containing details
of all the references cited in the book.
The companion website for this book (http://policypress.co.uk/resources/
kara-research) contains resources for students and practitioners: some information
about studying research methods and the role of networking, real-life scenarios
to consider and learn from, and resources on time management. There are also
some resources for lecturers who need to teach research methods.

Exercise
You are asked to advise a research commissioner who wants to fund three pieces
of research:

1. Evaluation of patient care in a hospice.

2. Skills audit of civilian staff in a police force.
3. Pilot study, in three local schools, of whether training teachers in oral
storytelling techniques can improve educational outcomes for primary school
children.

In each case, would you advise the commissioner to specify an insider or an

outsider researcher? Why?

Note
a
Deleuze, G. and Guattari, F. (1987) A thousand plateaus: Capitalism and schizophrenia, translated
from the original French by B Massumi, Minneapolis: University of Minnesota Press, p 13
(original work published 1980).

Further reading

Bell, J. and Waters, S. (2014) Doing your research project: A guide for first-time researchers
(6th edn). London: Open University Press.

This is a classic text that will help you to understand what is involved in doing research.

Fox, M., Martin, P. and Green, G. (2007) Doing practitioner research. London: Sage.

16
Another random document with
no related content on Scribd:
compliance. To SEND DONATIONS or determine the status of
compliance for any particular state visit www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states where

we have not met the solicitation requirements, we know of no
prohibition against accepting unsolicited donations from donors in
such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot make

any statements concerning tax treatment of donations received from
outside the United States. U.S. laws alone swamp our small staff.

Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could be
freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose network of
volunteer support.

Project Gutenberg™ eBooks are often created from several printed

editions, all of which are confirmed as not protected by copyright in
the U.S. unless a copyright notice is included. Thus, we do not
necessarily keep eBooks in compliance with any particular paper
edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg Literary
Archive Foundation, how to help produce our new eBooks, and how
to subscribe to our email newsletter to hear about new eBooks.