Scribd
Upload
49 views

DECIDE

Uploaded by

Muhammad Aamir Gulzar Ahmad
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
49 views

DECIDE

Uploaded by

Muhammad Aamir Gulzar Ahmad
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 12

D E C I D E:

An evaluation framework

www.id-book.com 1 ©2011
AIMS
• Introduce and explain the DECIDE
framework.
• Discuss the conceptual, practical, and
ethical issues involved in evaluation.

www.id-book.com 2 ©2011
DECIDE: a framework to guide
evaluation
• Proposed By: Preece et al. (2002)
• Purpose: Provide some high-level
organizational guidance for evaluation of
interaction designs.

www.id-book.com 3 ©2011
DECIDE: a framework to guide
evaluation
• Determine the Goals
1
• Explore the Questions
2

3 • Choose the Evaluation Paradigm & Techniques

• Identify Practical Issues


4
• Decide how to deal with the ethical issues
5
• Evaluate, analyze, interpret and present the data.
6

www.id-book.com 4 ©2011
1- Determine the goals
• What are the high-level goals of the
evaluation?
• Who wants it and why?
• The goals influence the methods used for
the study.
• Goals vary and could be to:
 identify the best metaphor for the design
 check that user requirements are met
 check for consistency
 investigate how technology affects working
practices
 improve the usability of an existing product
www.id-book.com 5 ©2011
2- Explore the questions
• Questions help to guide the evaluation.
• The goal of finding out why some customers
prefer to purchase paper airline tickets rather
than e-tickets can be broken down into sub-
questions:
– What are customers’ attitudes to e-tickets?
– Are they concerned about security?
– Is the interface for obtaining them poor?
• What questions might you ask about the
design of a cell phone?

www.id-book.com 6 ©2011
3- Choose the evaluation
approach & methods
• The evaluation method influences how data
is collected, analyzed and presented.

• E.g. field studies typically:


– Involve observation and interviews.
– Involve users in natural settings.
– Do not involve controlled tests.
– Produce qualitative data.

www.id-book.com 7 ©2011
4- Identify practical issues
For example, how to:

•Select users
•Find evaluators
•Select equipment
•Stay on budget
•Stay on schedule

www.id-book.com 8 ©2011
5- Decide about ethical issues
• Develop an informed approval form

• Participants have a right to:


- Know the goals of the study;
- Know what will happen to the findings;
- Privacy of personal information;
- Leave when they wish;
- Be treated politely.

www.id-book.com 9 ©2011
6- Evaluate, interpret &
present data
• Methods used influence how data is
evaluated, interpreted and presented.
• The following need to be considered:
- Reliability: can the study be replicated?
- Validity: is it measuring what you expected?
- Biases: is the process creating biases?
- Scope: can the findings be generalized?
- Ecological validity: is the environment
influencing the findings? i.e. Hawthorn effect.

www.id-book.com 10 ©2011
Key points
 Many issues to consider before
conducting an evaluation study.
 These include: goals of the study;
involvment or not of users; the methods to
use; practical & ethical issues; how data
will be collected, analyzed & presented.
 The DECIDE framework provides a
useful checklist for planning an
evaluation study.

www.id-book.com 11 ©2011
A project for you …
• Find an evaluation study from the list of
URLs on this site or one of your own
choice.
• Use the DECIDE framework to analyze it.
• Describe the aspects of DECIDE that are
explicitly addressed in the report and
which are not.
• On a scale of 1-5, where 1 = poor and 5
= excellent, how would you rate this
study?

www.id-book.com 12 ©2011

You might also like