W6 W7 W9 Usability Testing

USABILITY TESTING
Perancangan Interaksi - Information Systems - Telkom University - Genap 1819

Project-Based Learning
1. Apa yang membuat sebuah website/sistem mudah digunakan?

2. Apa yang membuat sebuah website/sistem sulit digunakan?
3. Dari skala 1-10 seberapa yakin Anda dengan jawaban Anda?
Project-Based Learning
Buat sebuah project untuk menganalisis Write test plan
usability sebuah sistem. 1. Choose system

2. Choose users
Deadline 3. Task and scenario list (5)
1. Plan (1 week from now) 4. Moderator script
Subject: UsabilityTestPlan SI4100 a. Intro
KelompokX b. Pre interview question
2. Report c. Post interview question

5. Schedule
What is Usability?
Definition In a nutshell...
“The extent to which a product can be ● How easy for users to achieve their
used by specified users to achieve objective
specified goals with effectiveness,
efficiency and satisfaction in a specified
context of use.” ( ISO 9241-11 )
Aspects of Usability
1. learnability: how easy to accomplish tasks the first time?

2. efficiency: once learned, how quickly to complete tasks?
3. memorability: how easy to reestablish proficiency after not having used a design for a period
of time?
4. errors: how many, how severe, how easy to recover?
5. satisfaction: how pleasant to use the design?
What is Usability Testing?
Definition In a nutshell...
A process that employs people Observing how people
as testing participants who are use a system
representative of the target to evaluate the design
audience to evaluate the of the system
degree to which a product
meets specific usability criteria.
Why usability test? Users can have their own
way to understand and use
our system
We need to check if our

system is useful for users
Where to test? 1. In a laboratory
2. In natural settings
When to test? 1. any time.
2. early: exploring potential
possible designs
3. late: close to end stage
to determine possible
showstoppers
4. after: investigate
reported problems
What to test? 1. Concept
2. Prototype
3. System
4. Features
5. Flow
6. Design
NOT users
How to Design
Usability Test
1. Choose your participants
● Who are the users of the system?

● Do you need expert users or non expert
users?
● Optimal number of users : 5
● Recruit and schedule
2. Choose evaluation tasks
usability tasks example: Netflix

• “rate a few movies”
• “it’s a Friday night, and you’re looking for a movie to watch. What do you do?”
• “you’re about to watch `Batman 3’, but want to watch the first two, first. How do you do
this?”
• “you want to watch Batman 1 through Netflix in your living room with your xbox. How do
you set that up?”
• “what do you think about the site?”
3. Preparation
1. Documents 2. Tools
a. User Task List a. Screen and voice recorder or a camera
b. Moderator Script
c. Post-test questionnaire
d. Observation Notes
Usability Testing Setup
1 Moderator
1 Note taker
Moderator’s Job
● Ask what is on user’s mind

● Do not try to influence their oponion
● Repeat what you are saying
● Let user talk
● Stay neutral
Usability Test Process
Introduction
Evaluation
Tasks
Post test
Pre-test interview interview
Introduction
1. Thank the user for participating in the study

2. Explain about the study
3. Explain about the system
4. Ask user to say what they think out loud
5. You are testing the system, not the user
6. Ask if they have questions
Pre-test Interview
1. How long have you been in this field?
2. How long have you been working with this system?
3. What are the various cases you use this system, such as operation, examination, practice...
4. How often do you use the system during each of the cases?
5. ..
Tasks
1. Skenario 1
Bayangkan bahwa hari ini adalah hari Sabtu. Anda sedang malas keluar dan ingin
menghabiskan waktu di rumah dengan bersantai. Seorang teman merekomendasikan
untuk menonton drama Korea di Viki.
1. Tugas:
Bagaimana Anda memulai sesuai dengan rekomendasi teman anda?
Tasks
1. Make the task realistic
User goal: Browse product offerings and purchase an item.
Poor task: Purchase a pair of orange Nike running shoes.
Better task: Buy a pair of shoes for less than $40.
1. Make the task actionable
User goal: Find movie and show times.
Poor task: You want to see a movie Sunday afternoon. Go to www.fandango.com and tell me where
you’d click next.
Better task: Use www.fandago.com to find a movie you’d be interested in seeing on Sunday afternoon.
1. Avoid giving clues and describing steps
User goal: Look up grades.
Poor task: You want to see the results of your midterm exams. Go to the website, sign in, and tell me
where you would click to get your transcript.
Better task: Look up the results of your midterm exams.
Post-Test Interview
1. Do you feel that the system is usable?

2. What things annoy you about the system?
3. What are the difficulties did you find while using the system?
4. What do you think are the strengths of the system?
5. In your opinion how could the system be improved?
6. Is there any function in the system that you do not use? Do you think it should be removed?
7. Is there anything that we have not asked, that are of importance?
Asking Questions
1. Open-ended questions
“Show me how …” -> you ask them to demonstrate a task
“Tell me about …” -> you ask them
1. Don’t ask participants about solutions

“To design the best UX, pay attention to what users do, not what they say. Users do
not know what they want.”
Asking Questions
3. No Leading questions
a. “I saw you were having difficulty with the navigation. What happened?”
b. “Why did you have difficulty with the navigation?”
c. “What was easy or difficult about getting to the content you wanted?”
Think Aloud Technique
Ask user to verbalize what they are thinking
example:
This is what I expect will happen if I click here
This page or resulting action is not what I expected
This text/instruction/image/etc. is confusing
This is easy/difficult
This is what I think this text/instruction/image/etc. means
I like/dislike this
This is not what I need — I need XX instead
https://www.nngroup.com/articles/thinking-aloud-demo-video/
How to Write
Usability Test
Report
Report Structure
1. Executive Summary / Abstrak Format makalah
2. Table of Contents/Daftar Isi 1. TNR 12
3. Researchers/Profil Peneliti 2. Margin 4333
4. Introduction/Pendahuluan 3. spasi 1.5
5. Participants/Partisipan 4. kata serapan italic
6. Methodology/Metodologi -- Think Aloud Protocol 5. Cover - judul, NIM, tahun
7. Scenario and Tasks
8. Result/ Hasil
a. Task Completion
b. Errors
c. Key Takeaways
d. Post Test Answers
9. Recommendations (Discussion) / Diskusi &
Rekomendasi
10.Conclusion/Kesimpulan
11.Appendix/Lampiran
Report Structure
● Executive Summary
○ Explain briefly the goal, process, result and finding of the usability test (half page)
● Method (Plan)
Describe the nature of the research, how it was set up, participant characteristics, and the
data collection methods used.
a. Goals
b. Participants
c. Methodology
d. Scenario & Tasks
Report Structure
● Results
■ Present summaries of data and stories gathered from the test
● Task completion
● Error
● Findings and Recommendations (Discussion)
Write problems found in the interface and give a severity rate
Write suggestions/recommendation based on findings
Report Structure
● Appendix
a. Test Script
b. Pre-test question
c. Task list
d. Post-test question
e. Photos of every session
f. Observer notes
How User
Research Change
Design
Heinz Ketchup
“I remember sitting in one of those households,” he says.

“There was a three-year-old and a six-year-old, and what
happened was that the kids asked for ketchup and Mom
brought it out. It was a forty-ounce bottle.
“And the three-year-old went to grab it himself, and Mom

intercepted the bottle and said, ‘No you’re not going to do that’.
She physically took the bottle away and doled out a little
dollop. You could see that the whole thing was a bummer.”
Heinz Ketchup
Heinz ketchup bottle was hard to use by youngsters.
They redesigned the bottle and consumption of ketchup grew

12%
Heinz Ketchup
Heinz redesigned their bottle based on data gathered during

user research.
The new design was really successful because it solved a

genuine problem, not just for Heinz but also for parents. And
for kids too. Great products solve genuine problems.
Thanks!
Any questions?

W6 W7 W9 Usability Testing

Uploaded by

Copyright:

Available Formats

W6 W7 W9 Usability Testing

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

W6 W7 W9 Usability Testing

Uploaded by

Copyright:

Available Formats

USABILITY TESTING

Perancangan Interaksi - Information Systems - Telkom University - Genap 1819

1. Apa yang membuat sebuah website/sistem mudah digunakan?

Buat sebuah project untuk menganalisis Write test plan

usability sebuah sistem. 1. Choose system

Deadline 3. Task and scenario list (5)

1. Plan (1 week from now) 4. Moderator script

Subject: UsabilityTestPlan SI4100 a. Intro

KelompokX b. Pre interview question

2. Report c. Post interview question

1. learnability: how easy to accomplish tasks the first time?

We need to check if our

● Who are the users of the system?

usability tasks example: Netflix

● Ask what is on user’s mind

1. Thank the user for participating in the study

1. Do you feel that the system is usable?

1. Don’t ask participants about solutions

Ask user to verbalize what they are thinking

“I remember sitting in one of those households,” he says.

“And the three-year-old went to grab it himself, and Mom

Heinz ketchup bottle was hard to use by youngsters.

They redesigned the bottle and consumption of ketchup grew

Heinz redesigned their bottle based on data gathered during

The new design was really successful because it solved a

You might also like