What is software?

Software quality & quality issues

Mission of a tester
Thinking like a tester
1 What is software?
What software does, how it misbehaves and
what is nearest future software trends
 Please give a
Definition of
computer software as
you feel it.
Software is all or part of the programs,
procedures, rules and relevant
documentation of the information
processing system (ISO / IEC 2382-1:
1993)

“
So from conceptual view
software is nothing more
than a “thing” that receives
some input, processes it in
some way and provides
output. That`s it!
But why we are takling about software?!
Because nowadays software is literally
everywhere!
 Please give some
examples of how
computer software
affects your life?
Software continues to
penetrate all parts of our life
Autopilot Big data Machine learning
Aim is to make all cars Not far time ago there simply Started more than half a century
automated and there will be no weren`t that much user data. ago nowadays this this topic
drivers at all in quite near future. Nowadays amount of data is experiencing a second youth.
But there are already a lot of growing exponentially and it Challenge here is for results to
challenges for making this needs to be handled properly be precise
happen

Artificial intelligence Social networks Internet

Despite a lot of good things AI Nobody could even imagine that Nowadays our life seems almost
could bring to us, there are still a social networks will affect impossible without Internet. But
lot of challenges and uncertainty humanity in such a way they what if it will start working
in controlling and predicting actually did. And one of incorrectly?!
behaviour of AI challenges is to control this
networks
There is one common thing
about all this stuff
General trend is to delegate more
and more control to software. At
this point we should be very aware
of quality of the software since it
may and will affect whole our lifes
 And now please give
some examples of how
computer software did
not work correctly
The Mariner-1: a loss of
18.5 million

Because of two
separate errors missile Place your screenshot here

was blown up for

security reasons on 3rd
minute of flight. One of
errors were missing
hyphen in code.
The Third World War

The failure in the Soviet

early warning system
Place your screenshot here
led to a false report on
the launch of five
ballistic missiles from
the territory of the
United States.
“Black” monday
October 19, 1987, the
Dow Jones index fell by
Place your screenshot here
508 points, losing 22.6%
of its value. The S & P
500 fell 20.4%. This
was the biggest loss for
Wall Street in history -
500 billion dollars for
one day.
Google error in Japan

Employees of the
company entered
Place your screenshot here
incorrect data in the
dynamic routing
protocol. Result was
that providers could not
connect to the Internet
or faced an extremely
low connection speed.
 Software quality
2
& quality issues
What is quality and what are issues that threaten quality
Please give a
definition of
quality
Quality: The degree to which a component, system
or process meets specified requirements and/or
user/customer needs and expectations [IEEE 610].

“
Quality is value for some person -Weinberg

“
Value for some person?!

Different persons
have different views on
which value really does matter
Who do you think is setting quality
expectations now?!
In acient times goverment set
quality standards
A complex idea can be conveyed
with just a single still image, namely
making it possible to absorb large
amounts of data quickly.
In times of industrial revolution
quality standards were set up by
manufacturing
A complex idea can be conveyed
with just a single still image, namely
making it possible to absorb large
amounts of data quickly.
Nowadays users set quality
standards
A complex idea can be conveyed
with just a single still image, namely
making it possible to absorb large
amount of data quickly.
 Please give a
definition of a
Bug (Mistake)
In September 9, 1945, Grace Hopper
work on the ’Mark II’ computer
at the Harvard Faculty.
Operators traced an error in the ’Mark II’
to a moth trapped in a
relay. Dead bug was carefully removed
and taped to the log book.
Stemming from the first bug, today we
call errors or glitches in a
program a bug.
So if quality is value for some person…
Then bug is anything that threatens
value of the product. E.g. calculator
will crash on attempt to add negative
numbers and zero
And issue is anything that threatens
value of the project rather than the
product itself. E.g. there are a lot of
Ads and such stuff on eBay. Am I
supposed to test that stuff? How
should I test it?
How do you think software bugs appear?!
How bugs appear concept

Error Defect Failure

Error
A human action that produces an
incorrect result. E.g. incorrect
algorithm implementation in
program
Defect
A flaw in a component or system
that can cause the component or
system to fail to perform its
required function, e.g. an incorrect
statement or data definition.
Failure
Deviation of the component or
system from its expected delivery,
service or result. A defect, if
encountered during execution, may
cause a failure of the component or
system.
What we can learn from this

Mistake (in code) is some human action

that leads to defect in code and may lead to
program failure to perform its expected
action only if this part of code will be
executed during program runtime (bug).
Some errors may be present in code for
years without being noticed…
What we can learn from this

One more important thing about bugs is

that what we observe a program
failure\misbehavior is not a mistake itself
but rather symptom\aftermath. Failure
occurs under conditions.
Bugs may appear because of
various reasons
◉ Human beings are fallible
◉ Time pressure
◉ Complexity of code, infrastructure
◉ Technology changes
◉ Complex systems interactions
◉ Lack of clarity in requirements
◉ etc.
Most
of defects are coming from
specification both obvious mistakes
and ones that are looking like
correct requirements but appears to
be incorrect
Most
expensive defects appear to be in
live use (operation). Most cheaper
are defects that are found during
specification and design phases
How do you think why it is so?
Cost of bugs

During development phase future product

is controlled at most because it does not
affect real users and any issue can be
fixed without any aftermath to customers.
And during live use it is very expensive
and time consuming to fix any issue
Cost of bugs

Specialty of software is that it can be

easily and quickly scaled and provided to
users. In such situation potential critical
bug may affect billions of people.
Gmail, Chrome, Android, Google Play,
Maps and YouTube each of this services
are used by more than 1 billion of people
Cost of bugs

At this point it should be clear that we

need to test product before releasing to
reveal and fix all important bugs, to be
sure it meets quality requirements and to
have knowledge about existing issues and
their influence on users
Task: write down all things you
threat as bugs on this picture
What do you think testing is?
 Let`s start with ISTQB
definition and principles
Testing: the process consisting of all lifecycle
activities, both static and dynamic, concerned
with planning, preparation and evaluation of
software products and related work products
to determine that they satisfy specified
requirements, to demonstrate that they are fit
for purpose and to detect defects.
ISTQB principles review

◉ Testing shows presence of defects

◉ Exhaustive testing is impossible
◉ Early testing
◉ Defect clustering
◉ Pesticide paradox
◉ Testing is context dependent
◉ Absence-of-errors fallacy
Testing shows presence of defects:
Testing can show that defects are
present, but cannot prove that there
are no defects. Testing reduces the
probability of undiscovered defects
remaining in the software but, even if
no defects are found, it is not a proof of
correctness.

“
Exhaustive testing is impossible:
Testing everything (all combinations of
inputs and preconditions) is not
feasible except for trivial cases. Instead
of exhaustive testing, we use risks and
priorities to focus testing efforts.

“
Early testing:
Testing activities should start as early
as possible in the software or system
development life cycle and should be
focused on defined objectives.

“
Defect clustering:
A small number of modules contain
most of the defects discovered during
prerelease testing or show the most
operational failures.

“
Pesticide paradox:
If the same tests are repeated over and over again,
eventually the same set of test cases will no longer
find any new defects. To overcome this “pesticide
paradox”, test cases need to be regularly reviewed
and revised, and new and different tests need to be
written to exercise different parts of the software or
system to find potentially more defects.

“
Testing is context dependent:
Testing is done differently in different
contexts. For example, safety-critical
software is tested differently from an
e-commerce site.

“
Absence-of-errors fallacy:
Finding and fixing defects does not
help if the system built is unusable and
does not fulfill the users needs and
expectations.

“
Testing definition by RST

Despite common terminology let`s distinct

two definitions:
◉ Testing
◉ Checking
Testing is the process of evaluating a
product by learning about it through
exploration and experimentation, which
includes to some degree: questioning,
study, modeling, observation, inference,
etc. Rapid Software Testing

“
Evaluating
means making a value judgment; is
it good? is it bad? pass? fail? how
good? how bad? Evaluations as a
noun refers to the product of the
evaluation, which in the context of
checking is going to be an artifact of
some kind; a string of bits.
Observations
are intended to encompass the
entire process of observing, and not
just the outcome. So the process
itself is important too. E.g. problems
that appeared during observation
are also should be called issues
Exploration
implies that testing is inherently
exploratory. All testing is
exploratory to some degree, but
may also be structured by scripted
elements
Experimentation
implies interaction with a subject
and observation of it as it is
operating, but we are also
referring to “thought experiments”
that involve purely hypothetical
interaction. We are merely trying
to express that experimentation is
a practice that characterizes
testing. It also implies that testing
is congruent with science
Checking is the process of making
evaluations by applying algorithmic
decision rules to specific observations of
a product. Rapid Software Testing

“
Algorithmic
means that it can be expressed
explicitly in a way that a tool
(automation tool e.g.) could perform
this kind of check
Specific observations
means that the observation process
results in a string of bits (otherwise,
the algorithmic decision rules could
not operate on them)
 RST vs ISTQB

Despite ISTQB, in RST it is preferable to put learning and

experimentation (rather than satisfying requirements
and demonstrating fitness for purpose) at the center of
testing. More preferable to think of a test as something
that people do as an act of investigation; as a
performance, not as an artifact.
3 Mission of a tester
What drives mission of a tester
 You are the headlights of the
project

Testing is done to find information. Critical

decisions about the project or the product are
made on the basis of that information.
 Find important bugs fast

◉Fixes and updates mean fresh risk

◉Test core functions before contributing functions
◉Test capability before reliability
◉Test common situations before esoteric situations
◉Test for high-impact problems before low-impact
problems
◉Test the most wanted areas before areas not
requested
 Cooperate with the programmers

When you test the things that the programmers

are building right now, or have most recently
built, your feedback will help them work more
efficiently. When they deliver it, you test it.
When they make a change, you test the change.
 You discover things that will "bug"
someone whose opinion matters
If you can show that the product will not be
valued even if it works as intended, it's your
duty to report your concerns.
 Question everything

Questions are fundamental to your role on the

project. It's possible to test without questioning,
but it's not possible to test well.
 You focus on failure, so your
clients can focus on success
Testing is the only role on the project that does
not directly focus on success. Testers focus on
failure because it improves their chances of
finding it. Look for key problems in the product
with all your creativity and skill. If you don't find
them, they can't be fixed, and then the users
may find them for you.
 You don't assure quality by
testing
It's all too easy to think of yourself as the
guardian of quality. But you don't create quality,
and you don't take it away. You may talk as if
you "break the product"; the truth is it came to
you already broken. Quality comes from the
people who build the product. A big part of
your mission is to help them.
4 Thinking like a tester
Testers think differently. How do they think differently?
 Testers like to break things

This is a common view. Here is an alternative

view. Testers don't like to break things; they like
to dispel the illusion that things work. Testers
don't enjoy giving bad news; they enjoy freeing
their clients from the thrall of false belief.
 Testing is in your head

The difference between excellent testing and mediocre

testing is how you think: your test design choices, your
ability to interpret what you observe, and your ability to
tell a compelling story about it.
 Good testers think technically,
creatively, critically and practically

◉Technical thinking. The ability to model technology

and understand causes and effects
◉Creative thinking. The ability to generate ideas and see
possibilities.
◉Critical thinking. The ability to evaluate ideas and
make inferences.
◉Practical thinking. The ability to put ideas into practice.
Overall, thinking like a tester leads you to
believe that things may not be as they seem.
However things are, they could be different. We find that
when the test process fails in the most damaging ways, the
root cause is most likely to be tunnel vision. In other words,
it's not that we ran 10,000 tests and should have run 10,001.
It's that we failed to imagine an entire category of test; testing
we wouldn't have performed even if we had twice the time
and resources. Lessons Learned in Software testing
“
 To test, you must explore

To test something well, you have to work with it. You

must get into it. This is an exploratory process, even if
you have a perfect description of the product.
Thanks!
Any questions ?