08/01/2024

Outline of The Course

CMU-CS 462: Part 1: Measurement theory

Software Measurement Part 2: Software product, process and
and Analysis project measurements
Part 3: Measurement management
Spring 2023-2024
Nguyen Duc Man
E: [email protected]
T: 0904235945

1 2

1
 08/01/2024

Contents
1. What is measurement?
Overview of Software Metrics 2. What are software metrics?
3. Scope of software engineering metrics:
a chronological review.
Nguyen Duc Man
E: [email protected]
T: 0904235945

10

3 4

2
 08/01/2024

Measurement: Definition /1 Measurement: Definition /2

◼ Measurement is the process by which numbers or ◼ Metrics are standards (i.e., commonly
symbols are assigned to attributes of entities accepted scales) that define measurable
(objects) in the real world in such a way as to attributes of entities, their units and their
ascribe them according to defined rules.
scopes.
Đoattribute (value
lường là1 quá trình , value
11 mà các
12 số )  ký
, ...hoặc ◼ Measure is a relation between an attribute
  các and a measurement scale.
hiệuattribute

Object = 
2 (
được gán chovalue 21 value 22 , ...)
các ,thuộc tính của
thực thể (đối tượng) trong thế giới thực
... để mô tả chúng
theo cách ... theo các quy  tắc
attribute value , value , ... 
(xác định.
)
 n n1 n2

5 6

5 6

3
 08/01/2024

Entity Attribute
◼ An entity in software measurement can be ◼ An attribute is a feature or property of an entity
e.g., blood pressure of a person, cost of a journey, duration of
any of the following: ◼

the software specification process

◼ Processes: any activity related to software
development and/or maintenance (e.g., requirements
◼ There are two general types of attributes:
engineering, testing) – these can be at different ◼ Internal attributes can be measured only
levels of granularity based on the entity itself,
◼ Products: any artifact produced or changed during Code ◼ e.g., entity: code, internal attribute: size,
software development and/or maintenance (e.g., modularity, coupling
source code, software design documents)
◼ External attributes can be measured only
◼ Resources: people, hardware or software needed for
the processes with respect to how the entity relates to its
environment
◼ e.g., entity: code, external attribute: reliability,
maintainability

7 8

4
 08/01/2024

Measurement Example Measurement: Types

◼ Measurements are needed as:
Entity Attribute 4
3
2
◼ Descriptors of entities already in existence.
Requirements Size, Reuse, Redundancy
1
0
◼ Prescriptors (standards, norms, failure intensity
Specification Size, Reuse, Redundancy
A B objectives, benchmarks) which entities of certain
Design Size, Reuse, Modularity, class or category should satisfy.
Cohesion, Coupling
◼ Predictors to estimate properties of entities yet
Code Size, Reuse, Modularity,
Cohesion, Coupling, Complexity
to be designed or implemented.
Test Cases Size, Coverage

10

9 10

5
 08/01/2024

Measurement : How to Example 2: Code

Entity: Code
◼ In order to make entities measurable: ◼

◼ Attribute: Size
◼ What entities (objects) should be selected? ◼ Possible measures:
◼ NCSLOC (Not Commented Source Lines of
◼ What attributes should be selected?
Code Code)
◼ What values should be assigned to the attributes? ◼ #Statements

◼ #Modules
◼ What shall be the rules (relationships) ascribed to the
◼ #Procedures
attributes and their entities?
◼ #Classes

◼ Note: assigned values and/or ascribed rules can be ◼ #Methods

◼ …
either quantitative or qualitative.

11

11 14

6
 08/01/2024

Example 3: Availability Software Metrics Challenges

◼ Measuring physical entities:
◼ Entity: Availability
entity attribute unit value
◼ Attributes: system uptime, system downtime Human Height cm 178
◼ Values: time in seconds ◼ Measuring non-physical entities:
◼ Relations: entity attribute unit value
Human IQ IQ index 89
Availability = uptime / (uptime + downtime)
◼ SE metrics are mostly non-physical
◼ Reliability, maturity, portability, flexibility,
maintainability, etc. and relations are unknown

22 16

15 16

7
 08/01/2024

Misleading Metrics! What Is Software Measurement?

◼ Fact (1): Knowledge is power ◼ Software metrics are measures that are used
◼ Fact (2): Time is money
◼ Relation (rule) : power = work / time
to quantify software, software development
Substituting “power” and “time” resources, and/or the software development
◼ Knowledge = work / money process.
◼ As knowledge approaches zero money ◼ This includes items which are directly
approaches infinity regardless of the amount measurable, such as lines of code, as well as
of work done!
What went wrong items which are calculated from
◼ Conclusion: here?

◼ The less you know, the more you make! - Counter intuitive measurements, such as software quality.
- Needs validation

17 18

17 18

8
 08/01/2024

Measurement in SE Measurement in SE /1
◼ Measurement in SE is selecting, measuring ◼ Before a measurement project can be planned
and putting together many different attributes ◼ Objectives and scope should be established
of the software, and adding our subjective ◼ Alternative solutions should be considered
interpretations in order to get a whole picture ◼ Technical and management constraints should be
of the software. identified.
◼ This is not a ◼ This information is required to estimate costs,
trivial task! project tasks, and a project schedule.
◼ 300+ metrics have
been defined.

19 20

19 20

9
 08/01/2024

Measurement in SE /2 Measurement in SE /3
◼ In order to manage software measurement project ◼ Software metrics help us understand the technical
one must understand and plan: process that is used to develop a software product.
◼ The goal and scope of work ◼ The process is measured to be improved.
◼ The product is measured to increase its quality.
◼ Risks
◼ Resources required But …
◼ Measuring software projects is controversial.
◼ Tasks to be accomplished
◼ Milestones to be tracked ◼ It is not yet clear which are the appropriate metrics

◼ Total costs of the project

for a software project or whether people, processes,
or products can be compared using metrics.
◼ Schedule to be followed

21 22

21 22

10
 08/01/2024

Scope of Software Metrics Scope of Software Metrics

◼ Cost and effort estimation ◼ Performance evaluation and models More about
More about scope in
◼ Productivity measures and models scope in ◼ Structural and complexity metrics
the next
◼ Data collection the next ◼ Capability maturity assessment session
session ◼ Management by metrics
◼ Quality models and measures ◼ Evaluation of methods and tools
◼ Reliability models
◼ Performance evaluation and models

23 International 24
School

23 24

11
 08/01/2024

Example 1 Example 1 (cont’d)

◼ We are going to buy a new colour laser printer for our ◼ Failure intensity
department. We have borrowed the printer for the test. objective:
λF = 1/10000 pages

Failure number
◼ Maker’s data shows that we need to change the toner every
10,000 pages. We would like to have only one failure during the ◼ Using reliability
period. demonstration
chart we can
◼ During test period, we observe that failures occur at 4,000 conclude that the
printer must be
pages, 6,000 pages, 10,000 pages, 11,000 pages, 12,000 pages
rejected. Normalized measurement unit

and 15,000 pages of output.

◼ What can we conclude about this printer?
33 34

25 26

12
 08/01/2024

Example 2 Example 2 (cont’d)

◼ We are going to initiate a new game and video on-demand ◼ How many service personnel do we need?
download service. The service is provided to the customers
who own PCs and register with the service. ◼ Using the value for failure intensity, each system
◼ The customers must use specialized software to download experiences 0.2 failure per 20 hours of operation or
games or videos from the server. The failure intensity of the 0.2 failure per week, on average.
software is 1 failure per 100 CPU hr. On average, the ◼ The total failures for 800 customers is 160 per week
specialized software system runs 20 CPU hr per week on
each client machine and there are 800 customers to be or 32 per day.
serviced. ◼ Each serviceperson can visit 4 sites per day,
◼ We would like to provide the customers with an on-site
repair service. Each serviceperson can make 4 service calls therefore, the number of required personnel is
per day. Service personnel are available 5 working days per 32 / 4 = 8.
week.
◼ How many service personnel do we need to hire?

35 36

27 28

13
 08/01/2024

Scope of Software Metrics

◼ Cost and effort estimation
◼ Productivity measures and models
Overview: ◼ Data collection
Scope of Software Metrics ◼ Quality models and measures
◼ Reliability models
Process, Product and Resources ◼ Performance evaluation and models

39

29 30

14
 08/01/2024

Scope of Software Metrics Scope of Software Metrics/1

◼ Performance evaluation and models Cost and effort estimation
◼ Structural and complexity metrics ◼ Software cost estimation is the process of predicting the
amount of effort required to build a software system.
◼ Capability maturity assessment ◼ Estimates for project cost and time requirements are
◼ Management by metrics derived during the planning stage of a project.
◼ Evaluation of methods and tools ◼ Models used to estimate cost can be categorized as either
cost models (e.g., Constructive Cost Model COCOMO)
or constraint models (e.g., SLIM).
◼ Experience is often the only guide used to derive these
estimates, but it may be insufficient if the project breaks
new ground.
◼ Many models are available as automated tools.

39 40

31 32

15
 08/01/2024

Scope of Software Metrics/2 Scope of Software Metrics/3

Productivity models and measures Data collection
◼ Definition: The rate of output per unit of input.
◼ Productivity = size / effort ◼ Very critical and very hard step.
◼ Productivity = LOC / person-month ◼ What data should be collected?
◼ Productivity model based on decomposition to measurable ◼ How it should be collected?
attributes:
◼ Is collected data reproducible?
◼ Example: software failure data collection
1) Time of failure
measurabl

2) Time interval between failures

3) Cumulative failure up to a given time
e

4) Failures experienced in a time interval

41 42

33 34

16
 08/01/2024

Scope of Software Metrics/4

Scope of Software Metrics/5
Quality models and measures Reliability models
◼ Software quality measurement (Rubey and Hartwick 1968~)
◼ Plot the change of failure intensity () against time.
◼ McCall’s quality factors (1977~), ISO 9126
◼ Many models are proposed. The most famous ones
are basic exponential model and logarithmic
Poisson model.
◼ The basic exponential model assumes finite failures
in infinite time; the logarithmic Poisson model
assumes infinite failures.
◼ Automated tools such as CASRE are available.

43 44

35 36

17
 08/01/2024

Reliability Models Scope of Software Metrics/6

◼ Failure intensity () Performance evaluation and models
versus execution ◼ Using externally observable performance

time (τ) characteristics such as response time and

completion rate
 0  ◼ Efficiency of algorithms
 − 
(B)  ( ) = 0 e  0 

0
(P)  ( ) =
0 +1

45 46

37 38

18
 08/01/2024

Scope of Software Metrics/7 Scope of Software Metrics/8

Structural and complexity Management by metrics
metrics ◼ Metrics for project control
◼ Control-flow structure ◼ Metrics related to specification quality
◼ Data-flow structure
◼ Metrics for the design model
◼ Data structure
◼ Metrics for documentation
◼ Information flow attributes
◼ Checking and testing metrics
◼ Complexity metrics V(F) = 5 ◼ Resource metrics
◼ Cyclomatic complexity (McCabe 1989)
◼ Change metrics
defining number of independent paths in
execution of a program
47 48

39 40

19
 08/01/2024

Scope of Software Metrics/9 Scope of Software Metrics /10

Evaluation of methods and tools Capability maturity assessment
◼ US Software Engineering Institute (SEI) model
◼ Efficiency of methods
(1989): CMM grading using five-level scale.
◼ Efficiency and reliability of tools
◼ ISO 9001: Quality systems: models for quality
◼ Certification test of acquired tools and assurance in design/development, production,
components installation and servicing (1991)
◼ Benchmarking ◼ ISO 9000-3: Guidelines for application of ISO 9001

to the development, supply and maintenance of

software (1991)

49 50

41 42

20
 08/01/2024

How to Implement? Who Benefits From Measurement

◼ The eight steps required to implement a software ◼ Managers
measurement program are: ◼ What does each process cost?
◼ How productive is the staff?
◼ Document the software development process ◼ How good is the code being developed?
◼ State the goals ◼ Will the user be satisfied with the product?
◼ Define metrics required to reach goals ◼ How can we improve?
◼ Identify data to collect ◼ Engineers
◼ Define data collection procedures ◼ Are the requirements testable?
◼ Have we found all the failures?
◼ Assemble a metrics toolset
◼ Have we met our product or process goals?
◼ Create a metrics database ◼ What can we predict about our software product in the
◼ Define the feedback mechanism future?

51 52

43 44

21
 08/01/2024

Exercise Exercise (cont’d)

◼ Suppose that you are asked to study various Tool Name/Vendor Languages Platforms Features
Supported
software development tools and recommend the Bean Machine Java Windows, OS2, Unix Best: Visual applet and JavaBean
best three to your company. The following table IBM generation
CodeWarrior Pro Java, C, C++, Unix, Windows, Mac Best: if you need to support Unix,
shows a list of available development tools. Metrowerks Pascal Windows, and Mac platforms
Java Workshop Java Solaris, Windows Better: Written 100% in Java; tools
◼ Work in Team 3 Students Sun Microsystems based on a web browser metaphor
JBuilder Imprise Java Windows, AS400 Better: database support
Visual Cafe for Java Java Windows Good: multithreaded debugger
Symantec
VisualAge Java Unix, Windows Good: includes incremental compiler
IBM and automatic version control
Visual J++ Java Windows Fair: All the bells and whistles for
Microsoft Windows

53 54

45 46

22
 08/01/2024

Exercise (cont’d) Conclusion

◼ What are the entities, attributes and their ◼ Without measurements there is no way to determine if the
process/product are improving.
values in your model? ◼ Metrics allow the establishment of meaningful goals for
improvement. A baseline from which improvements can
Entity Attribute Value be measured can be established.
Development Language Java, C, C++, Pascal ◼ Metrics allow us to identify the causes of defects which have
Tool supported major effect on software development.
Platform Win, Unix, Mac, ◼ When metrics are applied to a product they help identify:
OS2, AS400 ◼ which user requirements are likely to change
Feature Fair, Good, Better, ◼ which modules are most error prone
Best ◼ how much testing should be planned for each module

55 56

47 48

23
 08/01/2024

Readings and Assignment 1

◼ You are required to watch 02 videos in the link below
and Answer the following questions. The answers will be
submitted on e-Learning system (Assignment 1) before
17:00, 03-03, 2021.

◼ https://www.youtube.com/watch?v=bnydxXPN_rI
(overview SM)
◼ https://www.youtube.com/watch?v=jiOaywcVssQ
(Software Metrics)
◼ Questions
◼ How important software measures are?
◼ List at least 5 software metrics and short description
for each.
◼ What are the characteristics of software metrics?

49

24