2020 IEEE 5th International Conference on Intelligent Transportation Engineering

A Method of Improving the Efficiency of Verification and Validation for the

Railway Signaling System

Fagen Fang, Sha Wang, Qingyuan Shang, Qi Fang

UniTTEC Co., Ltd.
Hangzhou China
e-mail: [email protected]

Abstract—This paper describes how to make full use of the verification and validation tasks according to the
testing assets reusability to improve the efficiency and quality requirements in the aspects of time and quality? From the
of the project’s requirements validation, design verification perspective of how to use the reusability of the testing assets,
and system validation. The core technology of the scheme is this paper discusses the methods to solve this problem,
accurate text recognition, the reuse of requirement tracking eventually realizing that the projects’ verification and
matrix is taken as an example to introduce the implementation validation are completed efficiently and with high quality.
scheme and application effect.
TABLE I. WORKLOAD ANALYSIS
Keywords-testing assets, reusability, verification and
validation Stages Primary activities Workload
(man-day)

I. INTRODUCTION Project Traceability, verification and validation for

2
requirement the system requirement trace table
Verification refers to the affirmation that the specified Traceability, verification and validation for
requirements have been met by providing objective evidence, 25
the CI subsystem requirement trace table
in other words, verification is to check that whether the Traceability, verification and validation for
19
software meets the requirements of the required documents; the ATC subsystem requirement trace table
while validation refers to the affirmation that the specific Traceability, verification and validation for
11
the ATS subsystem requirement trace table
expected use or application requirements have been met by
Traceability, verification and validation for
providing objective evidence, in other words, validation is to the MSS subsystem requirement trace table
3
check that whether the finished product meets the customer’s System Design, verification and testing for the 96
requirements. Verification focuses on the process while design system
validation focuses on the results. Design and verification for the CI 24
During the implementation of safety product-related subsystem data
projects, verification and validation are indispensable and Design and verification for the ATC 8
subsystem data
important activities. During the product development or Design and verification for the ATS 8
project implementation, with respect to the deliverables of subsystem data
each stage of the project's life cycle, they need to be verified Design and verification for the MSS 8
item by item, need to be validated at the project requirement subsystem data
stage and system validation stage, and even need to be System Data, interface, function testing and 83
verified and validated with multiple iterations according to validation validation for the system
Data, interface, function testing and 83
relevant safety certification specifications. Taking the current validation for the system
project under implementation as an example, the main Data, interface, function testing and 1332
workload analysis of project verification and validation is validation for the CI subsystem
shown in Table ĉ. Data, interface, function testing and 310
According to the above table, it is estimated that a total of validation for the ATC subsystem
2207 man-days will be invested into a new project including Data, interface, function testing and 176
validation for the ATS subsystem
about 8 full-time persons. However, the actual project may Data, interface, function testing and 102
also have more than two iterations, so the human resources validation for the MSS subsystem
and equipment will be very scarce and cannot meet the
progress requirements if multiple projects are launched at the II. REUSE OF TESTING ASSETS
same time.
In this way, under the circumstance of more projects, less A. Testing Assets
persons and short time, how can we not only meet the
requirements of safety certification standards (EN50126[1] The testing assets referred here include not only various
[2], EN50128[3], EN50129[4]), but also finish various documents output by the testing process, but also the

978-1-7281-9409-7/20/$31.00 ©2020 IEEE 155

Authorized licensed use limited to: Carleton University. Downloaded on November 08,2020 at 05:19:00 UTC from IEEE Xplore. Restrictions apply.
experience, skills and thinking in aspects of verification, TABLE II. THE HEADER OF THE REQUIREMENTS TRACKING MATRIX
testing and validation accumulated by the verification and Requirements Design for
User Testing Validation
validation team. During the multi-project implementation, a Requirement
for general specific
Record Record
large number of process deliverables generated from the application applications
project planning, project preparation and project User requirements are the requirements specified in the
implementation to the project summary need to be verified contract to be delivered;
and validated by the verification and validation team for each The general application requirements are the product
function requirements of the released baseline;
project, among which the plans, methods, experience, and
The specific application requirements/design are the
even deliverables are all same or similar. However, if there is
specific requirements of the current project or design
no testing asset library, each project will start from scratch requirements of the line data;
and the experience and skills accumulated by the test team The testing log is the system testing log of the current
cannot be shared and inherited. In this way, not only the project;
work efficiency is low, but also it’s difficult to guarantee the The validation log is the system validation log of the
delivery quality. Therefore, testing assets reuse is current project.
indispensable for the inheritance and continuous The process of reusing an existing baseline project by a
improvement of the testing team's experience. new project is shown in figure 1.
The first step is to compare the user requirements of the
B. How to Reuse the Testing Assets
current project and the existing projects one by one. If the
The reuse of testing assets, applying the essence of user requirements are identical and do not involve specific
structured thinking, has a framework and a hierarchical core, design, the traceability contents and results of the general
that is to say, during the testing, the testing assets with application functional requirements, testing and validation of
different hierarchies must be reused according to the the existing projects are directly referenced; If the user
different goals. requirements are identical but involve specific design, the
The reuse of testing assets mainly include: documents general application requirements (including coverage status)
reuse, experience reuse and thinking reuse. of the existing baseline are referenced. In order to reduce the
The types of documents reuse generally include codes workload, the contents of existing specific application design
reuse, templates reuse and even deliverables reuse of existing (excluding coverage status), test records (excluding test
project. results), and validation records (excluding validation results)
The methods of reuse assets in different stages of project are also introduced, and then Conduct manual traceability
implementation are different, for example, the reuse in and verification according to current project user
planning stage is more appropriate to use document template. requirements; If only part of the user requirements are the
The verification and validation plan of new project reuses the same, the contents of the existing baseline general
verification and validation plan of existing project as application requirements (excluding coverage status),
template. If the owner has special requirements, it can be specific application design (excluding coverage status), test
slightly modified according to the actual situation; records (excluding test results), and validation records
In the reuse of system requirements stage, tools are used (excluding validation results) are referenced, and then they
to accurately identify and compare the new project are manually modified one by one according to the user
requirements and the existing project requirements matrix. If requirements of the current project Trace the contents, verify
the requirements terms are identical, the traceability, and validate them; If the same or similar requirement terms
verification and validation results of the requirements terms cannot be found, any content of the existing baseline will not
of the existing projects can be reused. If there are changes or be quoted, and the tracking content needs to be
new requirements, they can be traced and verified, tested and supplemented one by one according to the user requirements
confirmed manually; the reuse of design phase mainly refers of the current project, and the traceability, and validation
to the reuse of the same design or the same interface; the shall be carried out;
reuse of validation phase is mainly to reuse the same Step 2: If the current project uses the new version of the
requirements of the allocation of validation methods and test general application requirements for coverage traceability,
results. the function of “update and check the general application
requirements” is executed to re check the previously
III. IMPLEMENTATION SCHEME introduced requirements traceability relationship to
The reuse implementation scheme of requirements determine whether the requirements can meet the traceability
tracking matrix is as followings. coverage requirements, If a requirement of the new version
After the design liaison meeting clarifies the owner’s of the general application requirements changes compared
requirements for the new project, the requirements tracking with the previously introduced general application
matrix shall be established at the requirements stage to make requirements, the traceability relationship will not be
a requirements tracking and verify whether the user established, and the original traceability, test and validation
requirements can be met. The title of requirement tracking results need to be deleted, and the traceability, testing and
matrix is shown in Table Ċ validation can be carried out manually.

156

Authorized licensed use limited to: Carleton University. Downloaded on November 08,2020 at 05:19:00 UTC from IEEE Xplore. Restrictions apply.
 Start

Obtain reference baseline Update and check GA Update inspection and Update check and
Read baseline
requirements test outline Validating method
requirement
tracking table
GA requirements,
Exactly the same test and validation
Comparison and SA traceability methods and results Is the Is the test
Is the validation
with baseline is empty of reference baseline requirement Yes outline Yes
method changed?
requirements changed? changed?

Reference to the No No No
Exactly the same baseline of GA
and SA traceability requirements, SA
is not empty Manually trace and Manually trace and Manually trace and
design, testing, verify item by item verify item by item confirm item by item
validation methods according to actual according to actual according to the actual Yes
project project project

GA requirements,
SA design, test and
Partially same confirmation
methods of reference Requirement tracking
baseline integrity V& V

No reference to any
Totally different baseline content
End

Figure 1. The process of a new project reusing an existing baseline project

Step 3: the updated inspection of the testing outline the same (with a similarity of 50–100%) and totally different
functions shall be executed and the previously introduced (with a similarity of 0–50%). The Levenshtein Distance
testing traceability relationship shall be re-inspected to algorithm (also known as LD algorithm)[5] is adopted by the
determine whether the testing can meet the traceability text similarity algorithm. It was proposed by the Russian
coverage requirements if the testing outlines that are newer scientist Levenshtein, so it is also called Levenshtein
than the existing baseline version are used by the current Distance (LD algorithm), the primary functions of which are
project for coverage and traceability. The traceability as followings: measure the degree of differentiation between
relationship is not established, the original testing and two strings and at least how many operations are required to
validation results needs to be deleted, and the second manual convert string a to string b. These operations include adding,
testing and validation are required if the testing outlines of deleting and replacement. The primary principle is to adopt
the new version are changed from the previously introduced the idea of dynamic planning, see the details below:
testing outlines. Assuming that the lengths of the two strings are
Step 4: the updated inspection of the testing outline respectively i and j, we analyze based on a simple situation:
functions shall be executed and the previously introduced 1) If the two strings are both null, the Levenshtein
validation methods shall be re-inspected to determine Distance is 0.
whether the validation methods can meet the traceability 2) If one of the strings is null, the Levenshtein Distance
coverage requirements if the testing outlines that are newer is equal to the length of the non-null string.
than the existing baseline version are used by the current
3) If the two strings are both non-null, the minimum
project for coverage and traceability. The validation methods
are not established, the original relevant results needs to be value of the following three cases is sufficient:
deleted, and the second manual set of validation methods are a) If the Levenshtein Distances of the two strings with
required if the testing outlines of the new version are the length of i and j are known, add 1.
changed from the previously introduced testing outlines. b) If the Levenshtein Distances of the two strings with
The above steps are implemented with VBA scripts that the length of i and j-1 are known, add 1.
come with Excel tools, the core technology of which is text c) If the Levenshtein Distances of the two strings with
comparison technology. The similarity can be grouped into 3 the length of i-1 and j-1 are known, the two cases shall be
levels according to the text similarity value, such as
completely the same (with a similarity of 100%), partially

157

Authorized licensed use limited to: Carleton University. Downloaded on November 08,2020 at 05:19:00 UTC from IEEE Xplore. Restrictions apply.
considered: add 1 when the ith string is different from the The above red content means the differences between the
jth string; it needs not to add1 when they are different. two strings, and the number of red characters is just the
Example is as followings: Levenshtein Distances between the two strings.
GGATCGA and GAATTCAGTTA have a very similar
structure, and we have observed with naked eyes that one IV. ANALYSIS FOR THE CASE IMPLEMENTATION EFFECT
string can be converted into another through the following During the testing, Project 1 was used as the current
transformation, where “-”means addition. project and Project 0 was used as the baseline project.
Click “Get Reference Baseline” on Project 1’s CBI
Requirements Matrix and select Project 0’s CBI as the
reference baseline to generate the following reuse records
(the green content is completely the same, the yellow content
is partially the same, and the white content is total different).
Figure 2. Difference results There are 82 requirements and 47 green requirements,
which make full use of the project requirements traceability,
testing and validation results, as shown in Fig. 3


Figure 3. Complete reuse requirements

There are 10 orange partially same requirements, which need a second manual traceability, testing and validation, as shown
in Fig. 4


Figure 4. Partial reuse requirements

There are 25 white totally different requirements, which need a supplementary manual traceability, testing and validation,
as shown in Fig. 5

158

Authorized licensed use limited to: Carleton University. Downloaded on November 08,2020 at 05:19:00 UTC from IEEE Xplore. Restrictions apply.
 Figure 5. Non reusable requirements

Among them, the traceability records that can be fully the traceability relationship of the existing baseline
referred account for 57% (47/82 *100% = 57%), and there is requirement tracking matrix is complete and correct, the
no need for manual traceability, verification, testing and scheme can be applied in practical projects.
validation, so the efficiency is significantly improved. In
actual work, limited human resources can be concentrated on REFERENCES
the requirements traceability, verification, testing, and [1] EN 50126-1:2017 Railway applications—the specification and
validation of different content and the quality of the work demonstration of Reliability, Availability, maintainability and Safety
can be also provided. (RAMS) Part 1: Basic requirements and generic process
[2] EN 50126-2:2017 Railway applications—the specification and
V. CONCLUSION AND ANALYSIS demonstration of Reliability, Availability, maintainability and Safety
(RAMS) Part 2: Guide to the application of EN 50126-1 for safety
The above content is only to analyze and practice the [3] EN 50128: 2011 Railway application — Communication, signaling
reuse scheme of the verification and validation activities of and processing systems — Software for railway control and
the requirement tracking matrix in the whole project protection systems
implementation. The results show that the scheme can [4] EN50129: 2018 Railway application — Communication, signaling
significantly improve the efficiency and quality of and processing systems — Safety related electronic systems for
signaling
verification or validation. If the method is applied to other
verification or validation activities, such as traceability [5] LD principle and calculation formula: February 22, 2005 Bio
107/207 Winter 2005 Lecture 15 Linkage disequilibrium and
verification of test cases, it can also significantly improve the recombination
verification and accuracy Recognize efficiency. As long as

159

Authorized licensed use limited to: Carleton University. Downloaded on November 08,2020 at 05:19:00 UTC from IEEE Xplore. Restrictions apply.