Defect Analysis and Prevention For Software Process Quality Improvement
Defect Analysis and Prevention For Software Process Quality Improvement
Defect Analysis and Prevention For Software Process Quality Improvement
7, October 2010
R Baskaran
Asst. Professor, Dept. of Computer Science & Engg Anna University, Chennai.
while reducing overall costs, schedule and resources. This ensures a project can maintain cost schedule quality equilibrium. The purpose of defect prevention is to identify those defects in the beginning of the life cycle and prevent them from recurring so that the defect may not surface again. In this study, in order to improve software process quality, defects are first identified from a given set of projects, classified and analyzed for patterns. These patterns are then eliminated by finding the root causes, for which preventive mechanisms are established for reducing re-occurrences of similar defects in the subsequent projects, thus improving Quality. This Cycle will be continuous to improve Quality of the SDLC. The scope of this paper is to provide a comprehensive view on the defect prevention techniques and practices that can be followed in software development. The rest of the paper is organized as follows: Section 2 presents an overview of related work. Section 3 discusses the need for defect prevention. Section 4, presents the process improvement workflow along with the illustration of various stages. The distribution of the project defect data across project is illustrated in section 5. Section 6 presents the root cause analysis and determination of preventive action. Section 7 displays the reduction of defects in a new project that inherited the preventive ideas from old projects. Finally, in Section 8, the paper is concluded by highlighting the benefits of adopting preventive action in the subsequent project thereby improving the software process quality.
ABSTRACT
"An ounce of prevention is worth a pound of cure." In software, these expressions translate into the common observation that the longer a defect stays in process, the more expensive it is to fix [10]. Moreover software defects are expensive and time consuming. The cost of finding and correcting defects represents one of the most expensive software development activities. And that too, if the errors get carried away till the final acceptance testing stage of the project life cycle, then the project is at a greater risk in terms of its Time and Cost factors. A small amount of effort spent on quality assurance will see good amount of cost savings in terms of detecting and eliminating the defects. To gain a deeper understanding of the effectiveness of the software process, it is essential to examine the details of defects detected in the past projects and to study how the same can be eliminated due to process improvements and newer methodologies. This paper will focus on finding the total number of defects that has occurred in the software development process for five similar projects and aims at classifying various defects using first level of Orthogonal Defect Classification (ODC), finding root causes of the defects and use the learning of the projects as preventive ideas. The paper also showcases on how the preventive ideas are implemented in a new set of projects resulting in the reduction of the number of similar defects.
Keywords
Defect, Defect Analysis, Defect Prevention, Root Cause Analysis
2. RELATED WORK
The earlier studies in defect prevention were focused on defect prediction and decide upon the team size of the testing resources required in order to complete the project on time and lot of effort were utilized in the debugging and get the defects eliminated. With the advent of SDLC processes many companies formulated their own defect prevention mechanisms and many studies were conducted towards defect prediction and prevention. One study by Fang Chenbin [6] was introduction of a tool called Bug Tracing System (BTS) for defect tracing, has the advantage of popularity and low cost, and also improves the accuracy of tracking the identified defects. Work done by Stefan Wagner [9] summarizes the work on defect classification approaches that have been proposed by two companies IBM and HP. The IBM approach is called Orthogonal Defect Classification (ODC) and the HP approach is based on three dimensions -Defect Origin, Types and Modes. Pankaj Jalote and Naresh Agarwal [7] stressed on how analysis of defects found in first iteration can provide feedback for defect prevention in later iterations, leading to quality and 42
1. INTRODUCTION
Software Defect can be defined as Imperfections in software development process that would cause software to fail to meet the desired expectations. In software development, lot of defects would emerge during the development process. It is a fallacy to believe that defects get injected in the beginning of the cycle and are removed through the rest of the development process [8]. Defects occur all the way through the development process. Hence, defect prevention becomes an essential part of software process quality improvement. Defect prevention (DP) is a process of improving quality whose purpose is to identify the common causes of defects, and change the relevant process(es) to prevent that type of defect from recurring[2]. DP also increases the quality of a software product
International Journal of Computer Applications (0975 8887) Volume 8 No.7, October 2010 productivity improvement. Ajit Ashok Shenvi [1] worked under the philosophy that capturing defects in the earlier stage of the life cycle is a means of preventing defects in the later stages of the product life cycle and concentrated on finding out preventive action for functional defect types only. Suma V [11] aimed to provide information on various methods and practices supporting defect detection and prevention based on three case studies and studied about the defect detection and defect prevention strategies adopted in these three projects only. All the above methodologies lacked some dimension in the defect prevention process and needed more attention. In this study, we propose to combine the above methodologies used such as ODC, Iteration defect reduction, capturing defects at early stage and finding out defect prevention for better classified type of defects and have attempted to come out with a defect prevention cycle for continuous improvement of the Quality Processes and Defect Prevention.
43
International Journal of Computer Applications (0975 8887) Volume 8 No.7, October 2010
Figure 3: Defect - Size Correlation Figure 2: Defect Prevention in Software Lifecycle The project size can be measured either in terms of kilo lines of code (KLOC) produced or in terms of Function Point (FP). For the projects that are taken for study, the project size is measured in terms of KLOC. Comparison is then made between KLOC and number of defect produced by the project. This comparison is depicted in the above figure. From (fig 3), it is evident that, the number of defects in the project varies as the size of the project varies. Table 2: Categorization of defects across phases for five similar projects
Life cycle phases Requirements Design Code GUI Documentation Activity Review Review Testing (Function/unit) Review Review Defect Type REQ DSN LOG GUI TYP No Defects 74 58 420 55 30 of
TYP
Documentation
Proj 5
Maple - CRM module for a trading company Indesign Survey Automation Tool Stock Market Application Issue TrackerManages and maintains list of issues raised by an organization GRTNET General Reporting Tool
29
14 7 5
119 104 97
31
145
2400
0.005
Total
44
International Journal of Computer Applications (0975 8887) Volume 8 No.7, October 2010
Figure 4: Defect Type pattern across project Fig 4 illustrates the defect type pattern for five similar projects that are shown in Table 1. It is found that 70-80 percentages of defects were classified as coding defects. Approximately 10% of defects are GUI defects. Balance 10% defects are Requirements, Design and Documentation defects.
Figure 5: Defect Pareto chart The Pareto Chart shows the frequencies of occurrences of the various categories of problems encountered, in order to determine which of the existing problems occur most frequently. The problem categories or causes are shown on the x-axis of the bar graph and the cumulative percentage is shown on the y axis of the graph. From the Pareto chart, it is understood that 80% of the defect are falling under the category Logical, Requirement, Design defect type. These defect types should be given higher priority and must be attended first. Figure 6: Cause Effect Diagram for a Software Defect
International Journal of Computer Applications (0975 8887) Volume 8 No.7, October 2010 Table 5 shows possible preventive actions that were agreed by the project team members for the various root causes of the defect. Table 5: Root causes and preventive action
S.No Defect Type Root Cause Preventive Actions Domain knowledge: Training should be given to the team members before starting the next phase or a new project. The Training Calendar and the trainings attended by the team can be tracked as a part of the Project status review meeting. Make available of trained and experienced resources for coding and testing. Plan for trained resources well in advance and if they are not available train the existing resources. Any risk on the availability of the trained resources should be tracked as a part of the Risk Management Worksheet. Generally introduction of new programming language should be known well in advance to the team and proper training should be given well in advance. Discuss more about the boundary of the applications and granularity of each requirement Using Business Analysts /Domain professionals during requirement elicitation. Requirement workshop (For clarity & common understanding of implicit & explicit requirements with all teams including testing) Frequent communications with customer will help to know his requirements A formal sign off from all Business Users who would handle the application should be mandated before starting the design phase. Discuss more about the boundary of the applications and granularity of the requirement. The equivalent design conversion should be well documented in the Design Document and sign off should be received before starting the coding. Care should be taken in choosing right tool Training should be given in the usage of design tool The design document should be consistent with requirements specification. The review should be carried out with a Design review list as base and adequacy in review should be cross checked by the Quality team or Organisation Design review team. 4 GUI 1) Compatibility of browsers, supporting S/W, H/W etc. 2)Settings of the system Resolution, 3)Limitations of the Control
Most of the Graphical defects appear similar across all projects. Maintain a defect database and run test cases through it before starting up with the project A thorough check shall be done before delivering the artifact. Customer review of artifacts and deliverables
5.
TYP
Oversight
LOG
1) Lack of Domain knowledge. 2)Improper Algorithm 3) Developer without experience 4) Introduction of new programming language
DEFECT
To see the effectiveness of using the DP action, the above mentioned preventive action are implemented in the next set of five similar projects, and the process improvement was observed in terms of average defect density. Table 6: Defect Density after implementing Defect Preventive action
Second set of Project Proj. No Proj. 1 Proj. 2 eCampus HR Module Content Management System Additional Reporting MOSS based document management system UAE Finance module development R&D based BI tool 39 11 118 54 2900 925 0.003 0.005 Proj Name Description & KLOC No of Defects Efforts (P Hr) Defect Density
REQ
1) Assumption of the Requirement gathering person in the grey Area. 2) Ambiguity in requirement documentation 3)Incorrect requirement specification 4)Wrong elicitation technique 5)Not enough preparation for review by reviewers
Proj. 3
16
152
1950
0.010
Proj. 4
97
1290
0.011
Proj. 5
33
267
2450
0.008
DSN
1)Ambiguity in requirement documentation 2)Incorrect usage of design tool 3)Incomplete review 4)Inadequate participation of reviewers 5)Lack of system knowledge
Figure 7: Graph depicting Defect density comparison before and after implementation of preventive action The Graph represents the distribution of defect densities for 5 similar projects before and after implementing the Defect Prevention as provided in the Table 1 and Table 6. Trend line 46
International Journal of Computer Applications (0975 8887) Volume 8 No.7, October 2010 shows that the defect density after implementing DP is well below that of defect density before the DP implementation. The average defect density has gone down from 0.0108 (first set of projects-Table 1) to 0.0074 (second set of project Table 6). By implementing the defect preventive action, not only reduces the defect density, rework effort is also reduced due to which effort involved in various processes is also reduced considerably. [5]. Mukesh soni, 1997, Defect Prevention: Reducing cost and improving quality published in IEEE Computer, (Volume 30, Issue 8), August 1997. [6]. Pan Tiejun, Zheng Leina, Fang Chengbin, 2008, Defect Tracing System Based on Orthogonal Defect Classification published in Computer Engineering and Applications, vol 43, PP 9-10, May 2008. [7]. Pankaj Jalote, Naresh Agarwal, 2007, Using Defect Analysis Feedback for Improving Quality and Productivity in Iterative Software Development In proc- ITI 3rd International Conference on Information and Communications Technology, pp. 703-713. [8]. Ram Chillarege, Inderpal S Bhandari, Jarir K Chaar, Michael J Halliday, Diane S Moebus, Bonnie K Ray, Man-Yuen Wong, 1992, Orthogonal Defect Classification A Concept for In-Process Measurements, IEEE Transactions on Software Engineering, Vol. 18, No.11, Nov 1992. http://www.chillarege.com/odc/articles/odcconcept/odc.ht ml [9]. Stefan Wagner, 2008,Defect Classification and Defect Type Revisited Proceedings of the 2008 workshop on Defects in large software systems, (DEFECTS08) pages 73-83, ACM Press,2008 [10]. Steve McConnel, An ounce of prevention, IEEE software, May/June 2001 [11]. Suma V and T R Gopalakrishnan Nair , 2008, Effective Defect Prevention Approach in Software Process for Achieving Better Quality Levels Proceedings of World Academy of Science, Engineering and Technology Volume 32 August 2008
8. CONCLUSION
Implementation of defect preventive action not only helps to give a quality project, but it is also a valuable investment. Defect prevention practices enhance the ability of software developers to learn from those errors and, more importantly, learn from the mistakes of others. The benefits of adopting defect prevention strategy would be enormous and to list a few, Defect prevention reduces development time and cost, increases customer satisfaction, reduces rework effort, thereby decreases cost and improves product quality. This study confirms to implementation of first level of Orthogonal Defect Classification (ODC) for defect classification. To gain a deeper understanding about the defect, the defects are to be classified by implementing ODC to next level. Analysis of ODC classified data helps in getting better defect preventive ideas that would further improve the software quality process.
9. REFERENCES
[1]. Ajit Ashok Shenvi,2009,Defect Prevention with Orthogonal Defect Classification, In Proc- ISEC 09, February 23-26, 2009 [2]. Defect Prevention by SEIs CMM Model Version 1.1., http://www.dfs.mil/technology/pal/cmm/vl/dp [3]. Linda Westfall, Defect Density http://www.westfallteam.com/Papers/defect_density.pdf [4]. Megan Graham, 2005, Software Defect Prevention using Orthogonal Defect Prevention. http://twinspin.cs.umn.edufiles/ODC_TwinSPIN
47