About Acceptance Testing

In software engineering, acceptance testing is formal testing conducted to determine

whether a system satisfies its acceptance criteria and thus whether the customer should
accept the system. The main types of software testing are:

1.Component.
2.Interface.
3.System.
4.Acceptance.
5.Release.
Acceptance Testing checks the system against the "Requirements". It is similar to
systems testing in that the whole system is checked but the important difference is the
change in focus: Systems testing checks that the system that was specified has been
delivered. Acceptance Testing checks that the system delivers what was requested.

The customer and not the developer should always do acceptance testing.

The customer knows what is required from the system to achieve value in the business
and is the only person qualified to make that judgment. The forms of the tests may follow
those in system testing, but at all times they are informed by the business needs.

The test procedures that lead to formal 'acceptance' of new or changed systems. User
Acceptance Testing is a critical phase of any 'systems' project and requires significant
participation by the 'End Users'. To be of real use, an Acceptance Test Plan should be
developed in order to plan precisely, and in detail, the means by which 'Acceptance' will
be achieved. The final part of the UAT can also include a parallel run to prove the system
against the current system.

Factors influencing Acceptance Testing

The User Acceptance Test Plan will vary from system to system but, in general, the
testing should be planned in order to provide a realistic and adequate exposure of the
system to all reasonably expected events. The testing can be based upon the User
Requirements Specification to which the system should conform.

As in any system though, problems will arise and it is important to have determined what
will be the expected and required responses from the various parties concerned; including
Users; Project Team; Vendors and possibly Consultants / Contractors.

In order to agree what such responses should be, the End Users and the Project Team
need to develop and agree a range of 'Severity Levels'. These levels will range from (say)
1 to 6 and will represent the relative severity, in terms of business / commercial impact,
of a problem with the system, found during testing. Here is an example which has been
used successfully; '1' is the most severe; and '6' has the least impact :
1.'Show Stopper' i.e. it is impossible to continue with the testing because of the severity
of this error / bug
2.Critical Problem; testing can continue but we cannot go into production (live) with
this problem
3.Major Problem; testing can continue but live this feature will cause severe disruption
to business processes in live operation
4.Medium Problem; testing can continue and the system is likely to go live with only
minimal departure from agreed business processes Minor Problem ; both testing and live
operations may progress. This problem should be corrected, but little or no changes to
business processes are envisaged
5.'Cosmetic' Problem e.g. colors; fonts; pitch size However, if such features are key to
the business requirements they will warrant a higher severity level. The users of the
system, in consultation with the executive sponsor of the project, must then agree upon
the responsibilities and required actions for each category of problem. For example, you
may demand that any problems in severity level 1, receive priority response and that all
testing will cease until such level 1 problem are resolved.

Even where the severity levels and the responses to each have been agreed by all parties;
the allocation of a problem into its appropriate severity level can be subjective and open
to question. To avoid the risk of lengthy and protracted exchanges over the categorization
of problems; we strongly advised that a range of examples are agreed in advance to
ensure that there are no fundamental areas of disagreement; or if there are, these will be
known in advance and your organization is forewarned.

Finally, it is crucial to agree the Criteria for Acceptance. Because no system is entirely
fault free, it must be agreed between End User and vendor, the maximum number of
acceptable 'outstanding' in any particular category. Again, prior consideration of this is
advisable.

In some cases, users may agree to accept ('sign off') the system subject to a range of
conditions. These conditions need to be analyzed as they may, perhaps unintentionally,
seek additional functionality which could be classified as scope creep. In any event, any
and all fixes from the software developers, must be subjected to rigorous System Testing
and, where appropriate Regression Testing.
Conclusion
Hence the goal of acceptance testing should verify the overall quality, correct operation,
scalability, completeness, usability, portability, and robustness of the functional
components supplied by the Software system.

Brief about Integration Testing

One of the most significant aspects of a software development project is the integration
strategy. Integration may be performed all at once, top-down, bottom-up, critical piece
first, or by first integrating functional subsystems and then integrating the subsystems in
separate phases using any of the basic strategies. In general, the larger the project, the
more important the integration strategy.

Very small systems are often assembled and tested in one phase. For most real systems,
this is impractical for two major reasons. First, the system would fail in so many places at
once that the debugging and retesting effort would be impractical.

Second, satisfying any white box testing criterion would be very difficult, because of the
vast amount of detail separating the input data from the individual code modules. In fact,
most integration testing has been traditionally limited to "black box" techniques.

Large systems may require many integration phases, beginning with assembling modules
into low-level subsystems, then assembling subsystems into larger subsystems, and
finally assembling the highest level subsystems into the complete system.

To be most effective, an integration testing technique should fit well with the overall
integration strategy. In a multi-phase integration, testing at each phase helps detect errors
early and keep the system under control. Performing only cursory testing at early
integration phases and then applying a more rigorous criterion for the final stage is really
just a variant of the high-risk "big bang" approach. However, performing rigorous testing
of the entire software involved in each integration phase involves a lot of wasteful
duplication of effort across phases. The key is to leverage the overall integration structure
to allow rigorous testing at each phase while minimizing duplication of effort.

It is important to understand the relationship between module testing and integration

testing. In one view, modules are rigorously tested in isolation using stubs and drivers
before any integration is attempted. Then, integration testing concentrates entirely on
module interactions, assuming that the details within each module are accurate. At the
other extreme, module and integration testing can be combined, verifying the details of
each module's implementation in an integration context. Many projects compromise,
combining module testing with the lowest level of subsystem integration testing, and then
performing pure integration testing at higher levels. Each of these views of integration
testing may be appropriate for any given project, so an integration testing method should
be flexible enough to accommodate them all. Combining module testing with bottom-up
integration.
Generalization of module testing criteria

Module testing criteria can often be generalized in several possible ways to support
integration testing. As discussed in the previous subsection, the most obvious
generalization is to satisfy the module testing criterion in an integration context, in effect
using the entire program as a test driver environment for each module. However, this
trivial kind of generalization does not take advantage of the differences between module
and integration testing. Applying it to each phase of a multi-phase integration strategy,
for example, leads to an excessive amount of redundant testing.
More useful generalizations adapt the module testing criterion to focus on interactions
between modules rather than attempting to test all of the details of each module's
implementation in an integration context. The statement coverage module testing
criterion, in which each statement is required to be exercised during module testing, can
be generalized to require each module call statement to be exercised during integration
testing. Although the specifics of the generalization of structured testing are more
detailed, the approach is the same. Since structured testing at the module level requires
that all the decision logic in a module's control flow graph be tested independently, the
appropriate generalization to the integration level requires that just the decision logic
involved with calls to other modules be tested independently.

Module design complexity

Rather than testing all decision outcomes within a module independently, structured
testing at the integration level focuses on the decision outcomes that are involved with
module calls. The design reduction technique helps identify those decision outcomes, so
that it is possible to exercise them independently during integration testing. The idea
behind design reduction is to start with a module control flow graph, remove all control
structures that are not involved with module calls, and then use the resultant "reduced"
flow graph to drive integration testing. Below figure shows a systematic set of rules for
performing design reduction. Although not strictly a reduction rule, the call rule states
that function call ("black dot") nodes cannot be reduced. The remaining rules work
together to eliminate the parts of the flow graph that are not involved with module calls.
The sequential rule eliminates sequences of non-call ("white dot") nodes. Since
application of this rule removes one node and one edge from the flow graph, it leaves the
cyclomatic complexity unchanged. However, it does simplify the graph so that the other
rules can be applied.

The repetitive rule eliminates top-test loops that are not involved with module calls. The
conditional rule eliminates conditional statements that do not contain calls in their bodies.
The looping rule eliminates bottom-test loops that are not involved with module calls. It
is important to preserve the module's connectivity when using the looping rule, since for
poorly-structured code it may be hard to distinguish the "top" of the loop from the
"bottom."

For the rule to apply there must be a path from the module entry to the top of the loop and
a path from the bottom of the loop to the module exit. Since the repetitive, conditional,
and looping rules each remove one edge from the flow graph, they each reduce
cyclomatic complexity by one.

Rules 1 through 4 are intended to be applied iteratively until none of them can be applied,
at which point the design reduction is complete. By this process, even very complex logic
can be eliminated as long as it does not involve any module calls.
Brief about GUI Testing

What is GUI Testing?

GUI is the abbreviation for Graphic User Interface. It is absolutely essential that any
application has to be user-friendly. The end user should be comfortable while using all
the components on screen and the components should also perform their functionality
with utmost clarity. Hence it becomes very essential to test the GUI components of any
application. GUI Testing can refer to just ensuring that the look-and-feel of the
application is acceptable to the user, or it can refer to testing the functionality of each and
every component involved.

The following is a set of guidelines to ensure effective GUI Testing and can be used even
as a checklist while testing a product/application.

Windows Compliance Testing

Application Start Application by Double Clicking on its ICON. The Loading message
should show the application name, version number, and a bigger pictorial representation
of the icon. No Login is necessary. The main window of the application should have the
same caption as the caption of the icon in Program Manager. Closing the application
should result in an "Are you Sure" message box .Try to start the application twice as it is
loading. On each window, if the application is busy, then the hour glass should be
displayed. If there is no hour glass, then some enquiry in progress message should be
displayed. All screens should have a Help button (i.e.) F1 key should work the same.

If Window has a Minimize Button, click it. Window should return to an icon on the
bottom of the screen. This icon should correspond to the Original Icon under Program
Manager. Double Click the Icon to return the Window to its original size. The window
caption for every application should have the name of the application and the window
name - especially the error messages. These should be checked for spelling, English and
clarity, especially on the top of the screen. Check does the title of the window make
sense. If the screen has a Control menu, then use all un-grayed options.

Check all text on window for Spelling/Tense and Grammar. Use TAB to move focus
around the Window. Use SHIFT+TAB to move focus backwards. Tab order should be
left to right, and Up to Down within a group box on the screen. All controls should get
focus - indicated by dotted box, or cursor. Tabbing to an entry field with text in it should
highlight the entire text in the field. The text in the Micro Help line should change -
Check for spelling, clarity and non-updateable etc. If a field is disabled (grayed) then it
should not get focus. It should not be possible to select them with either the mouse or by
using TAB. Try this for every grayed control.

Never updateable fields should be displayed with black text on a gray background with a
black label. All text should be left justified, followed by a colon tight to it. In a field that
may or may not be updateable, the label text and contents changes from black to gray
depending on the current status. List boxes are always white background with black text
whether they are disabled or not. All others are gray.

In general, double-clicking is not essential. In general, everything can be done using both
the mouse and the keyboard. All tab buttons should have a distinct letter.

Text Boxes

Move the Mouse Cursor over all Enterable Text Boxes. Cursor should change from arrow
to Insert Bar. If it doesn't then the text in the box should be gray or non-updateable. Refer
to previous page. Enter text into Box Try to overflow the text by typing to many
characters - should be stopped Check the field width with capitals W. Enter invalid
characters - Letters in amount fields, try strange characters like + , - * etc. in All fields.
SHIFT and Arrow should Select Characters. Selection should also be possible with
mouse. Double Click should select all text in box.

Option (Radio Buttons)

Left and Right arrows should move 'ON' Selection. So should Up and Down. Select with
mouse by clicking.

Check Boxes

Clicking with the mouse on the box, or on the text should SET/UNSET the box. SPACE
should do the same.

Command Buttons

If Command Button leads to another Screen, and if the user can enter or change details on
the other screen then the Text on the button should be followed by three dots. All Buttons
except for OK and Cancel should have a letter Access to them. This is indicated by a
letter underlined in the button text. Pressing ALT+Letter should activate the button. Make
sure there is no duplication. Click each button once with the mouse - This should activate
Tab to each button - Press SPACE - This should activate Tab to each button - Press
RETURN - This should activate The above are VERY IMPORTANT, and should be
done for EVERY command Button. Tab to another type of control (not a command
button). One button on the screen should be default (indicated by a thick black border).
Pressing Return in ANY no command button control should activate it. If there is a
Cancel Button on the screen, then pressing should activate it. If pressing the Command
button results in uncorrectable data e.g. closing an action step, there should be a message
phrased positively with Yes/No answers where Yes results in the completion of the
action.

Drop Down List Boxes

Pressing the Arrow should give list of options. This List may be scrollable. You should
not be able to type text in the box. Pressing a letter should bring you to the first item in
the list with that start with that letter. Pressing Ctrl - F4 should open/drop down the list
box. Spacing should be compatible with the existing windows spacing (word etc.). Items
should be in alphabetical order with the exception of blank/none, which is at the top or
the bottom of the list box. Drop down with the item selected should be display the list
with the selected item on the top. Make sure only one space appears, shouldn't have a
blank line at the bottom.

Combo Boxes

Should allow text to be entered. Clicking Arrow should allow user to choose from list

List Boxes

Should allow a single selection to be chosen, by clicking with the mouse, or using the Up
and Down Arrow keys. Pressing a letter should take you to the first item in the list
starting with that letter. If there is a 'View' or 'Open' button besides the list box then
double clicking on a line in the List Box, should act in the same way as selecting and
item in the list box, then clicking the command button. Force the scroll bar to appear,
make sure all the data can be seen in the box.

Screen Validation Checklist

Aesthetic Conditions:

• Is the general screen background the correct color?

• Are the field prompts the correct color?
• Are the field backgrounds the correct color?
• In read-only mode, are the field prompts the correct color?
• In read-only mode, are the field backgrounds the correct color?
• Are all the screen prompts specified in the correct screen font?
• Is the text in all fields specified in the correct screen font?
• Are all the field prompts aligned perfectly on the screen?
• Are all the field edits boxes aligned perfectly on the screen?
• Are all group boxes aligned correctly on the screen?
• Should the screen be resizable?
• Should the screen be allowed to minimize?
• Are all the field prompts spelt correctly?
• Are all characters or alphanumeric fields left justified? This is the default unless
otherwise specified.
• Are all numeric fields right justified? This is the default unless otherwise
specified.
• Is all the micro-help text spelt correctly on this screen?
• Is all the error message text spelt correctly on this screen? Is all user input
captured in UPPER case or lowercase consistently?
• Where the database requires a value (other than null) then this should be defaulted
into fields. The user must either enter an alternative valid value or leave the
default value intact.
• Assure that all windows have a consistent look and feel.
• Assure that all dialog boxes have a consistent look and feel.

Validation Conditions:

• Does a failure of validation on every field cause a sensible user error message?
• Is the user required to fix entries, which have failed validation tests?
• Have any fields got multiple validation rules and if so are all rules being applied?
• If the user enters an invalid value and clicks on the OK button (i.e. does not TAB
off the field) is the invalid entry identified and highlighted correctly with an error
message?
• Is validation consistently applied at screen level unless specifically required at
field level? For all numeric fields check whether negative numbers can and should
be able to be entered.
 • For all numeric fields check the minimum and maximum values and also some
mid-range values allowable?
• For all character/alphanumeric fields check the field to ensure that there is a
character limit specified and that this limit is exactly correct for the specified
database size?
• Do all mandatory fields require user input?
• If any of the database columns don't allow null values then the corresponding
screen fields must be mandatory. (If any field, which initially was mandatory, has
become optional then check whether null values are allowed in this field.)

Navigation Conditions:

• Can the screen be accessed correctly from the menu?

• Can the screen be accessed correctly from the toolbar?
• Can the screen be accessed correctly by double clicking on a list control on the
previous screen? Can all screens accessible via buttons on this screen be accessed
correctly?
• Can all screens accessible by double clicking on a list control be accessed
correctly?
• Is the screen modal? (i.e.) Is the user prevented from accessing other functions
when this screen is active and is this correct?
• Can a number of instances of this screen be opened at the same time and is this
correct?
• Top of Form
Usability Conditions:

• Are all the dropdowns on this screen sorted correctly? Alphabetic sorting is the
default unless otherwise specified.
• Is all date entry required in the correct format?
• Do the Shortcut keys work correctly?
• Have the menu options that apply to your screen got fast keys associated and
should they have?
• Does the Tab Order specified on the screen go in sequence from Top Left to
bottom right? This is the default unless otherwise specified.
• Are all read-only fields avoided in the TAB sequence?
• Are all disabled fields avoided in the TAB sequence?
• Can the cursor be placed in the microhelp text box by clicking on the text box
with the mouse?
• Can the cursor be placed in read-only fields by clicking in the field with the
mouse?
• Is the cursor positioned in the first input field or control when the screen is
opened?
• Is there a default button specified on the screen? Does the default button work
correctly?
 • When an error message occurs does the focus return to the field in error when the
user cancels it?
• When the user Alt+Tab's to another application does this have any impact on the
screen upon return to the application?
• Do all the fields edit boxes indicate the number of characters they will hold by
there length? e.g. a 30 character field should be a lot longer .
• Have all pushbuttons on the screen been given appropriate Shortcut keys?

Data Integrity Conditions:

• Is the data saved when the window is closed by double clicking on the close box?
• Check the maximum field lengths to ensure that there are no truncated characters?
• Where the database requires a value (other than null) then this should be defaulted
into fields. The user must either enter an alternative valid value or leave the
default value intact.
• Check maximum and minimum field values for numeric fields?
• If numeric fields accept negative values can these be stored correctly on the
database and does it make sense for the field to accept negative numbers?
• If a set of radio buttons represents a fixed set of values such as A, B and C then
what happens if a blank value is retrieved from the database? (In some situations
rows can be created on the database by other functions, which are not screen
based, and thus the required initial values can be incorrect.)
• If a particular set of data is saved to the database check that each value gets saved
fully to the database. (i.e.) Beware of truncation (of strings) and rounding of
numeric values.

Modes (Editable Read-only) Conditions:

• Are the screen and field colors adjusted correctly for read-only mode?
• Should a read-only mode be provided for this screen?
• Are all fields and controls disabled in read-only mode?
• Can the screen be accessed from the previous screen/menu/toolbar in read-only
mode?
• Can all screens available from this screen be accessed in read-only mode?
• Check that no validation is performed in read-only mode.

General Conditions:

• Assure the existence of the "Help" menu.

• Assure that the proper commands and options are in each menu.
• Assure that all buttons on all tool bars have corresponding key commands.
• Assure that each menu command has an alternative (hot-key) key sequence,
which will invoke it where appropriate.
• In drop down list boxes, ensure that the names are not abbreviations / cut short
• In drop down list boxes, assure that the list and each entry in the list can be
accessed via appropriate key / hot key combinations.
• Ensure that duplicate hot keys do not exist on each screen
• Ensure the proper usage of the escape key (which is to undo any changes that
have been made) and generates a caution message "Changes will be lost -
Continue yes/no"
• Assure that the cancel button functions the same as the escape key.
• Assure that the Cancel button operates, as a Close button when changes have been
made that cannot be undone. Assure that only command buttons, which are used
by a particular window, or in a particular dialog box, are present. – (i.e) make sure
they don't work on the screen behind the current screen.
• When a command button is used sometimes and not at other times, assures that it
is grayed out when it should not be used.
• Assure that OK and Cancel buttons are grouped separately from other command
buttons.
• Assure that command button names are not abbreviations.
• Assure that all field labels/names are not technical labels, but rather are names
meaningful to system users.
• Assure that command buttons are all of similar size and shape, and same font &
font size.
• Assure that each command button can be accessed via a hot key combination.
• Assure that command buttons in the same window/dialog box do not have
duplicate hot keys.
• Assure that each window/dialog box has a clearly marked default value
(command button, or other object) which is invoked when the Enter key is pressed
- and NOT the Cancel or Close button Assure that focus is set to an object/button,
which makes sense according to the function of the window/dialog box.
• Assure that all option buttons (and radio buttons) names are not abbreviations.

General Conditions (contd.):

• Assure that option button names are not technical labels, but rather are names
meaningful to system users. If hot keys are used to access option buttons, assure
that duplicate hot keys do not exist in the same window/dialog box.
• Assure that option box names are not abbreviations.
• Assure that option boxes, option buttons, and command buttons are logically
grouped together in clearly demarcated areas "Group Box"
• Assure that the Tab key sequence, which traverses the screens, does so in a logical
way.
• Assure consistency of mouse actions across windows. Assure that the color red is
not used to highlight active objects (many individuals are red-green color blind).
• Assure that the user will have control of the desktop with respect to general color
and highlighting (the application should not dictate the desktop background
characteristics).
 • Assure that the screen/window does not have a cluttered appearance
• Ctrl + F6 opens next tab within tabbed window
• Shift + Ctrl + F6 opens previous tab within tabbed window
• Tabbing will open next tab within tabbed window if on last field of current tab
• Tabbing will go onto the 'Continue' button if on last field of last tab within tabbed
window
• Tabbing will go onto the next editable field in the window
• Banner style & size & display exact same as existing windows
• If 8 or less options in a list box, display all options on open of list box - should be
no need to scroll
• Errors on continue will cause user to be returned to the tab and the focus should
be on the field causing the error. (i.e the tab is opened, highlighting the field with
the error on it)
• Pressing continue while on the first tab of a tabbed window (assuming all fields
filled correctly) will not open all the tabs.
• On open of tab focus will be on first editable field
• All fonts to be the same
• Alt+F4 will close the tabbed window and return you to main screen or previous
screen (as appropriate), generating "changes will be lost" message if necessary.
• Micro help text for every enabled field & button
• Ensure all fields are disabled in read-only mode
• Progress messages on load of tabbed screens
• Return operates continue
• If retrieve on load of tabbed window fails window should not open

Specific Field Tests

Date Field Checks

• Assure that leap years are validated correctly & do not cause
errors/miscalculations.
• Assure that month code 00 and 13 are validated correctly & do not cause
errors/miscalculations.
• Assure that 00 and 13 are reported as errors.
• Assure that day values 00 and 32 are validated correctly & do not cause
errors/miscalculations.
• Assure that Feb. 28, 29, 30 are validated correctly & do not cause errors/
miscalculations.
• Assure that Feb. 30 is reported as an error.
• Assure that century change is validated correctly & does not cause errors/
miscalculations.
 • Assure that out of cycle dates are validated correctly & do not cause
errors/miscalculations.

Numeric Fields

• Assure that lowest and highest values are handled correctly.

• Assure that invalid values are logged and reported.
• Assure that valid values are handles by the correct procedure. Assure that numeric
fields with a blank in position 1 are processed or reported as an error.
• Assure that fields with a blank in the last position are processed or reported as an
error an error.
• Assure that both + and - values are correctly processed.
• Assure that division by zero does not occur.
• Include value zero in all calculations.
• Include at least one in-range value. Include maximum and minimum range values.
• Include out of range values above the maximum and below the minimum.
• Assure that upper and lower values in ranges are handled correctly.

Alpha Field Checks

• Use blank and non-blank data.

• Include lowest and highest values. Include invalid characters & symbols.
• Include valid characters. Include data items with first position blank.
• Include data items with last position blank.

Validation Testing - Standard Actions

Examples of Standard Actions - Substitute your specific commands

• Add
• View
• Change
• Delete
• Continue - (i.e. continue saving changes or additions)
• Add
• View
• Change
• Delete
• Cancel - (i.e. abandon changes or additions)
• Fill each field - Valid data
• Fill each field - Invalid data
• Different Check Box / Radio Box combinations
 • Scroll Lists / Drop Down List Boxes
• Help
• Fill Lists and Scroll
• Tab
• Tab Sequence
• Shift Tab

Shortcut Keys / Hot Keys

Note: The following keys are used in some windows applications, and are included as a
guides

Key No Modifier Shift CTRL ALT

F1 Help Enter Help Mode N/A N/A
F2 N/A N/A N/A N/A
F3 N/A N/A N/A N/A
F4 N/A N/A Close Document / Close Application
Child Window
F5 N/A N/A N/A N/A
F6 N/A N/A N/A N/A
F7 N/A N/A N/A N/A
F8 Toggle extend Toggle Add N/A N/A
mode, if mode, if
supported. supported
F9 N/A N/A N/A N/A
F10 Toggle menu N/A N/A N/A
bar activation.
F11, N/A N/A N/A N/A
F12
TAB Move to next Move to previous Move to next open Switch to previously
active / editable active/editable Document or Child used application.
field field. window. (Adding (Holding down the ALT
SHIFT reverses the key displays all open
order of movement). applications).
Alt Puts focus on N/A N/A N/A
first menu
command (e.g.
'File').
Control Shortcut Keys
Key Function
CTRL + Z Undo
CTRL + x Cut
CTRL + C Copy
CTRL + Z Paste
CTRL + N New
CTRL + O Open
CTRL + P Print
CTRL + S Save
CTRL + B Bold
CTRL + I Italic
CTRL + U Underline

Brief about Regression Testing

What is regression testing?

• Regression testing is the process of testing changes to computer programs to make

sure that the older programming still works with the new changes.
• Regression testing is a normal part of the program development process. Test
department coders develop code test scenarios and exercises that will test new
units of code after they have been written.
• Before a new version of a software product is released, the old test cases are run
against the new version to make sure that all the old capabilities still work. The
reason they might not work because changing or adding new code to a program
can easily introduce errors into code that is not intended to be changed.
• The selective retesting of a software system that has been modified to ensure that
any bugs have been fixed and that no other previously working functions have
failed as a result of the reparations and that newly added features have not created
problems with previous versions of the software. Also referred to as verification
testing.
• Regression testing is initiated after a programmer has attempted to fix a
recognized problem or has added source code to a program that may have
inadvertently introduced errors.
• It is a quality control measure to ensure that the newly modified code still
complies with its specified requirements and that unmodified code has not been
affected by the maintenance activity.

Test Execution
Test Execution is the heart of the testing process. Each time your application changes,
you will want to execute the relevant parts of your test plan in order to locate defects and
assess quality.

Create Test Cycles

During this stage you decide the subset of tests from your test database you want to
execute.

Usually you do not run all the tests at once. At different stages of the quality assurance
process, you need to execute different tests in order to address specific goals. A related
group of tests is called a test cycle, and can include both manual and automated tests

Example: You can create a cycle containing basic tests that run on each build of the
application throughout development. You can run the cycle each time a new build is
ready, to determine the application's stability before beginning more rigorous testing.

Example: You can create another set of tests for a particular module in your application.
This test cycle includes tests that check that module in depth to decide which test cycles
to build, refer to the testing goals you defined at the beginning of the process. Also
consider issues such as the current state of the application and whether new functions
have been added or modified.

Following are examples of some general categories of test cycles to consider:

sanity cycle checks the entire system at a basic level (breadth, rather than depth) to see
that it is functional and stable. This cycle should include basic-level tests containing
mostly positive checks.
normal cycle tests the system a little more in depth than the sanity cycle. This cycle can
group medium-level tests, containing both positive and negative checks.
advanced cycle tests both breadth and depth. This cycle can be run when more time is
available for testing. The tests in the cycle cover the entire application (breadth), and also
test advanced options in the application (depth).
regression cycle tests maintenance builds. The goal of this type of cycle is to verify that
a change to one part of the software did not break the rest of the application. A regression
cycle includes sanity-level tests for testing the entire software, as well as in-depth tests
for the specific area of the application that was modified.
Run Test Cycles (Automated & Manual Tests)

Once you have created cycles that cover your testing objectives, you begin executing the
tests in the cycle. You perform manual tests using the test steps. Testing Tools executes
automated tests for you. A test cycle is complete only when all tests-automatic and
manual-have been run.
With Manual Test Execution you follow the instructions in the test steps of each test. You
use the application, enter input, compare the application output with the expected output,
and log the results. For each test step you assign either pass or fail status.
During Automated Test Execution you create a batch of tests and launch the entire batch
at once. Testing Tools runs the tests one at a time. It then imports results, providing
outcome summaries for each test.

Analyze Test Results

After every test run one analyze and validate the test results. And have to identify all the
failed steps in the tests and to determine whether a bug has been detected, or if the
expected result needs to be updated.

Change Request

Initiating a Change Request: A user or developer wants to suggest a modification that

would improve an existing application, notices a problem with an application, or wants to
recommend an enhancement. Any major or minor request is considered a problem with
an application and will be entered as a change request.

Type of Change Request

1.Bug the application works incorrectly or provides incorrect information. (for example,
a letter is allowed to be entered in a number field)
2.Change a modification of the existing application. (for example, sorting the files
alphabetically by the second field rather than numerically by the first field makes them
easier to find)
3.Enhancement new functionality or item added to the application. (for example, a new
report, a new field, or a new button)

Priority for the request

1.Low the application works but this would make the function easier or more user
friendly.
2.High the application works, but this is necessary to perform a job.
3.Critical the application does not work, job functions are impaired and there is no work
around. This also applies to any Section 508 infraction.

Bug Tracking
1. Locating and repairing software bugs is an essential part of software development.
2. Bugs can be detected and reported by engineers, testers, and end-users in all
phases of the testing process.
3. Information about bugs must be detailed and organized in order to schedule bug
fixes and determine software release dates.
Bug Tracking involves two main stages: Reporting and Tracking.
Report Bugs

Once you execute the manual and automated tests in a cycle, you report the bugs (or
defects) that you detected. The bugs are stored in a database so that you can manage them
and analyze the status of your application.

When you report a bug, you record all the information necessary to reproduce and fix it.
You also make sure that the QA and development personnel involved in fixing the bug
are notified.

Track and Analyze Bugs

The lifecycle of a bug begins when it is reported and ends when it is fixed, verified, and
closed.

1. First you report New bugs to the database, and provide all necessary information
to reproduce, fix, and follow up the bug.
2. The Quality Assurance manager or Project manager periodically reviews all New
bugs and decides which should be fixed. These bugs are given the status Open
and are assigned to a member of the development team.
3. Software developers fix the Open bugs and assign them the status Fixed.
4. QA personnel test a new build of the application. If a bug does not reoccur, it is
Closed. If a bug is detected again, it is reopened.

Communication is an essential part of bug tracking; all members of the development and
quality assurance team must be well informed in order to insure that bugs information is
up to date and that the most important problems are addressed.

The number of open or fixed bugs is a good indicator of the quality status of your
application. You can use data analysis tools such as re-ports and graphs in interpret bug
data.

Brief About System Testing

For most organizations, software and system testing represents a significant element of a
project's cost in terms of money and management time. Making this function more
effective can deliver a range of benefits including reductions in risk, development costs
and improved 'time to market' for new systems.

Systems with software components and software-intensive systems are more and more
complex everyday. Industry sectors such as telecom, automotive, railway, and
aeronautical and space, are good examples. It is often agreed that testing is essential to
manufacture reliable products. However, the validation process does not often receive the
required attention. Moreover, the validation process is close to other activities such as
conformance, acceptance and qualification testing.
The difference between function testing and system testing is that now the focus is on the
whole application and its environment . Therefore the program has to be given
completely. This does not mean that now single functions of the whole program are
tested,because this would be too redundant. The main goal is rather to demonstrate the
discrepancies of the product from its requirements and its documentation. In other words,
this again includes the question, ``Did we build the right product?'' and not just, Did we
build the product right?''

However, system testing does not only deal with this more economical problem,

it also contains some aspects that are orientated on the word ``system'' . This means that
those tests should be done in the environment for which the program was designed, like a
mulituser network or whetever. Even security guide lines have to be included. Once
again, it is beyond doubt that this test cannot be done completely, and nevertheless, while
this is one of the most incomplete test methods, it is one of the most important.

A number of time-domain software reliability models attempt to predict the growth of a

system's reliability during the system test phase of the development life cycle. In this
paper we examine the results of applying several types of Poisson-process models to the
development of a large system for which system test was performed in two parallel
tracks, using different strategies for test data selection.

We will test that the functionality of your systems meets with your specifications,
integrating with which-ever type of development methodology you are applying. We test
for errors that users are likely to make as they interact with the application as well as your
applications ability to trap errors gracefully. These techniques can be applied flexibly,
whether testing a financial system, e-commerce, an online casino or games testing.

System Testing is more than just functional testing, however, and can, when appropriate,
also encompass many other types of testing, such as:

1. security
2. load/stress
3. performance
4. browser compatibility
5. localization

Need for System Testing Effective software testing, as a part of software engineering, has
been proven over the last 3 decades to deliver real business benefits including:

Reduction of costs Reduce rework and support overheads

Increased productivity More effort spent on developing new
functionality and less on "bug fixing" as
 quality increases
If it goes wrong, what is the potential impact
Reduce commercial on your commercial goals? Knowledge is
risks power, so why take a leap of faith while your
competition step forward with confidence?

These benefits are achieved as a result of some fundamental principles of testing

for example, increased independence naturally increases objectivity.

Your test strategy must take into consideration the risks to your organization, commercial
and technical. You will have a personal interest in its success in which case it is only
human for your objectivity to be compromised.

System Testing Techniques

1. Goal is to evaluate the system as a whole, not its parts

2. Techniques can be structural or functional
3. Techniques can be used in any stage that tests the system as a whole (acceptance,
installation, etc.)
4. Techniques not mutually exclusive
5. Structural techniques
6. Stress testing - test larger-than-normal capacity in terms of transactions, data,
users, speed, etc.
7. Execution testing- test performance in terms of speed, precision, etc.
8. Recovery testing - test how the system recovers from a disaster, how it handles
corrupted data, etc.
9. Operations testing - test how the system fits in with existing operations and
procedures in the user organization
10. Compliance testing - test adherence to standards
11. Security testing - test security requirements
12. Functional techniques
13. Requirements testing - fundamental form of testing - makes sure the system does
what its required to do
14. Regression testing - make sure unchanged functionality remains unchanged
15. Error-handling testing - test required error-handling functions (usually user error)
16. Manual-support testing - test that the system can be used properly - includes user
documentation
17. Intersystem handling testing - test that the system is compatible with other
systems in the environment
18. Control testing - test required control mechanisms
19. Parallel testing - feed same input into two versions of the system to make sure
they produce the same output.

Functional techniques
 1. Input domain testing - pick test cases representative of the range of allowable
input, including high, low, and average values
2. Equivalence partitioning - partition the range of allowable input so that the
program is expected to behave similarly for all inputs in a given partition, then
pick a test case from each partition
3. Boundary value - choose test cases with input values at the boundary (both inside
and outside) of the allowable range
4. Syntax checking - choose test cases that violate the format rules for input
5. Special values - design test cases that use input values that represent special
situations
7. Output domain testing - pick test cases that will produce output at the extremes of
the output domain
8. Structural techniques
9. Statement testing - ensure the set of test cases exercises every statement at least
once
10. Branch testing - each branch of an if/then statement is exercised
11. Conditional testing - each truth statement is exercised both true and false
12. Expression testing - every part of every expression is exercised
13. Path testing - every path is exercised (impossible in practice)
14. Error-based techniques

Basic idea is that if you know something about the nature of the defects in the code, you
can estimate whether or not youve found all of them or not

1. Fault seeding - put a certain number of known faults into the code, then test until
they are all found
2. Mutation testing - create mutants of the program by making single changes, then
run test cases until all mutants have been killed
3. Historical test data - an organization keeps records of the average numbers of
defects in the products it produces, then tests a new product until the number of
defects found approaches the expected number

Conclusion:

Hence the system Test phase should begin once modules are integrated enough to
perform tests in a whole system environment. System testing can occur in parallel with
integration test, especially with the top-down method.

Brief about Unit Testing

Unit testing. Isn't that some annoying requirement that we're going to ignore? Many
developers get very nervous when you mention unit tests. Usually this is a vision of a
grand table with every single method listed, along with the expected results and pass/fail
date. It's important, but not relevant in most programming projects.
The unit test will motivate the code that you write. In a sense, it is a little design
document that says, "What will this bit of code do?" Or, in the language of object
oriented programming, "What will these clusters of objects do?"

The crucial issue in constructing a unit test is scope. If the scope is too narrow, then the
tests will be trivial and the objects might pass the tests, but there will be no design of
their interactions. Certainly, interactions of objects are the crux of any object oriented
design.

Likewise, if the scope is too broad, then there is a high chance that not every component
of the new code will get tested. The programmer is then reduced to testing-by-poking-
around, which is not an effective test strategy.

Need of Unit Testing

How do you know that a method doesn't need a unit test? First, can it be tested by
inspection? If the code is simple enough that the developer can just look at it and verify
its correctness then it is simple enough to not require a unit test. The developer should
know when this is the case.

Unit tests will most likely be defined at the method level, so the art is to define the unit
test on the methods that cannot be checked by inspection. Usually this is the case when
the method involves a cluster of objects. Unit tests that isolate clusters of objects for
testing are doubly useful, because they test for failures, and they also identify those
segments of code that are related. People who revisit the code will use the unit tests to
discover which objects are related, or which objects form a cluster. Hence: Unit tests
isolate clusters of objects for future developers.

Another good litmus test is to look at the code and see if it throws an error or catches an
error. If error handling is performed in a method, then that method can break. Generally,
any method that can break is a good candidate for having a unit test, because it may break
at some time, and then the unit test will be there to help you fix it.

The danger of not implementing a unit test on every method is that the coverage may be
incomplete. Just because we don't test every method explicitly doesn't mean that methods
can get away with not being tested. The programmer should know that their unit testing is
complete when the unit tests cover at the very least the functional requirements of all the
code. The careful programmer will know that their unit testing is complete when they
have verified that their unit tests cover every cluster of objects that form their application

Life Cycle Approach to Testing

Testing will occur throughout the project lifecycle i.e., from Requirements till User
Acceptance Testing. The main Objective to Unit Testing is as follows:
 1. To execute a program with the intent of finding an error;
2. To uncover an as-yet undiscovered error and
3. Prepare a test case with a high probability of finding an as-yet undiscovered error.

Levels of Unit Testing

1. UNIT
2. 100% code coverage
3. INTEGRATION
4. SYSTEM
5. ACCEPTANCE
6. MAINTENANCE AND REGRESSION

Concepts in Unit Testing:

1. The most 'micro' scale of testing;

2. To test particular functions or code modules.
3. Typically done by the programmer and not by testers.
4. As it requires detailed knowledge of the internal program design and code.
5. Not always easily done unless the application has a well-designed architecture
with tight code.

Unit Testing Flow

Types of Errors detected

The following are the Types of errors that may be caught :

1. Error in Data Structures

2. Performance Errors
3. Logic Errors
4. Validity of alternate and exception flows
5. Identified at analysis/design stages

Unit Testing Black Box Approach

1. Field Level Check

 2. Field Level Validation
3. User Interface Check
4. Functional Level Check

Unit Testing White Box Approach

1. STATEMENT COVERAGE
2. DECISION COVERAGE
3. CONDITION COVERAGE
4. MULTIPLE CONDITION COVERAGE (nested conditions)
5. CONDITION/DECISION COVERAGE
6. PATH COVERAGE

Unit Testing FIELD LEVEL CHECKS

1. Null / Not Null Checks

2. Uniqueness Checks
3. Length Checks
4. Date Field Checks
5. Numeric Checks
6. Negative Checks

Unit Testing Field Level Validations

1. Test all Validations for an Input field

2. Date Range Checks (From Date/To Dates)
3. Date Check Validation with System date

Unit Testing User Interface Checks

1. Readability of the Controls

2. Tool Tips Validation
3. Ease of Use of Interface Across
4. Tab related Checks
5. User Interface Dialog
6. GUI compliance checks

Unit Testing - Functionality Checks

1. Screen Functionalities
2. Field Dependencies
3. Auto Generation
4. Algorithms and Computations
5. Normal and Abnormal terminations
6. Specific Business Rules if any...
Unit Testing - OTHER MEASURES

1. FUNCTION COVERAGE
2. LOOP COVERAGE
3. RACE COVERAGE

Execution of Unit Tests

1. Design a test case for every statement to be executed.

2. Select the unique set of test cases.
3. This measure reports whether each executable statement is encountered.
4. Also known as: line coverage, segment coverage and basic block coverage.
5. Basic block coverage is the same as statement coverage except the unit of code
measured is each sequence of non-branching statements.

Advantage of Unit Testing

1. Can be applied directly to object code and does not require processing source
code.
2. Performance profilers commonly implement this measure.

Disadvantage of Unit Testing

1. Insensitive to some control structures (number of iterations)

2. Does not report whether loops reach their termination condition
3. Statement coverage is completely insensitive to the logical operators (|| and &&).

Method for Statement Coverage

Design a test-case for the pass/failure of every decision point -Select unique set of test
cases

1. This measure reports whether Boolean expressions tested in control structures

(such as the if-statement and while-statement) evaluated to both true and false.
2. The entire Boolean expression is considered one true-or-false predicate regardless
of whether it contains logical-and or logical-or operators.
3. Additionally, this measure includes coverage of switch-statement cases, exception
handlers, and interrupt handlers
4. Also known as: branch coverage, all-edges coverage, basis path coverage,
decision-decision-path testing
5. "Basis path" testing selects paths that achieve decision coverage.

ADVANTAGE:

Simplicity without the problems of statement coverage.

DISADVANTAGE:

This measure ignores branches within Boolean expressions which occur due to short-
circuit operators.

Method for Condition Coverage:

1. Test if every condition (sub-expression) in decision for true/false -Select unique

set of test cases.
2. Reports the true or false outcome of each Boolean sub-expression, separated by
logical-and and logical-or if they occur.
3. Condition coverage measures the sub-expressions independently of each other.
4. Reports whether every possible combination of Boolean sub-expressions occurs.
As with condition coverage, the sub-expressions are separated by logical-and and
logical-or, when present.
5. The test cases required for full multiple condition coverage of a condition are
given by the logical operator truth table for the condition.

DISADVANTAGES:

1. Tedious to determine the minimum set of test cases required, especially for very
complex Boolean expressions
2. Number of test cases required could vary substantially among conditions that
have similar complexity
3. Condition/Decision Coverage is a hybrid measure composed by the union of
condition coverage and decision coverage.
4. It has the advantage of simplicity but without the shortcomings of its component
measures
5. This measure reports whether each of the possible paths in each function have
been followed.
6. A path is a unique sequence of branches from the function entry to the exit.
7. Also known as predicate coverage. Predicate coverage views paths as possible
combinations of logical conditions
8. Path coverage has the advantage of requiring very thorough testing

Function Coverage:

1. This measure reports whether you invoked each function or procedure.

2. It is useful during preliminary testing to assure at least some coverage in all areas
of the software.
3. Broad, shallow testing finds gross deficiencies in a test suite quickly.

Loop Coverage:

This measure reports whether you executed each loop body zero times, exactly once,
twice and more than twice (consecutively). For do-while loops loop coverage reports
whether you executed the body exactly once and more than once.

The valuable aspect of this measure is determining whether while-loops and for-loops
execute more than once, information not reported by others measure.

Race Coverage:
his measure reports whether multiple threads execute the same code at the same time.
Helps detect failure to synchronize access to resources.
Useful for testing multi-threaded programs such as in an operating system.

Conclusion

1. Testing irrespective of the phases of testing should encompass the following :

2. Cost of Failure associated with defective products getting shipped and used by
customer is enormous
3. To find out whether the integrated product work as per the customer requirements
4. To evaluate the product with an independent perspective
5. To identify as many defects as possible before the customer finds
6. To reduce the risk of releasing the product