Silk Test
Silk Test
Silk Test
Tools - Comparison
WinRunner, QTP are from Mercury Interactive USD 1bn company and market leader; Robot from Rational IBM - USD 100bn company Silk from Segue USD 50mn company Script here is object oriented and close to QTP script. Winrunner TSL is like C and Robot SQA Basic is like Visual Basic GUI to create verification points are better in WinRunner, QTP and Robot Silk is more towards moving scripts on MS Windows and Unix; whereas Robot or WinRunner scripts cannot be moved to other OS as those tools do not support Rational does not provide GUI specification in a separate file; WinRunner, QTP and Silk provide that You need to buy Java Plugin for WinRunner; Silk and Robot come with Java plugin Record, replay, verification method, data driven tests, database connectivity, bitmap verification are all same at concept level and differ only at script syntax level Wizards are less in SilkTest, compared to other tools Cost-wise, QTP and WinRunner are most expensive; Silk stands in between WinRunner and Rational Robot
Architecture
Project .vtp (for maintenance) Testplan .pln (organizing the tests) Test Frame* .inc (AUT Window details) Scripts .t (test scripts) Data driven tests .g.t (scripts with many data) Include file .inc Suite .s (for batch runs) Result .res
5
Test Process
Manage requirements using SilkCentral Test Manager Manage test plans and cases using Silk test Associate test plans/cases to requirements using test manager Declare GUI declarations using SilkTest Develop scripts for atomic activities first so that they can be reused later Develop scripts using SilkTest and run them Track defects using Defect Tracker
7
Test Frames
Before test case recording, do this step This creates an .inc file that has GUI declarations Identifies GUI controls as classes Label or caption is the best way to identify objects; if not available, object index is used (available tags are caption, prior text, index, window id, coordinates) Class and tag are the key elements in declarations Record window declarations is the way to store test frames When objects are not identified properly, use different tags to uniquely identify the objects. This will vary from application to application If object X,Y coordinates are to be used, declare constants and then use those constants for further declarations To declare new objects within a window, it is better to use AnyWin or CustomWin classes. Use ~ActiveApp to specify the coordinates
8
Test Frames
Along with window declarations, we can declare methods that are specific to that window and objects The method declarations within a window object is similar to object oriented programming Uses can use these method declarations as wrapper classes for executing a series of steps that are recorded To the extent possible, do not have any hard coded values in the GUI Window methods While recording new test cases, ensure that you update window declarations as well; but check the frame.inc file, not to have duplicate declarations for same window Default is frame.inc; but we can have any number of include files Unless the window declarations are in scope, the test script will not recognize the controls
10
Recording
Recording will automatically generate statements on every user action To capture the mouse click coordinates and keyboard events, we can use low level enabler for every control in recording options We can record the base state as well as test steps Soon after recording, ensure the test frame declarations, for any changes or duplicates Recording can be done for window declarations or test case or actions Every recorded script must start from a known base state and must return back to the same known base state BaseState and DefaultBaseState can also be specified using recording (appState is a function that ensures the pre-condition state for a test case)
11
12
13
Test Cases
Each test case must be independent Initialize (pre-requisite setup), run and verify, cleanup One test case must have One purpose Establish application state for every test case. Record the base state Test Case contains commands for every test step. Record the steps and edit if needed Establish verification points for every test case Record the clean up steps as well Sequencing of test cases is very important when it comes to automation These are also functions, with the type as testcase We can use parameters for testcase as well A testcase is also a function by itself, but it can have base states
14
4Test Elements
Variables INTEGER, STRING, BOOLEAN, ANYTYPE, REAL, LIST Variables do have scope public or private Const is for constants Regular arithmetic and string operations hold good 2 types of for statements Foreach and While loops are also available IF, select-case and SWITCH-case are available for branching Do-except is for exception handling Exception functions will be useful for tracing exception number and details Test steps will have window.object.method or window.object.property style of commands User defined functions with in, out, inout parameters Functions can return values as well Use built-in functions for list and array manipulations Print and Printf commands are used for sending output to results file
15
17
18
19
20
Method examples
Find.OK.Click Find.MatchCase.Check Find.FindText.SetText(abc) Notepad.CloseWindows Find.Up.VerifyValue(TRUE) Fonts.FontName.verifyContents() Notepad.Editor.SelectAll() // this is a custom method to select all contents. This needs to be declared in the windows declarations User defined functions can have in, out, inout and optional parameters ClipBoard functions are used to manipulate clipboard text Cursor class methods can be used to manipulate coordinates Property prefixes b for boolean, I for int, ls for list, s for string
21
For every object, Get methods are used to obtain attribute values into a variable. After getting to a variable, we need to manipulate For every object, Set methods are used to set attribute values on the screen. This will work only when the object cooperates with SilkTest operations For every object, Verify methods are used to check the attribute values implicit Get and an if-then-else condition. It must have an expected value as argument Whatever properties that can be used for get statement, can be used for set and verify statements Verify methods evaluate to pass or fail status Any other custom methods can be used, if they are already declared in window declarations Most of the properties and methods are inherited from AnyWin to all other GUI classes Dlls can be loaded using dll declaration and the functions can be invoked from 4Test Capture and Compare bitmap commands are used for image checks
22
Utility Elements
Array functions are used to manipulate arrays (ArraySize, ArrayFind) List functions are used to manipulate lists (ListSort, ListAppend) DateStr, TimeStr, Sleep, StartTimer, StopTimer are used for timer functions FileOpen, FileClose, FileWriteLine, FileReadLine are used for file operations Wild cards - ? And *; MatchStr command is used for expressions Multitags enable an object to be identified with many labels Registry Operations are available not used widely (Reg_CreateKey) LogError, LogWarning, Print to write to results file Timer commands to create, start, stop, destroy and get the interval Date format and functions to manipulate date fields DLLs can be loaded and then functions can be executed; but this needs to be kept at minimum To get script, environment and browser related information, we can use the respective built-in functions (GetArgs, GetBrowserType) Date related functions are available; but it is advised not to use inside script to manipulate dates
23
Database Test
Database tests follow the verification of GUI verifications Database commands start with DB_ Connect to DB (DSN needed) Execute Query (give SELECT) Fetch the records and compare with baseline Finish DB operation (release resources) Disconnect after every database operation Do not keep the database connections for a long time in 4Test scripts
24
Verification
Object properties are the key elements Control state, value, label, text, count, size, location are the most common properties Feed the expected result to the verify command Bitmaps can be verified using CaptureBitmap and VerifyBitmap Use Bitmap tool to capture baseline image Is* methods evaluate to a boolean value (IsActive, IsEnabled) LinkTester checks the links in web applications
25
27
Run Tests
One can run a test case or test plan or test suite Suite contains many test cases in it Using distributed run mode, one can run scripts in another machine that has Silk Agent We can run in run mode or debug mode Debug mode enables us to step, step into, step out, declare breakpoints etc All other hosts run the Agent Spawn, Parallel, Rendezvous are used in distributed testing but unless required, distributed testing is not widely used One must connect and disconnect to target machines
28
Traceability
Requirements to test traceability is thru Test Manager Test case to tester traceability is again part of test manager We need to define naming conventions by ourselves for test plans, test cases, requirements Suggested guideline is to name the test case with the test case ID itself and have the test case description as comments Use base states for capturing test pre-requisites Version control is integrated with PVCS for check-in, check-out and labeling features
29
Recovery Scenarios
Recovery system enables the test to come to the Base State Record application state is used to specify the recovery scenarios Exception handler itself is script recovery feature StopRunning() method bypasses all recovery settings
30
Test Extensions
These are used to help SilkTest to identify new objects and their properties Silktest can automatically enable extensions for open applications Java, .Net, AOL Browser, Netscape Navigator, Internet Explorer require extensions For browsers, enabling DOM or Virtual Object mode would be configured to identify objects
31
32