Friday 21 September 2012

Analyzing Test Results & Reporting



Analyzing Test Results & Reporting

Analyzing the Test Result is not a responsibility of QTP. After Running Tests, User (Test Engineer) has to analyze the test result. and send defect reports.

QTP Result Window:

After Running/Executing a test, then QTP Provide result window. we export test result from result window in Document (.Doc), HTML, XML and PDF formats.

Note: QTP doen't provide batch wise test result, if we execute test batches. It provides test wise result only.

Status of the Result:

QTP has 4 types of result status

a) Pass
b) Fail
c) Done
d) Warning
Defining our own Result:
Generally QTP provides, test results, if we use qtp tool features like checkpoints. suppose if we use flow control statements (conditional and Loop statements), then we have to define our own result.

For defining our own results, QTP is providing an utility object called "Reporter"

by using reporter, we can define our own results.

Syntax:

Reporter.ReportEvent status of the Result,"Result Step","message"

Example"

Reporter.ReportEvent micPass,"Res","Login operation Successful"

exaplanation:

Reporter- Utility object
ReportEvent-Method
micPass-Status of the Result.
Res-Result step
"Login operation Successful"- Message

Status of the Result:

1) Pass -micPass or 0

2) Fail  -micPass or 1

3) Done- micDone or 2

4) Warning- micDone or 3
Reporting Defects
QTP is an Object based Test tool. We can create and execute tests using QTP. QTP Provides, various methods for crating and enhancing Tests.

After Running Tests, QTP provides result in Result window and also provides a facility to define our own Results. Result Exporting facility also available in QTP Result window.

Any how Analyzing Results is the Responsibility of Test Engineer after running tests.

After analyzing results, we can report defects in various methods.

If we are using QTP for Functional Test Automation and Defect management manual, then entering defect details in company prescribed format and sending.

If we are using Bugzilla or Zera or PR-Tracker like Bug Tracking tool, then after analyzing results, if we find any mismatches sending defects through the Bug tracking tool (Every Bug Tracking tool has its own defect report template)

If we are using Quality center for Test management, then we can send defects directly from QTP result window. In between QTP and Quality Center back to back integration is available.

Transaction Points



Inserting Transaction Points

QTP is providing a Utility object called Services for measuring transaction time
Suppose, we want to measure how much time that one test or part of the test is taking for execution/running, we can measure using these transaction point (Start and End) 

Syntax:

Services.StartTransaction “Transaction Name”
Statements
-------
------
Services.EndTransaction “Transaction Name”

Note: these options (Start and End Transactions) available in QTP tool Insert menu

Example:

services.StartTransaction "Login"
SystemUtil.Run "C:\Program Files\Mercury Interactive\QuickTest Professional\samples\flight\app\flight4a.exe","","C:\Program Files\Mercury Interactive\QuickTest Professional\samples\flight\app\","open"
Dialog("Login").Activate
Dialog("Login").WinEdit("Agent Name:").Set "naga"
Dialog("Login").WinEdit("Password:").SetSecure "4baf50f18b0ae0d5f5425fe760653e96da50bde7"
Dialog("Login").WinButton("OK").Click
Window("Flight Reservation").Activate
services.EndTransaction "Login"
Note: We can insert number of transaction points in a test

Using Timer Function 


‘Timer’ is a VB Script Built-in function, that can be used for measuring transaction time.
ST=Timer’ It returns elapsed time since 12:00 AM Mid night in seconds
Example:
SystemUtil.Run "C:\Program Files\Mercury Interactive\QuickTest Professional\samples\flight\app\flight4a.exe","","C:\Program Files\Mercury Interactive\QuickTest Professional\samples\flight\app\","open"
Dialog("Login").Activate
Dialog("Login").WinEdit("Agent Name:").Set "naga"
Dialog("Login").WinEdit("Password:").SetSecure "4baf50f18b0ae0d5f5425fe760653e96da50bde7"
Dialog("Login").WinButton("OK").Click
Window("Flight Reservation").Activate
ET=Timer
TT=ET-ST
msgbox TT


Note: here ST, ET and TT are variables, you can take any variable, I used:
ST for Start Transaction
ET for End transaction
TT for Transaction Time

Step Generator