Results and Reporting
Use these commands and functions to customize the results created by SenseTalk scripts in Eggplant Functional. (Also see The ScriptLogging in Global Properties.)
Assert
Command
Behavior: When you run an assert
command, Eggplant Functional evaluates the specified condition (an expression) as true or false. If it is evaluated as false then by default assert
throws an exception. It can also log a success, failure, or warning, depending on how the behavior is set. The failure behavior for the assert
command can be set in-line using the Assert
command properties as shown in the syntax and examples below. This behavior can also be set using The AssertionBehavior
Global Property. To improve readability, you can insert the word that
into an assert
command.
Parameters: A condition (expression) you want Eggplant Functional to evaluate.
Syntax:
assert {that} condition {with [warning | error | exception | pause]}
Example: Using The AssertionBehavior
Global Property to set the behavior of the assert
command upon evaluating the condition to be false.
set the AssertionBehavior to "warning" // Changes the default behavior of assertions to generate a warning instead of an exception
assert that x < 10 // Logs a warning message if the value of x is not less than 10 due to the AssertionBehavior global property set previously
Example:
assert that ConfirmationNumber is "128901823" with Exception "Validating the order confirmation number." // Customizes the assertion message
Example:
params Platform, Version, DebuggingMode
assert paramCount() is greater than 2 with error // Asserts that at least 3 parameter values were passed into the script, and if not, logs an error
Log the assertionErrorCount&&"assertions have failed." // Logs the total number of assertion errors that have occurred during the execution
Example:
Assert that ["iOS","Android","Windows","MacOS"] contains SUTPlatform // Asserts that the value stored in SUTPlatform is one of 4 acceptable values and if not, throws an exception to stop the execution.
Example:
put ReadText("ResultsUpperLeft","ResultsLowerRight") into UnfilteredResults // Uses OCR to read a number from the SUT and stores it in variable UnfilteredResults
Tap "Filter" // Performs an action on the SUT that causes a filter to be turned onWaitFor 10, "FilterOn"
put ReadText("ResultsUpperLeft","ResultsLowerRight") into FilteredResults
assert FilteredResults is less than UnfilteredResults with Warning // Asserts that the number read by OCR after applying a filter on the SUT is smaller than before applying the filter
BeginTestCase
and EndTestCase
Commands
Behavior: The BeginTestCase
command is used to open a test case, and the EndTestCase
command is used to close one. Each test case is given a name, as shown in the examples below.
Syntax:
BeginTestCase CaseName
actions
EndTestCase CaseName
Example:
BeginTestCase "SubmitLogin"
Login // Runs a script named Login. Note that calling the script with RunWithNewResults instead would automatically create a separate test case for Login
EndTestCase "SubmitLogin"
Example:
Click "SearchButton"
BeginTestCase "ResultsLoaded"
WaitFor 20, "Results"
EndTestCase "ResultsLoaded"
set myResult to the result
put myResult.testcase & comma & myResult.duration & comma & myResult.starttime & CRLF after file ResourcePath ("myTimings.csv") // Writes the duration of the test case for the "Results" image appearing out to a csv file
Example:
BeginTestCase "A" // Marks the start of the reporting period for case A
// Other commands here
EndTestCase "A" // Ends the reporting period for case A and logs the information that was gathered
BeginTestCase "B" // Starts recording information for case B
// other commands here
BeginTestCase "C" // Starts recording information for case C
// other commands here
EndTestCase "B" // Ends case B and reports information gathered since case B was opened
// Other commands here
// If the run ends with test case C still open, it will be ended and reported automatically
Tech Talk
A test case is used to block out any section of code within a script (or across scripts during a run) for purposes of gathering information about that portion of the test. A script run may include multiple test cases, which may be sequential portions of a script or may overlap with one another as desired. Information about open test cases for the current script run can be obtained at any time using the OpenTestCases
global property.
The information gathered for each test case includes:
- The duration of the test case (in seconds)
- The number of Errors logged while the test case is open
- The number of Successes logged while the test case is open
- The number of Warnings logged while the test case is open
- The number of Exceptions logged while the test case is open
- The name of the test case
- The date and time at which the test case started
One test case is created automatically at the beginning of each run, with the name of the script being run.
At the end of the run, any test cases that are still open are ended and reported automatically.
The initial handler (the script itself) is considered a test case and will be reported as one If no declared test cases are open or no test cases are declared at all, the script will be reported as the only open test case.
Test cases can overlap.
Beginning in version 5.2, Eggplant Manager automatically parses test case data from script results and displays the duration for the test cases in a dashboard chart. Parsed test case data is exportable from Eggplant Manager.
CaptureScreen
Command
Behavior: Captures a snapshot of the entire SUT screen, or a rectangle indicated in the property list. You can customize this command with a list of any number of the CaptureScreen
properties.
When CaptureScreen
is executed as part of a script, the screenshot is saved in the results folder for the script. When the CaptureScreen
command is run as part of a selection or from the Ad Hoc Do Box, the resulting screenshot is saved in your local user's document directory (e.g., C:/Users/<username>/Documents/
).
It can be helpful to know where your screenshot is being saved if you don't already know. Accessing the file path of the image captured can be done using the Result
function like this:
CaptureScreen
put the result -- Returns the full path for the screenshot that was captured on the previous line using CaptureScreen.
Images captured using CaptureScreen
in JPG format should not be used for image searching. Because JPG compression requires data loss (is not lossless), it will not reliably match using the same search settings as a PNG of the same image would.
Parameters: Parameters can be passed to CaptureScreen
in two ways. You can either pass up to three ordered properties, or an optional property list.
Passing Ordered Properties
When using ordered properties, they must appear in this order:
- File name
- Rectangle
- shouldIncrement (Whether or not an increment number should be added.)
The default file format saved by CaptureScreen
is PNG. In order to change the file format, scale, or JPG compression quality for an image, either pass a property list instead of using ordered parameters, or use one of the related global properties (the CaptureScreenImageFormat, the CaptureScreenScaleFactor, or the CaptureScreenJPGCompressionQuality).
Passing a Property List
Property keys provided for use with the optional property list allow you to define the file name, capture rectangle, and to choose whether or not the file names are incremental, just like you can when passing ordered properties. However, it also adds the ability to pass image related information as a property list, and set the file format, scale, and compression quality for the screenshot.
FileName
(or the synonymsImageName
orName
): Default: "Screen_Capture". An image file name and optional path information.Rectangle
(orRect
): A coordinate pair. Default: The entire remote screen. Use this property key to pass a coordinate pair indicating the top left and bottom right corners of the area on the remote screen you want it to capture.Increment
: Boolean. Default: Off. Whether to append an automatically incremented number to the image. This property takes a boolean:true
orfalse
,yes
orno
. Incrementing is helpful if you are capturing several snapshots within a script (as in a loop or frequently called handler). The images can all have the same image name, with the incremented number distinguishing them; otherwise, each image would overwrite the previous image with the same name.ImageInfo
: An image property list. If you want to supply image information, passing parameters via a property list usingImageInfo
is the way to do it.ImageFormat
: PNG, JPG, or TIFF. Default: PNG. Modifies the file format used byCaptureScreen
. Options include:PNG
,JPG
, andTIFF
.ScaleFactor
: 0-1. Default 1 (full size). This allows you to scale the image proportionally, reducing its resolution. Setting theScaleFactor
to .5, for instance, will cut the image dimensions (both height and width) in half. This is different from the Scale Factor used with image searching. For more on how to scale captured images for search purposes, see Image Scaling on the Finding Images page.JPGCompressionQuality
: 0-100. Default: 100 (no compression; as lossless as possible). Only applicable to JPG screen captures. Adjusts the compression quality of JPG formatted screenshots captured by theCaptureScreen
command. The number indicates the level of image quality preserved during compression, 0% being low quality with a higher amount of image data loss, and 100 meaning that the compression preserves as close to 100% original quality as possbile.
Syntax:
CaptureScreen {fileName {, rectangle {, shouldIncrement}}}
CaptureScreen optionsPropertyList
Returns: The full path of the screenshot that was captured can be accessed using the Result Command.
Examples: Calling CaptureScreen
with the Optional Property List
This example passes a name for the screenshot to be captured, a Rectangle for the capture area(defined by a coordinate for one corner and an image for the other). It also sets the Increment
value to true, so screenshots captured will be named incrementally.
CaptureScreen {Name: "ImageFileName", Rectangle: [[67, 33], imagelocation("OtherCorner")], increment:true}
This example uses the SuiteInfo function to define the image path for saving the screenshot, as well as defining a capture rectangle
.
CaptureScreen {name:SuiteInfo().ImagesFolder & slash & "CurrentLogo", rectangle:["WelcomeImage", "Border"]}
This example adjusts the image file type, size, and quality, by using the ImageFormat
, ScaleFactor
, and JPGCompressionQuality
properties. It also names the screenshot using both the unchanging name "RegressionTest", as well as the current date.
CaptureScreen {name: "RegressionTest" & today, rectangle: ["TLImage","BRImage"], imageFormat: JPG, scaleFactor: .75, JPGCompressionQuality: 85}
Example: Calling CaptureScreen
with Ordered Parameters
This example shows how to call CaptureScreen
using ordered parameters. It passes all three parameters, declaring the name and location for the image to be saved as the first fileName parameter, then passing a list of coordinates defining the top left and bottom right corners of the area to be captured as the Rectangle parameter, and finally passing "true" as the shouldIncrement parameter.
CaptureScreen "~/Documents/ScreenCaptures/TestScreenshot", [58,579,714,736], true
Example:
This example uses a property list to pass an image name and turns incremental image naming on. The first time this line is called in a script, it will capture an image file named “LoginScreen_0001”, the next time it is called during the same script run it will capture an image file named “LoginScreen_0002”, and so forth.
CaptureScreen name:"LoginScreen", increment:Yes
Example:
Use a script like this to determine a page has changed from the previous state, when there is no appropriate element to tell you that you are seeing the new state.
Repeat at least once until not imagefound(image:RxResult, waitFor:0) -- A special repeat loop that ensures at least one iteration, ignoring the "not imagefound()" condition on the first iteration only. On subsequent iterations, the imagefound function will search for the screenshot captured by the CaptureScreen command.
put ImageLocation("RxNumberUpperLeft") into RxLocation -- Stores the hot spot location of image RxNumberUpperLeft in variable RxLocation
CaptureScreen {name:"PrescriptionNumber", Rectangle:[RxLocation, RxLocation + [140,100]]} -- Captures a screenshot of the unique number displayed to the right of the RxLocation.
put the result into RxResult -- Stores the full file path of the CaptureScreen screenshot
Click "ClosePrescription" -- Performs an action that will cause the screen to change
wait 1
if the repeatIndex = 5 then Throw "Image not Found","Closing the current prescription did not work." -- Throws an exception if the repeat loop repeats 5 times
end repeat
Example:
Use a script like this to detect when you've scrolled to the bottom of the page:
put RemoteScreenSize() into DUTScreenSize -- Stores the resolution of the SUT in a variable called DUTScreenSize
put [.25*DUTScreenSize.x,.25*DUTScreenSize.y,.75*DUTScreenSize.x,.75*DUTScreenSize.y] into ClippingRectangle -- Creates a rectangle that is a subset of the full SUT size. Using a subset helps avoid capturing dynamic elements on the screen, such as the clock or notifications area.
CaptureScreen {Name: "state", Rectangle: ClippingRectangle} -- Captures a screenshot of the SUT using the ClippingRectangle coordinates
put the result into refImage -- Stores the file path of the saved CaptureScreen screenshot into a variable for easy referencing later
repeat at least once while not imageFound (image:refImage, searchRectangle:ClippingRectangle, waitFor:0) -- Searches for the CaptureScreen image in the same location on the screen where it was originally captured and ensures that at least one scrolling action is executed against the SUT. Exits the repeat loop in the event that the CaptureScreen image is present inside the searchRectangle following a scrolling event.
CaptureScreen {Name: "state", Rectangle: ClippingRectangle} -- Creates a new CaptureScreen image to reflect the fact that a scroll action has occurred
put the result into refImageSwipeUp -- Repeats a scrolling action to scroll down the page
Wait 2.5 -- A wait time is generally required to allow the UI to stop moving before continuing
end repeat
Example:
Use a script like this to determine whether an element on the screen, such as a video player, is changing:
put imageLocation("VideoNameHeader") into VideoLoc -- Stores the location of an reference image for the changing element
put [VideoLoc,VideoLoc+[250,230]] into myRectangle -- Uses the location stored in VideoLoc, and an offset of 250 pixels to the right and 230 pixels down, to create a rectangle
CaptureScreen{name:"VideoCheck",rectangle:myRectangle} -- Uses the rectangle stored in myRectangle to specify the subset of the screen captured by CaptureScreen
put the result into imageFilePath
set the searchRectangle to myRectangle -- Limits the search area to the same subset of the screen used by the CaptureScreen.
repeat until not imagefound (image:imageFilePath, waitfor:0) -- Repeats searching for the CaptureScreen image until it is not showing
Wait 3 -- Allows the element some additional time to change before searching again for the CaptureScreen image
if the repeatIndex is greater than 10 then Throw "Video stream exception","The video is not advancing." -- Provides a way to stop the script execution in the event that the element doesn't change after 10 iterations of the repeat loop
end repeat
set the searchRectangle to empty -- Sets the searchRectangle back to full screen
Related Global Properties
ColorAtLocation
Function
Behavior: Returns the color value of the pixel in the given location. Coordinates are relative to the top-left corner of the SUT. The top-left corner is (0,0), with the x value increasing toward the right, and the y value increasing toward the bottom.
Parameters: One coordinate location, image, or text (OCR) reference.
Returns: The color value of the pixel in the given location. For more information, see Working with Color, in the SenseTalk Reference Manual
.
Syntax:
ColorAtLocation( xCoordinate , yCoordinate )
ColorAtLocation( point )
Example:
log ColorAtLocation(593,110)
Example:
if imagefound(Text:"Document") then
put colorAtLocation( the topleft of FoundImageInfo().imagerectangle) into myColor //Determine the RGB value of a pixel 5 pixels to the left of the top left corner of string "Document" on the SUT
if myColor=color("black") then //Uses the color function to return the RGB value of black and compare it to the actual RGB value
LogSuccess "Black background"
else
LogWarning "Background color" && myColor && "unexpected."
end if
end if
Example:
Set the ColorFormat to "HTML" //Changes the color format returned by ColorAtLocation() and Color() to hexadecimal
put ColorAtLocation ("StatusIndicator") into myColor
If myColor isn't "#AFF7DF" then LogError "Color is not #AFF7DF. Color is " &myColor&period
CompareScreen
Command or Function
Behavior: Use CompareScreen
as either a command or function to check for changes to a screen or window. Typically, you would call CompareScreen
when the script has just performed an action on the SUT that results in displaying a new screen or window. When differences are detected, you use the Compare Screen panel in Eggplant Functional to determine how to proceed.
CompareScreen
visually compares the current SUT screen against a baseline image, which is specified as the first parameter. If the baseline image doesn't exist, it is automatically captured the first time CompareScreen
is executed for that image name.
If you use CompareScreen
without specifying a rectangle, the comparison applies to the entire current SUT screen. Optionally, you can specify a rectangle so that only a region of the screen (such as a single application window) is included in the comparison. If the rectangle given (or the size of the current SUT screen) is different from the baseline reference image, an exception is thrown.
When CompareScreen
is executed, the script locates the baseline image in the special [Baseline_Screens] folder within the Images folder of the suite (or in an active helper suite). If no image by that name is found, an initial baseline image is captured and stored automatically, then the script proceeds without any further action.
The [Baseline_Screens] folder is created automatically the first time you run CompareScreen
in a suite. You do not need to create this folder yourself.
If an existing baseline image is present, that image is compared to the current SUT screen (or to the specified rectangle of the SUT screen). The comparison is performed using the search criteria (e.g., search type, tolerance) of the baseline image. Note that you can open the baseline image from the Images pane of the suite and adjust these properties if desired, which will affect future comparisons.
When you call CompareScreen
as a command, the behavior following the comparison is controlled by the Compare Screen Action mode, which can be set on the Run menu in Eggplant Functional (Run > Compare Screen) or by using the CompareScreenAction
global property. The Compare Screen Action mode can be set to:
- Show Panel: If a screen difference is detected, the Compare Screen panel opens.
- Log Warning: If a screen difference is detected, a warning message is entered in the log.
- Log Error: If a screen difference is detected, an error message is entered in the log.
- Show Panel Always: The Compare Screen panel opens with every
CompareScreen
command, regardless of whether any screen difference is detected.
For information about using the Compare Screen panel, see Using Compare Screen in Eggplant Functional for UI Testing.
When you call CompareScreen
as a function, the setting of the Compare Screen Action mode is ignored. The Compare Screen panel is never displayed. If no significant differences are found, the function returns an empty list.
If there are differences between the baseline image and the current SUT screen, the function records information about the differences in the run log so that the differences can be reviewed later from the Results pane. It also returns a list of property lists, where each screen difference is one property list with these properties:
- difference: New, Missing, Moved, Changed
- differenceType: Text, Element, Area, DynamicText, DynamicContent
- source: Baseline, Current, Combined
- rectangle: SenseTalk rectangle on baseline
When differenceType is Text, these properties are also present:
- text: baseline text
- textNow: current text
When difference is Moved, this property is also included:
- rectangleMovedTo: SenseTalk rectangle of destination on current screen
The following combinations of difference and differenceType are possible:
- New: differenceType can be Text or Element
- Missing: differenceType can be Text, Element, DynamicText, or DynamicContent
- Moved: differenceType can be Text or Element
- Changed: differenceType can be Text, Element, or Area
When you call CompareScreen
as a command, the list of differences is available within the script by calling the result()
function on the very next line of the script.
Parameters: An image name and (optional) specified rectangle.
Syntax:
CompareScreen baselineName
CompareScreen baselineName, rectangle: rectangle
CompareScreen name: baselineName, rectangle: rectangleAs Function: CompareScreen( baselineName )
CompareScreen( baselineName, rectangle: rectangle )
CompareScreen( name: baselineName, rectangle: rectangle )
Example:
CompareScreen
command for a full-screen compare, captures the image the first time it's run:
CompareScreen "tutorialSut"
Example:
CompareScreen
command for a specific rectangle comparison:
CompareScreen name: "tutorialSut_menu", rectangle: (22,575,415,640)
Example:
How to use the CompareScreen
command for a specific rectangle defined based on images, then uses the result
function to save a list of changes:
CompareScreen "calcHist", rectangle:("Calculator_A","Calculator_B")
put the result.archive into ChangeHist
Example:
The results of CompareScreen
are stored in the variable:
put CompareScreen("HomeScreen") into homeDiffs
Example:
Logs a message if differences are detected:
if CompareScreen("LoginPanel", rectangle:[100,100,500,400]) isn't empty then log "There were differences"
Example:
Creates a log message showing the results from CompareScreen
:
log "Diffs: " & CompareScreen(Name: "NewAccount", Rectangle:["AcctWindowTopLeft","AcctWindowBottomRight"])
Related:
Log
Command
Behavior: Creates an entry in the script’s Log file. If you set multiple message parameters, each message is inserted as a separate entry. (If the ScriptLogging
global property value is Minimal
or Silent
, this command does nothing.)
Parameters: One or more Log
messages.
Syntax:
Log message
Example:
Log "Beginning the login sequence now."
Example:
log "Today is" && the long date & period -- Concatenates a string with the output from the long date function
Example:
log "This is my first message.", "This is my second message" -- Logs each message with a separate log entry in the results
Log entries can make it easier to determine where particular log events are occurring in the run.
LogError
Command
Behavior: Behaves just like the Log
and LogWarning
commands, but displays the log entries in red text, and logs the script as a failure.
Parameters: One or more Log messages.
Syntax:
LogError errorMessage
Example:
LogError "This is my error message."
Example:
If imageFound("ExceptionDialog") then
CaptureScreen -- Creates a screenshot at the point of failure, as LogError does not do so automatically like the ImageNotFound exception does
LogError "Known critical error occurred. Ending execution."
exit all -- Terminates execution of the current handler, the handler that called it, and all other handlers further up the call stack
end if
If the ScriptLogging
global property is set to Silent
, no error messages are inserted into the log file, but the error count for the run is still be incremented, and the ultimate status of the run is Failure
.
LogSuccess
Command
Behavior: Behaves just like the Log
commands, but adds one to the success count for the script.
Parameters: One or more Log
messages.
Syntax:
LogSuccess successMessage
Example:
LogSuccess "The page loaded properly at" && the short time & "."