Results and Reporting
Use these commands and functions to customize the results created by SenseTalk scripts in Eggplant Functional. (Also see The ScriptLogging in Global Properties.)
Assert
Command
Behavior: When you run an assert
command, Eggplant Functional evaluates the specified condition (an expression) as true or false. If it is evaluated as false then by default assert
throws an exception. It can also log a success, failure, or warning, depending on how the behavior is set. The failure behavior for the assert
command can be set in-line using the Assert
command properties as shown in the syntax and examples below. This behavior can also be set using The AssertionBehavior
Global Property. To improve readability, you can insert the word that
into an assert
command.
Parameters: A condition (expression) you want Eggplant Functional to evaluate.
Syntax:
assert {that} condition {with [warning | error | exception | pause]}
Example: Using The AssertionBehavior
Global Property to set the behavior of the assert
command upon evaluating the condition to be false.
set the AssertionBehavior to "warning" // Changes the default behavior of assertions to generate a warning instead of an exception
assert that x < 10 // Logs a warning message if the value of x is not less than 10 due to the AssertionBehavior global property set previously
Example:
assert that ConfirmationNumber is "128901823" with Exception "Validating the order confirmation number." // Customizes the assertion message
Example:
params Platform, Version, DebuggingMode
assert paramCount() is greater than 2 with error // Asserts that at least 3 parameter values were passed into the script, and if not, logs an error
Log the assertionErrorCount&&"assertions have failed." // Logs the total number of assertion errors that have occurred during the execution
Example:
Assert that ["iOS","Android","Windows","MacOS"] contains SUTPlatform // Asserts that the value stored in SUTPlatform is one of 4 acceptable values and if not, throws an exception to stop the execution.
Example:
put ReadText("ResultsUpperLeft","ResultsLowerRight") into UnfilteredResults // Uses OCR to read a number from the SUT and stores it in variable UnfilteredResults
Tap "Filter" // Performs an action on the SUT that causes a filter to be turned onWaitFor 10, "FilterOn"
put ReadText("ResultsUpperLeft","ResultsLowerRight") into FilteredResults
assert FilteredResults is less than UnfilteredResults with Warning // Asserts that the number read by OCR after applying a filter on the SUT is smaller than before applying the filter
BeginTestCase
and EndTestCase
Commands
Behavior: The BeginTestCase
command is used to open a test case, and the EndTestCase
command is used to close one. Each test case is given a name, as shown in the examples below.
Syntax:
BeginTestCase CaseName
actions
EndTestCase CaseName
Example:
BeginTestCase "SubmitLogin"
Login // Runs a script named Login. Note that calling the script with RunWithNewResults instead would automatically create a separate test case for Login
EndTestCase "SubmitLogin"
Example:
Click "SearchButton"
BeginTestCase "ResultsLoaded"
WaitFor 20, "Results"
EndTestCase "ResultsLoaded"
set myResult to the result
put myResult.testcase & comma & myResult.duration & comma & myResult.starttime & CRLF after file ResourcePath ("myTimings.csv") // Writes the duration of the test case for the "Results" image appearing out to a csv file
Example:
BeginTestCase "A" // Marks the start of the reporting period for case A
// Other commands here
EndTestCase "A" // Ends the reporting period for case A and logs the information that was gathered
BeginTestCase "B" // Starts recording information for case B
// other commands here
BeginTestCase "C" // Starts recording information for case C
// other commands here
EndTestCase "B" // Ends case B and reports information gathered since case B was opened
// Other commands here
// If the run ends with test case C still open, it will be ended and reported automatically
Tech Talk
A test case is used to block out any section of code within a script (or across scripts during a run) for purposes of gathering information about that portion of the test. A script run may include multiple test cases, which may be sequential portions of a script or may overlap with one another as desired. Information about open test cases for the current script run can be obtained at any time using the OpenTestCases
global property.
The information gathered for each test case includes:
- The duration of the test case (in seconds)
- The number of Errors logged while the test case is open
- The number of Successes logged while the test case is open
- The number of Warnings logged while the test case is open
- The number of Exceptions logged while the test case is open
- The name of the test case
- The date and time at which the test case started
One test case is created automatically at the beginning of each run, with the name of the script being run.
At the end of the run, any test cases that are still open are ended and reported automatically.
The initial handler (the script itself) is considered a test case and will be reported as one If no declared test cases are open or no test cases are declared at all, the script will be reported as the only open test case.
Test cases can overlap.
Beginning in version 5.2, Eggplant Manager automatically parses test case data from script results and displays the duration for the test cases in a dashboard chart. Parsed test case data is exportable from Eggplant Manager.
CaptureScreen
Command
Behavior: Captures a snapshot of the entire SUT screen, or a rectangle indicated in the property list. You can customize this command with a list of any number of the CaptureScreen
properties.
When CaptureScreen
is executed as part of a script, the screenshot is saved in the results folder for the script. When the CaptureScreen
command is run as part of a selection or from the Ad Hoc Do Box, the resulting screenshot is saved in your local user's document directory (e.g., C:/Users/<username>/Documents/
).
It can be helpful to know where your screenshot is being saved if you don't already know. Accessing the file path of the image captured can be done using the Result
function like this:
CaptureScreen
put the result -- Returns the full path for the screenshot that was captured on the previous line using CaptureScreen.
Images captured using CaptureScreen
in JPG format should not be used for image searching. Because JPG compression requires data loss (is not lossless), it will not reliably match using the same search settings as a PNG of the same image would.
Parameters: Parameters can be passed to CaptureScreen
in two ways. You can either pass up to three ordered properties, or an optional property list.
Passing Ordered Properties
When using ordered properties, they must appear in this order:
- File name
- Rectangle
- shouldIncrement (Whether or not an increment number should be added.)
The default file format saved by CaptureScreen
is PNG. In order to change the file format, scale, or JPG compression quality for an image, either pass a property list instead of using ordered parameters, or use one of the related global properties (the CaptureScreenImageFormat, the CaptureScreenScaleFactor, or the CaptureScreenJPGCompressionQuality).
Passing a Property List
Property keys provided for use with the optional property list allow you to define the file name, capture rectangle, and to choose whether or not the file names are incremental, just like you can when passing ordered properties. However, it also adds the ability to pass image related information as a property list, and set the file format, scale, and compression quality for the screenshot.
FileName
(or the synonymsImageName
orName
): Default: "Screen_Capture". An image file name and optional path information.Rectangle
(orRect
): A coordinate pair. Default: The entire remote screen. Use this property key to pass a coordinate pair indicating the top left and bottom right corners of the area on the remote screen you want it to capture.Increment
: Boolean. Default: Off. Whether to append an automatically incremented number to the image. This property takes a boolean:true
orfalse
,yes
orno
. Incrementing is helpful if you are capturing several snapshots within a script (as in a loop or frequently called handler). The images can all have the same image name, with the incremented number distinguishing them; otherwise, each image would overwrite the previous image with the same name.ImageInfo
: An image property list. If you want to supply image information, passing parameters via a property list usingImageInfo
is the way to do it.ImageFormat
: PNG, JPG, or TIFF. Default: PNG. Modifies the file format used byCaptureScreen
. Options include:PNG
,JPG
, andTIFF
.ScaleFactor
: 0-1. Default 1 (full size). This allows you to scale the image proportionally, reducing its resolution. Setting theScaleFactor
to .5, for instance, will cut the image dimensions (both height and width) in half. This is different from the Scale Factor used with image searching. For more on how to scale captured images for search purposes, see Image Scaling on the Finding Images page.JPGCompressionQuality
: 0-100. Default: 100 (no compression; as lossless as possible). Only applicable to JPG screen captures. Adjusts the compression quality of JPG formatted screenshots captured by theCaptureScreen
command. The number indicates the level of image quality preserved during compression, 0% being low quality with a higher amount of image data loss, and 100 meaning that the compression preserves as close to 100% original quality as possbile.
Syntax:
CaptureScreen {fileName {, rectangle {, shouldIncrement}}}
CaptureScreen optionsPropertyList
Returns: The full path of the screenshot that was captured can be accessed using the Result Command.
Examples: Calling CaptureScreen
with the Optional Property List
This example passes a name for the screenshot to be captured, a Rectangle for the capture area(defined by a coordinate for one corner and an image for the other). It also sets the Increment
value to true, so screenshots captured will be named incrementally.
CaptureScreen {Name: "ImageFileName", Rectangle: [[67, 33], imagelocation("OtherCorner")], increment:true}
This example uses the SuiteInfo function to define the image path for saving the screenshot, as well as defining a capture rectangle
.
CaptureScreen {name:SuiteInfo().ImagesFolder & slash & "CurrentLogo", rectangle:["WelcomeImage", "Border"]}
This example adjusts the image file type, size, and quality, by using the ImageFormat
, ScaleFactor
, and JPGCompressionQuality
properties. It also names the screenshot using both the unchanging name "RegressionTest", as well as the current date.
CaptureScreen {name: "RegressionTest" & today, rectangle: ["TLImage","BRImage"], imageFormat: JPG, scaleFactor: .75, JPGCompressionQuality: 85}
Example: Calling CaptureScreen
with Ordered Parameters
This example shows how to call CaptureScreen
using ordered parameters. It passes all three parameters, declaring the name and location for the image to be saved as the first fileName parameter, then passing a list of coordinates defining the top left and bottom right corners of the area to be captured as the Rectangle parameter, and finally passing "true" as the shouldIncrement parameter.
CaptureScreen "~/Documents/ScreenCaptures/TestScreenshot", [58,579,714,736], true