Image and OCR Searches
These commands and functions execute image or optical character recognition (OCR) searches on the system under test (SUT).
You can further control the search behavior of the image and OCR search commands and functions below by using the ImageSearchCount, the ImageSearchDelay, and the ImageSearchTime global properties.
EveryImageLocation Function
Behavior: Searches the SUT for all occurrences of the given images or text (OCR); returns a list of the hot spot coordinates for every occurrence of the images, or a list for the center of the text string when using OCR. Coordinates are listed in the order in which they are found, where images and text closest to the top left corner of the SUT are found first. If no image or text (OCR) is found, EveryImageLocation returns an empty list.
Parameters: One or more Image References or properties, or Text Properties (for OCR).
Syntax:
EveryImageLocation( imageOrTextReference )
Returns: The coordinates of every instance of the given image or images. For example:
[[288,325],[288,356],[288,387],[288,448]]
Example:
Including the ImageName property is required when passing additional image search properties as in this example.
log EveryImageLocation(ImageName:"CheckBoxes",scale:[1,.5]) // Logs a list of hot spot coordinates for every occurrence of the image found at both the original size and half scale. Note the use of the ImageName property when including additional properties for the image search.
Example:
click the penultimate item of EveryImageLocation(text:"AddressField") // Clicks the penultimate (second to last) item in the list of occurrences of the text (OCR).
Example:
set the RemoteWorkInterval to .1 // Shortens the RemoteWorkInterval (default=.7) to decrease the pause between actions that Eggplant Functional sends to your SUT. This change allows the series of clicks to execute much faster.
repeat with EveryImageLocation("RadioButton") // Iterates through each occurrence of the image found with everyImageLocation().
click it // Using the it variable, clicks the current occurrence the repeat loop is operating on.
end repeat
set the RemoteWorkInterval to .7
Example:
// The following example provides an alternate syntax for the previous example.
repeat with each item of EveryImageLocation("RadioButton")
click it // Using the it variable, clicks the current occurrence the repeat loop is operating on.
end repeat
When you call the EveryImageLocation function, you can immediately call the result to return ImageInfo for every image represented in EveryImageLocation.
Example:
put EveryImageLocation("GreenSaveButton","BlueSaveButton","RedSaveButton") // Returns a list of hot spot coordinates for every occurrence of the image.
put the result // Returns imageInfo for every image represented in EveryImageLocation.
EveryImageRectangle Function
Behavior: Searches the SUT for all occurrences of the given images or text (OCR); returns a list of the rectangle coordinates for every occurrence of the images or text (OCR). Coordinates are listed in the order in which they are found, where images and text closest to the top left corner of the SUT are found first. If no image or text (OCR) is found, EveryImageRectangle returns an empty list.
Parameters: One or more Image References or properties, or Text Properties (for OCR).
Syntax:
EveryImageRectangle( imageOrTextReference )
Returns: The rectangle coordinates of every instance of the given image or images.
Example:
put EveryImageRectangle("TotalAmountLabel", "ItemizedAmountLabel") // Returns a list of rectangle coordinates for every occurrence of the images.
Example:
assert that the number of items in EveryImageRectangle(ImageName:"CheckBoxes",searchRectangle:["SelectOptionsUpperLeft","SelectOptionsLowerRight"]) is 4 // Asserts whether the number of occurrences of the image is equal to the expected number, 4. Example output: AssertionFailed (assertion: the number of items in EveryImageRectangle(ImageName:"ImageName") is 4; actually: 5 IS NOT equal to 4)
Example:
Set StatusRectangles to EveryImageRectangle(text:"StatusLabel") // Stores the return of EveryImageRectangle in a variable called StatusRectangles.
Log "There are"&&the number of items of StatusRectangles&&"statuses to check." // Uses concatenation to join two strings with the number of occurrences in the list stored in StatusRectangles, and logs the output.
repeat with each Rectangle of StatusRectangles // Iterates on each item in the list using the variable name Rectangle.
Set StatusIndicatorLocation to the TopRight of Rectangle plus (15,0) // Makes a coordinate adjustment 15 pixels to the right of the top right corner of the current occurrence and stores it in variable StatusIndicatorLocation.
If ColorAtLocation(StatusIndicatorLocation) is "59,170,43" then Log "The status is green." // Uses the ColorAtLocation() function to find the RGB value at StatusIndicatorLocation, then compares it to the expected RGB value, 59,170,43.
end repeat
When you call the EveryImageRectangle function, you can immediately call the result to return ImageInfo for every image represented in EveryImageRectangle.
Example:
put EveryImageRectangle("GreenSaveButton","BlueSaveButton","RedSaveButton") // Returns a list of rectangle coordinates for every occurrence of these images.
put the result // Returns imageInfo for every image represented in EveryImageRectangle.
ImageFound Function
Behavior: Searches for the given image(s) or text (for OCR) on the SUT via the Viewer window; returns true if the image/text is found, and false if it is not found. A maximum wait time parameter allows time for an image/text to be found before false is returned.
Parameters: One or more Image References or properties, or Text Properties (for OCR).
Syntax:
ImageFound( imageOrTextReference )
Returns: True or False.
Example:
if ImageFound(ImageName:"OKButton") then // Checks whether the image is found.
click FoundImageLocation() // If ImageFound returns true, uses the FoundImageLocation function to click the stored location where the image was found by the ImageFound function.
else
TypeText AltKey,F4 // If ImageFound returns false, executes a keystroke to close the application.
end if
Example:
Including the ImageName property is required when passing additional image search properties as in this example.
Repeat until ImageFound(ImageName:"NextButton",WaitFor:0) // This repeat loop iterates until the ImageFound function returns true.
Typetext PageDown // Sends a Page Down keyboard action to scroll down the page.
Wait 2 // Waits 2 seconds to allow the application to stop moving after the scrolling action.
if the repeatindex is greater than 5 then throw "Image Not Found:", "NextButton not found when scrolling." // This conditional statement checks to see whether the repeat loop has iterated more than 5 times, and if so, throws an exception to stop the execution.
end repeat
Example:
To find text using imagefound(), provide a text property. Setting a search area using SearchRectangle() is useful when searching for text.
if imagefound(text:"Hello!", SearchRectangle:("Image1","Image2"))
then
click foundimagelocation()
log "clicked it!"
else
logerror "didn't find the word"
end if
A maximum time of 0 causes Eggplant Functional to perform an immediate search, without refreshing the Viewer window or repositioning the mouse in a further attempt to find the image. This is the fastest way to search for an image that may not be present.
ImageLocation Function
Behavior: Searches for given image or text (OCR) on the SUT; returns the hot spot coordinates or the center of the text (OCR) rectangle of the first occurrence of the first image found. If no image is found, an exception is thrown.
Parameters: One or more Image References or properties, or Text Properties (for OCR).
Syntax:
ImageLocation( imageOrTextReference )
Returns: Coordinates of the hot spot location for the image or the center of text (OCR) rectangle. For example: [338, 335]
Example:
set SectionHeaderLocation to ImageLocation(text:"Billing Address") // Stores the return of ImageLocation in a variable called SectionHeaderLocation. Because it is a text (OCR) search, the return is the coordinates of the center of the "Billing Address" rectangle.
Example:
click ImageLocation("NameLabel") + [50,-10] // Clicks 50 pixels to the right and 10 pixels up from the hot spot location of the image.
Example:
tap {ImageName:"OKButton",searchRectangle:["PopupHeader",ImageLocation("LowerRightAnchor") - [400,500]]} // Sets the lower right corner of the searchRectangle, adjusted 400 pixels up and 500 pixels to the left of the hot spot of the LowerRightAnchor image.
Example:
By storing the location of the scroll bar down button and clicking the stored location, Eggplant Functional has to search for the image only once, which increases the execution speed of clicking the button 10 times:
set ScrollDownButtonLocation to ImageLocation("ScrollDownButton")
repeat 10 times
click ScrollDownButtonLocation
end repeat
ImageRectangle Function
Behavior: Searches for given images or text (OCR) on the SUT; returns the rectangle coordinates of the first occurrence of the first image found. For image searches, the ImageRectangle honors the size and shape of the captured image, unless the ClipRectangle image search property is in use. For OCR searches, the ImageRectangle is automatically fitted around the text string as it appears on the SUT. If no image or text (OCR) is found, an exception is thrown.
Parameters: One or more Image References.
Syntax:
ImageRectangle( imageOrTextReference )
Returns: The rectangle coordinates of the found image. For example: [348,451,460,554]
Example:
put ImageRectangle("CompanyLogo") // Prints the return of ImageRectangle in the Run window only.
Example:
log "The coordinates of the rectangle are" && ImageRectangle(ImageName:"PhoneTypeLabel",searchRectangle:["PhoneContactHeader","SectionDivider"]) // Uses concatenation to join a string and the return of ImageRectangle with a space between them. Note the use of the ImageName property when including additional properties for the image search.
Example:
click the topLeft of ImageRectangle(ImageName:"EnableSetting") // The ImageRectangle Function can be combined with commands that control the SUT. Note the use of the topLeft function to select a specific corner of the ImageRectangle.
Example:
set NameFieldRectangle to ImageRectangle(Text:"Name") // Stores the rectangle coordinates of the found text (OCR) into a variable.
set NameFieldValue to ReadText(Right of NameFieldRectangle, Top of NameFieldRectangle, Width of the RemoteScreenSize, Bottom of NameFieldRectangle) // Uses ReadText to read the area directly to the right of the NameFieldLocation, all the way to the right edge of the SUT screen.
WaitFor Command, WaitFor Function
Behavior: Halts the script execution until the image or text (OCR) is found on the SUT. If no image or text (OCR) is found within the specified wait time, an exception is thrown. The actual wait time can be higher than the specified wait time because the wait time adjusts automatically to factors such as SUT resolution, Discrepancy settings, network latency, and image collection size.
Parameters: A wait time, one or more Image References or properties, or Text Properties (for OCR). The default unit of wait time is seconds.
Command Syntax:
WaitFor time , imageOrTextReference1 {, imageOrTextReference2 ...}Function Syntax:
WaitFor( time , imageOrTextReference1 {, imageOrTextReference2 ...} )
Returns: The length of time spent waiting until one of the specified elements was found. When WaitFor is called as a command, the returned value is available on the next line of the script using the result.
Example:
WaitFor 6.5, "OKButton", "ExitButton" // Waits up to 6.5 seconds for either image to appear.
Example:
WaitFor 2 minutes, {ImageName:"InstallationCompleteMessage",searchRectangle:["InstallProgressDialogUpperLeft","InstallProgressDialogLowerRight"]}
Put the result into elapsedTime // record the actual time that passed until the image was found
WaitFor can also be used as a property of any image or text (OCR) search command. Like the WaitFor command, the WaitFor property is the maximum time Eggplant Functional waits for the given image to appear on the SUT.
Example:
Click {ImageName: "Done_button", waitFor: 2 minutes}
You can also use the ImageSearchTime Global Property to affect the search time for a section of image or text (OCR) searches.
The wait command is distinct from the waitFor and waitForAll commands in that the script always waits the specific amount of time you set, and then the script proceeds to the next step. The waitFor and waitForAll commands wait only until the specified element is detected on the SUT screen; if the element isn't detected within the wait time, an exception is thrown.
Related:
Wait: Use this command to insert a wait of a specific amount of time.WaitForNotFound: Use this command to wait for an element to disappear from the screen.WaitForAll: Use this command to wait for a number of elements to all be found simultaneously.
WaitForNotFound Command, WaitForNotFound Function
Behavior: Halts the script execution until none of the specified elements is found on the SUT. This command searches the SUT for one or more screen elements (images or text). If any of the given elements is found on the SUT, the search is repeated until none are found. If the specified wait time elapses without the element(s) being gone from the SUT, an exception is thrown.
WaitForNotFound is functionally the opposite of the WaitFor command. Where WaitFor waits a length of time for something to appear on the screen, WaitForNotFound waits a length of time for something to be gone from the screen.
Parameters: A wait time, one or more Image References or properties, or Text Properties (for OCR). The default unit of wait time is seconds.
Command Syntax:
WaitForNotFound time , imageOrTextReference1 {, imageOrTextReference2 ...}Function Syntax:
WaitForNotFound( time , imageOrTextReference1 {, imageOrTextReference2 ...} )
Returns: The length of time spent waiting until none of the specified elements were found. When WaitForNotFound is called as a command, the returned value is available on the next line of the script using the result.
Example:
WaitForNotFound 6.5, "OKButton", "ExitButton" // Waits up to 6.5 seconds until neither image is found.
Example:
WaitForNotFound 2 minutes, {ImageName:"InstallationCompleteMessage",searchRectangle:["InstallProgressDialogUpperLeft","InstallProgressDialogLowerRight"]} // Waits until the specified image is gone
Put the result into timeToDismiss // Records the actual time taken until the image was gone
Related:
WaitFor: Use this command to wait for an element to be displayed.
WaitForAll Command
Behavior: Halts the script execution until all of the image or text (OCR) parameters are found in the Viewer window. If not all of the elements are found simultaneously within the maximum wait time, an exception is thrown.
Parameters: A wait time, and one or more Image References or properties, or Text Properties (for OCR). The default unit of wait time is seconds.
Syntax:
WaitForAll time , imageOrTextReference1 {, imageOrTextReference2 ...}
Example:
In this instance, WaitForAll is waiting up to 15 seconds for a single text string, "Aretha Franklin".
WaitForAll 15, text:"Aretha Franklin"
Example:
If you need to wait for two images, the names can be passed as strings, like this:
WaitForAll 15, "OKButton", "ExitButton"
Example:
In order to use WaitForAll to search for images and text strings simultaneously, or multiple text strings simultaneously, you must pass separate property lists for each, as shown in the below examples.
WaitForAll 30, {text:"Updates Available"}, {image:"OKButton"}
WaitForAll 15, {text:"Diana Ross"}, {text:"Baby Love"}
Example:
WaitForAll 5, {image:"Welcome"}, {text:"Susan",CaseSensitive:"yes"}
If an enclosed list of images is passed as a WaitForAll parameter, Eggplant Functional considers that parameter "found" when any one of the items on the list is found.
Example:
WaitForAll 20, "crust", "sauce", ["peppers", "olives"]
This command has three parameters: "crust", "sauce", and a list of two other items (enclosed in square brackets). It can be thought to mean:
WaitFor20, "crust" // and simultaneously...
WaitFor 20, "sauce" // and simultaneously...
WaitFor 20, "peppers", "olives" // ... and simultaneously wait for either of these items
The wait command is distinct from the waitFor and waitForAll commands in that the script always waits the specific amount of time you set, and then the script proceeds to the next step. The waitFor and waitForAll commands wait only until the specified element is detected on the SUT screen; if the element isn't detected within the wait time, an exception is thrown.
Related:
Wait: Use this command to insert a wait of a specific amount of time.
WaitForColor Command
Behavior: Halts the next line of script execution until the specified color is found in the Viewer window. If the color is found within the maximum wait time, an exception is thrown.
Parameters: A wait time, a color, and a rectangle that you've defined. The waitForColor command accepts colors in any format supported by Eggplant Functional, as well as any rectangle formatting supported in Eggplant Functional.
Syntax:
WaitForColor time , color , rectangle
Example:
WaitForColor 5, "blue", [22,576,86,640]
SetSearchRectangle Command (Deprecated)
The SetSearchRectangle command has been deprecated by the SearchRectangle Eggplant Functional global property. Setting the SearchRectangle Eggplant Functional global property is the recommended way to set a search rectangle. Use the SetSearchrectangle command shown here as an alternative in certain situations.
Behavior: Takes two pairs of screen coordinates as parameters; these points define two diagonal corners of the search area.
Parameters: Takes two sets of coordinates or two images, defining the top left corner and bottom right corner of an area of the screen.
The top left corner of the SUTscreen is always 0,0.
Syntax:
SetSearchRectangle( rectangle )
Example:
SetSearchRectangle [[0,0],[100,100]]
Example:
SetSearchRectangle ["TLImage","BRImage"]