Finding Images
Eggplant Functional is an image-based testing tool, so it relies heavily on the images that you capture as you create your test. The most reliable images will be those captured using Image Capture Best Practices.
This section discusses common image matching problems and methods of working with images once they have already been captured using the best practices mentioned above.
Most Common Image Matching Problems
The most common image matching problems and their solutions are:
| ### Problem | ### Solution |
|---|---|
| The element of the screen is in a different state than when the image of it was captured. | Use an Image Collection to Find an Image with Several Possible States. |
| The element of the screen does not appear quickly enough, so that it appears after Eggplant Functional has completed a search for it. | Use best practices to avoid timing problems. |
| Multiple instances of the same image are appearing on the screen, and Eggplant Functional is interacting with the wrong one. | Either use the EveryImageLocation() function, or search a specific part of the screen. Both of these approaches are described in Working with Multiple Instances of the Same Image. |
| The element of the screen is displaying at a different size than it was when the image was captured. This could be because the test is being run against a different SUT than the SUT used to create it, such as in cross-device testing with mobile applications, or because the resolution of the SUT was changed. | Search for the image using the Adaptive to Image search type, Image Scaling, or both. |
Finding an Image with Several Possible States
When you are scripting, you can’t always count on images on the SUT to stay the same after you capture them. For instance, if you want to click the close button on a window, that window might or might not be in an active state. On Windows, a close button might appear red or grey depending on whether or not the window is active.
To deal with situations like this, you can create an Image Collection. An Image Collection allows Eggplant Functional to search for more than one image with a single image search. It can then run the same line of code against whichever image in the collection happens to be present. This is part of what gives Eggplant Functional the unique ability to run the same script across multiple browsers, platforms, and/or devices.
Avoiding Timing Problems
Sometimes a script which has run successfully a number of times suddenly reports that it is unable to find an image in the Viewer window. More often than not, this is a timing issue, rather than a problem with image recognition.
There is a quick test to determine whether or not it is a timing problem: select the failed line and click the Run Selection button in the Script Editor. If the line runs successfully, then it is likely that Eggplant Functional tried to search for the image before it became visible on the screen. This is a timing issue. If the image is still not found, examine the Viewer window for changes in the image, or try changing the search type of your saved image.
The following tips can help you prevent timing problems:
The WaitFor Command
The WaitFor command actually holds up the next line of the script until the given image is found. This is particularly helpful when the preceding step opens a new application or web page. (It is not usually needed when you open other parts of an application that is already open.)
Example: Using the WaitFor Command
Click "SaveAs"
WaitFor 5, "SaveDialog"
//Waits for up to 5 seconds for SaveDialog.
TypeText "FileName"
In this example, the script waits up to five seconds for the Save dialog to appear before attempting to type in it. (If the Save dialog does not appear within five seconds, the script fails.)
Be generous in the amount of time you allow for a WaitFor; you don’t need to try and guess exactly how long a wait you need. Since the script proceeds as soon as the image is found, you don’t lose any time on a success. A good rule of thumb is to set the MaxWait time for as long as you think a reasonable user would wait; if that is not enough time, you have probably found a problem in your application.
You can pass properties other than time in a WaitFor, such as a searchRectangle:
WaitFor 2, {ImageName:"MyImage",searchRectangle:[0,0,1920,540]}
WaitFor vs. Wait
The Wait command pauses the script for length of time you specify in the time parameter: (A number by itself represents seconds; you can also type the words minutes and milliseconds.)
During the Wait command, Eggplant Functional does nothing. This is useful if you just want to pause the script at a certain point, but if you are waiting for something to happen on the SUT, you can get more reliable (and potentially faster) results with the WaitFor command.
When you use the WaitFor command, the time you specify is a maximum time, not an absolute time. If the image appears before the maximum time, the script continues immediately.
Example: Comparing Wait and WaitFor commands
Wait 8
// Pauses the script for 8 seconds.
WaitFor 8.0,"OK Button"
// Pauses the script until the image is found, up to 8 seconds.
Remote Work Interval
If you are having frequent, random timing problems with a SUT, you might need to slow Eggplant Functional down a little. You can do this by increasing the Remote Work Interval, the time Eggplant Functional waits between commands sent to the SUT. There are two ways to do this: change the Remote Work Interval in Run Option preferences, or set the RemoteWorkInterval global property.
Remote Work Interval Preference
If most of your SUTs are having frequent timing problems, change the default Remote Work Interval in Run Option preferences.
- Select Eggplant> Preferences > Run > System.
- Increase the Remote Work Interval in small amounts—one or two tenths of a second might be all the difference you need.
The RemoteWorkInterval Global Property
If you only need to slow down the SUT in particular scripts (or parts of scripts) try setting the RemoteWorkInterval global property on a case-by-case basis.
Examples: Changing the RemoteWorkInterval
add .1 to the remoteWorkInterval
subtract .1 from the remoteWorkInterval
setOption remoteWorkInterval, .1
Image Search Time
Image Search Time is the approximate maximum time Eggplant Functional can search for an image before reporting a failure; however, the actual time it takes for a successful search might well be much less because when it finds the image, the search stops. So, it only reaches the Image Search Time limit if the image isn't found.
You can change the default Image Search Time value in Run Option preferences, or on a case-by-case basis within a script using the ImageSearchTime global property. Be careful about setting the Image Search Time too high because this can make your script runs very long if images aren't found.
When you change the Image Search Time, the Search Count and Search Delay values are also changed, and vice versa. Image Search Time is always equal to the product of the Search Delay and one less than the Search Count values:
Image Search Time = (Search Delay) * (Search Count - 1)
Image Search Time Preference
If your Image Search Time is consistently too low, you can change the default value in Run Option preferences. (A side effect of increasing the Image Search Time as a preference is that conditional blocks take a lot longer.)
- Select Eggplant > Preferences > Run > Screen.
- Increase the Image Search Time in small increments- no more than a half-second.
The ImageSearchTime Global Property
If you only need to increase the Image Search Time temporarily (which is usually the best option), you can set the ImageSearchTime global property as needed.
Example: Changing ImageSearchTime temporarily
put getOption (ImageSearchTime) into IST
// Stores the starting ImageSearchTime value.
set the ImageSearchTime to 2
// Changes the ImageSearchTime value.
(*Proceed with the script here, then...*)
setOption ImageSearchTime, IST
// Restores the starting ImageSearchTime value.
Working with Multiple Instances of the Same Image
When identical images appear in more than one place on the screen, you often need a way to specify which one you need. There are a few different approaches to this, and which one you choose will depend on the situation. A few of these methods are described below.
Use a Relative Hot Spot
The hot spot associated with an image is a movable point, and it does not need to stay within the bounds of the image capture.
If you are searching for a folder on the desktop, instead of using EveryImageLocation or specifying a searchRectangle, capture an image of the folder's label, and move the hot spot to the location of the folder relative to its label. Assuming that the label is unique, this will be the easiest way to interact with the folder icon.
For more information, see Using the Hot Spot.
Use EveryImageLocation
The EveryImageLocation function returns a list of every instance of an image match on the screen. This can be used to determine how many instances of an image are on the screen, or find them all in order to specify a particular instance of an image to interact with.
Since Eggplant Functional searches the screen of the SUT from the top left corner to the bottom right corner as if it is reading English text, that is the order in which the images will be listed in the result of the EveryImageLocation function.
For example, if there are multiple instances of an image on the screen, and you know that you will always need to interact with the third instance, you might use code like this:
Put EveryImageLocation("MyImage") into FoundImages
Click item 3 of FoundImages
Searching Part of the Screen
You can search part of the screen by setting the SearchRectangle global property.
The SearchRectangle global property takes two pairs of screen coordinates as parameters; these points define two diagonal corners of the search area. (The top-left corner of the screen is 0,0.)
The purpose of the sample script shown below is to click the Customize button (crossed hammer and wrench) in the active window. Since the Customize button in the active window is identical to the Customize buttons in other windows, there is no guarantee that a “Customize” image you find is the right one; however, the red Close button in the top-left corner of the active window is unique. (It is colorless in background windows.) If you make the red Close button the top-left corner of your search rectangle, Eggplant Functional (searching left-to-right, top-to-bottom) surely finds the Customize button on the same window first.
Example: Adjusting the search rectangle
put ImageLocation("CloseButton") into UpperLeft
put RemoteScreenSize() into LowerRight
Set the SearchRectangle to (UpperLeft, LowerRight)
Click "Customize"
Set the SearchRectangle to ()
-- Restores the search rectangle to the full SUT screen
Image Scaling
By default, Eggplant Functional searches for all images at their native resolution (a scale of 1.0). There are multiple ways to modify the scale at which Eggplant Functional searches for a target image. The most common way to do this is by setting the scale factor. Alternate approaches to scaling include scaling to a size and dynamic scaling.
Scaling by Factor
The most common method for scaling images uses a multiplier of the image's original capture size. You can adjust the image scale factors at a number of different levels; In-line with an image search in your script by setting the Scale parameter, at the script level by setting the scaleFactors global property, or setting the scale factors for a whole suite in that suite's Settings tab. When a script is run, the initial value of the scaleFactors global property is set to the value provided in the Scale Factors field on the suite's settings, which can be accessed by clicking Settings in the bottom left corner of the suite window. This field is given precedence over the scale of an image set in the Image Viewer but is overridden by the ScaleFactors global property and any Scale properties defined with images in-line in your script.
Examples: Scaling by Factor in-line with image searches
Click {image: "OK_button", scale: 0.5)} -- Looks for the image at 50% scale
Click {image: "OK_button", scale: (0.5, 1.0, 1.5)} -- Looks for the image at all 3 sizes, in the order given
Click {image: "OK_button", scale: 0.5 to 1.5 by .25} -- Looks for the provided range of sizes, incrementing by .25 (.5, .75, 1.0, 1.25, 1.5)
Notes: The Image parameter must be used when image scaling is specified in-line.
If the scaleFactors global property is set, this will override it.
Determining Scale Factors
In some cases, it might be difficult to tell what the scale factor is when moving from one device to another or changing the size of elements on the SUT (one use case of this would be testing against the Dock on Mac).
To deal with scenarios like this, you can start with a range of scale factors (as shown in the examples for Scaling by Factor above) and get some additional information to determine what the correct factor was using the foundImageInfo function. The first scale factor at which an image is found is recorded in the result returned by the foundImageInfo function.
Example: Determining the scale factor at which an image was found.
Click {image:"OK_Button", scale: 0.5..1.5 by 0.25}
Log foundImageInfo().Scale -- Logs the scale at which the image was found
When a search is conducted with a Scale other than 1 and the image search type is set to Tolerant of Background, Eggplant Functional will automatically change the search type to Smoothed for Text when the script is run.
Alternate Scaling Methods
While scaling by factor is the most commonly used method of scaling images with Eggplant Functional, these alternate scaling techniques might be more appropriate solutions in specific situations. For instance, scaling to a size allows you to scale non-proportionally, while dynamic scaling allows you to base the search scale on the dimensions of the SUT screen at the time of original capture.
Scaling to a Size
This method of scaling images allows you to specify a specific size for the new image, using the scaleToSize parameter. To do this, set the scaleToSize parameter with any image search. For instance, it can be used with a MoveTo command as part of an imageLocation function.
Example: Scaling to a size
MoveTo imageLocation(Image:"anImage", scaleToSize:[300,100])
Note: This example can result in variations of image dimensions if the new dimensions specified are not in proportion to the original image capture, so that the image appears distorted.
Dynamic Scaling Based on SUT Screen Size
Eggplant Functional records the screen size of the SUT (System Under Test) at the time of image capture. This recorded size can then be used to automatically scale an image based on the difference between the screen size of the SUT at time of capture, compared to the screen size of the SUT at the time of the image search.
To do this, you can specify either the Proportional or Stretch parameter. Neither of these will search for the image at its original capture size.
Proportional:The image will scale the same amount in both directions (width and height), keeping its proportions. It will scale proportional to the smallest change in SUT height or width, so if the screen size of the SUT has changed more in one dimension than the other, this image will scale according to the smaller difference.Stretch:The image will scale in both directions (width and height), not necessarily keeping the same proportions as the original captured image.
Example: Dynamic Scaling
Click {image: "OK_button", scale: ["Proportional", 1.0, 1.5]}
-- Looks for the image at all 3 sizes.
A scale of 1 must be included if you want Eggplant Functional to search for the image at its original capture size
Image Rotation
If you expect an image to appear at a different angle than was originally captured, you can set a rotation parameter with your image search.
The examples below show how to use the Rotate parameter with a Click command.
Examples:
Click {name: "appIcon", rotate: 180}
-- Searches for appIcon at 180 degrees from its original angle
Click {name: "appIcon", rotate: 45 to 90 by 5}
-- Searches for appIcon tilted 45 to 90 degrees, in 5 degree increments