Image Capture Best Practices

Eggplant Functional is an image-based test automation tool, so capturing images the right way is integral to the success of your tests. This page and the following video describe image capture practices that can make your tests more successful.

Setting Up Image Capture

In the Viewer window, you can move the Capture Area by clicking the desired location, dragging the Capture Area, or pressing the arrow keys. Pressing an arrow key alone moves the Capture Area one pixel in the given direction; to move the Capture Area in 10-pixel increments, hold down Shift as you press an arrow key.

To resize the Capture Area, drag the edges or corners of the Capture Area, or press Alt+arrow (Option+arrow on Mac) keys. Add the shift key to resize the Capture Area in 10-pixel increments.

To adjust the image scale to offer a better perspective for screen actions, click to enlarge the image size and to reduce the image size.

note

The initial view of the captured image is shown at a 1:1 scale.

To move the image hot spot, Ctrl+click (Cmd+click on Mac) the desired location, drag the hot spot, or press the arrow keys. Add the Shift key to move the hot spot in 10-pixel increments.

Identifying Good Images

An easy way to keep Eggplant Functional scripts as robust as possible is to capture images with just enough content to uniquely identify an interface element. For example, here is an image of a folder on a desktop:

Folder with desktop background

If you capture some of the desktop in your image, you always have to rely on the desktop color to be the same for a match. Even if you can always count on the desktop being the same color, in this example, it is shaded when the folder icon is selected:

Selected folder with dark blue area in the background

Of course, you could always capture an image of the folder with the selected background as well, but since the icon itself does not change, this is an unnecessary complication. If you capture the icon without the desktop in the background, you can cover both states with one image:

Folder with no background

This image might look less like a folder icon to you, but shape isn’t especially important to Eggplant Functional. As long as the inside of the image does not match anything else on the screen, the edges are not important.

When you have the Capture Area in roughly the size and position you need, you can use the arrow keys to fine tune it:

• To nudge the Capture Area one pixel at a time, press the arrow keys.
• To adjust the size of the Capture Area one pixel at a time, press Alt+arrow (Option+arrow on Mac) keys.
• To nudge the hot spot one pixel at a time, use Ctrl+arrow (Cmd+arrow on Mac) keys.

Add Shift to any of these shortcuts to make the adjustments in ten-pixel increments.

Using the Hot Spot

The hot spot is the point that Eggplant Functional targets for mouse actions such as Click, DoubleClick, and MoveTo. It is also the point used to describe the location of an image found in the Viewer window.

The position of the hot spot is defined as an (x, y) offset relative to the upper-left corner of the captured image. For example, if you capture an image that is 20 pixels wide and 10 pixels high, and you leave the hot spot at its default location in the center of the image, the hot spot location is reported as (10, 5)—10 pixels to the right of and 5 pixels down from the upper-left corner.

The Relative Hot Spot

Although the hot spot is associated with an image, it doesn’t have to be inside the image. For example, if you need to select text in a text box, there is no way to do an image match unless you know in advance what the text is going to be. Instead, you can capture an image of the text box’s label, and set the hot spot a few pixels over, where the actual text box begins, as shown in the image below:

An image captured of a text box's label, as seen in the Viewer Window. The hot spot is moved outside the captured image to where the text box is expected to display.

Another example is a Country pop-up menu that can display any one of dozens of countries. Instead of capturing an image of each possibility, you could capture the label Country with the hot spot on the actual pop-up menu. Since the pop-up menu is not included in the image, the country that is displayed has no bearing on the image match.

Moving the Hot Spot

You can change an image's hot spot in the Viewer window, the Capture Image panel, an image property list in a script, and in the Images pane of the Suite Editor.

There are several ways to move a hot spot:

• Ctrl+click in or around an image. (Cmd+click on Mac.)
• Ctrl+drag the red crosshairs that indicate the hot spot. (Cmd+drag on Mac.)
• Press Ctrl+arrow keys to nudge the hot spot one pixel at a time. (Cmd+arrow on Mac.)
• Press Shift+Ctrl+arrow to nudge the hot spot ten pixels at a time. (Shift+Cmd+arrow on Mac.)
• In the Viewer window, resize the Capture Area to snap the hot spot back to the center of the image.

Choosing the Best Search Type

When you save an image, the Capture Image panel gives you a choice of search types. Eggplant Functional tries to automatically detect the appropriate search type for your image, but in some cases you might want to override the default setting. If an Eggplant Functional script suddenly fails to find an image that it found on a previous run, try changing the search type in the Info panel of the Images pane. Search type changes are saved immediately, and the new search type is used the next time a script calls that image.

Important

The search types visible will depend on whether or not you have Show all search types enabled in your General Preferences. By default, three search types will be visible: Tolerant of Background, Smoothed for Text, or Adaptive to Image. Selecting this checkbox in your preferences will allow Eggplant Functional to also display Precise to Pixel, Pulsing to Element, and Smooth and Pulsing. Precise and Pulsing can be chosen by Eggplant Functional as the default search type, even if those are not shown.

Tolerant of Background

Typically, Eggplant Functional chooses the Tolerant of Background search type. The Tolerant setting works well for most images, and allows for some variation in the colors of the image when it is found. A good example is a semi-transparent window or menu. In the images below, you can see objects in the background through the menu on the right. The Tolerant setting allows Eggplant Functional to accurately locate menu items, even if the screen behind the menu is in a different state than it was when the image was captured.

Basic menu appearance

Menu with desktop items showing through Precise to Pixel

Smoothed for Text

This search type is designed to deal with text and buttons that sometimes change their appearance subtly, primarily due to Mac text anti-aliasing. If you are testing against Mac systems, using the Smooth (or Smooth and Pulsing) search type can make a big difference in your images that include text.

Adaptive to Image

This search type uses a flexible search algorithm to analyze individual pixels and then also look at the overall similarity before determining a match. The effect is that of seeing the image as a whole, similar to how a human would see it. This makes it significantly easier to match images across different devices as resolution and scale change. Because of this, when doing cross-device or cross-platform testing, you might want to use the Adaptive to Image search type along with Image Scaling.

Precise to Pixel

The Precise to Pixel search type requires a very high degree of precision to consider an image matched on the SUT. This setting is useful for low contrast images or images with subtle color differences.

Pulsing to Element

Pulsing buttons, such as the Save buttons in some versions of Mac, can also be detected automatically by Eggplant Functional. The Pulsing setting causes Eggplant Functional to use a "mask" on the image to filter out the pixels that are changing.

Light end of pulse

Dark end of pulse

Captured image with Pulsing mask applied

Smooth and Pulsing

Similar to Smoothed for Text, the Smooth and Pulsing search type accounts for anti-aliasing of text, particularly on Mac.

note

The above search types—other than Adaptive to Image—were previously known as the following, respectively: Tolerant, Smooth, Precise, Pulsing, and Smooth and Pulsing.

Capturing Tooltips and Other Transient GUI Elements

The Freeze Screen option makes it easy to capture images of tooltips and other on-screen objects that come and go quickly. To use Freeze Screen:

1. In the Viewer window, move the mouse to a position in your SUT that brings up the element that you want to capture.
2. Press Control (Command on Mac) to toggle into Capture Mode.
3. To freeze the screen, press Ctrl+5 (Cmd+5 on Mac), or go to Control > Freeze Screen. We recommend using the keyboard shortcut, as tooltips and other transient images might disappear in the time it takes to navigate to Freeze Screen via the Control menu.
4. Move the Capture Area to the object you want to capture.
5. Click Capture Image or double-click the Capture Area to capture the image.

Keeping the Tooltip During Script Execution

When you are trying to capture a tooltip, moving the mouse out of the way can cause a problem, because the tooltip disappears as soon as the mouse moves away. To remedy this behavior, the default value of the shouldRepositionMouse global property is no.This setting means that, if Eggplant Functional fails to find an image it is looking for on the first search, it does not move the mouse to the lower right-hand corner of the screen.

If hovering over an element obscures an element, it might make sense to have the cursor automatically get out of the way. To accomplish this change, set the shouldRepositionMouse global property to yes. Doing so changes Eggplant Functional's behavior such that if it fails to find an image it is looking for on the first search, it moves the mouse to the lower right-hand corner of the screen. This change makes sure the image is no longer being obscured by the cursor.

Comparing Images with Graphics Software

If Eggplant Functional continuously fails to find a particular image, and you have ruled out search type and timing problems using the Image Update Panel, it might be that the SUT is not displaying the image consistently.

In these rare cases, it is sometimes helpful (or at least informative) to be able to perform a detailed comparison of the captured image to one being displayed by the SUT.

The procedure outlined below describes how to view the differences between these images on a pixel-by-pixel basis. Adobe Photoshop Elements is used here, but you can adapt this technique for use with the graphics program you typically use.

1. Open both images. In the graphics program, open the saved image and the Screen_Error file for a script run that failed because of this image.
2. Overlap the images. Copy the saved image, and paste it on top of the Screen_Error file. In Photoshop, the copied image is automatically pasted into a separate layer.
3. Align the images. Use the Move tool to drag the saved image to its corresponding location in the Screen_Error file. Then zoom in and adjust the image further by nudging it with the arrow keys.
4. Highlight the differences. In the Layers palette, click the Blending Mode pop-up menu and choose Difference.
5. Evaluate the differences. Pixels that match perfectly are displayed as black. For other pixels, look at the Info panel to see the RGB values for each image. The greatest difference between the three values is the tolerance Eggplant Functional must allow to consider that pixel a match in both images. For example, if a pixel in the saved image has RGB values (99, 99, 135), and the same pixel in the Screen_Error has (100, 100, 150), the search tolerance must be at least 15, or the difference between the blue values.

Reusing Captured Images

When you are scripting, you often need to use the same image more than once within a suite or script. For example, you might click the File menu repeatedly, or open a number of dialogs with OK buttons.

Here are four easy ways to insert a previously-saved image into your script:

• Alt+click (Option+click on Mac): In the Viewer window toolbar, Alt+click (Option+click on Mac) a command button. Instead of capturing an image, the command opens a Use Image panel so you can select an existing image. The command that you click is both added to the script and performed on the SUT.
• Insert Pop-up Menu: In the Script Editor, choose a command (or Additional Image for no command) from the Insert pop-up menu, then select your image in the file browser. The command you choose here is not performed on the SUT.
• Drag-and-drop: Drag an image from the Images pane of the Suite Editor to the script.
• Copy and paste: Copy and paste from another part of the script. You can always type an image name in quotation marks, too.