The Image Update Panels

The Image Update panels help you manage, modify, and work with previously existing images during and after runtime. The panels are triggered when an image or text search is unsuccessful. When a search failure occurs, depending on the settings selected, you may either see the Image Update panel, or the Auto Update panel (this is shown following completion of the test run).

The Image Update panel is shown during runtime at the time of search failure. It allows you to change existing image search settings, capture new images, create collections, or replace previously existing images in order to remedy the search failure before continuing the test run. The Auto Update panel is displayed following a test run where an image search failure occurred, but instead of triggering the Image Update panel Auto Update was engaged, modifying the search settings in order to find a match. In cases where that match is successfully found by Auto Update, the Auto Update panel appears following the test run, allowing you to save any of the changes it made to the search settings.

Note: The Image Update panel was previously referred to as the Image Doctor. Menu options used to enable the Image Update panel also apply to the OCR Update Panel.

See the Image Diagnostics section on The Image Viewer page for information about image search settings and selecting diagnostic searches.

Enabling the Image Update Panel

The Image Update panel lets you work with image diagnostics and fix image searches when an image is not found during a script run. Depending on the selected settings, this panel also appears when multiple instances of an image are found that were not anticipated by the script, and can also trigger the Auto Update functionality, allowing you to save any changes made to images during the script run.

You can enable the Image Update panel and related Auto Update panel functionality on the main menu by adjusting the options shown in the Run > Image Update menu.

Note: This functionality is the same as for the OCR Update panel. Both features are enabled or disabled using this menu option.

Image Update Menu in Eggplant Functional

The Image Update panel menu, found under Run>Image Update

Your choices when using the Image Update menu are as follows.

Note: The Image Update panel will not display if you set the Throw Exception (Update Off) option. The panel appears at runtime when you choose the manual setting using the Guided Update (Show Panel) option. If Auto Update (Log Changes) is selected, then the Auto Update panel is displayed following the script run.

When Image/Text is not found:

  • Update Off (Throw Exception): Eggplant Functional throws an exception and the script fails when an image or OCR search does not return a match. The Image Update functionality is not used.
  • Guided Update (Show Panel): This option opens the Image Update panel during runtime when an image search fails, so that you can adjust settings to find an appropriate match. This option lets you use the Image Update panel interactively, which gives the most complete control over text matches. This method is best for attended test runs. This option was formerly called "Show Panel (Manual Update)".
  • Auto Update (Log Changes): Eggplant Functional logs a warning, and then tries various search settings to find the image or text that wasn't found. If it finds the image with the new settings, it logs which settings were used, and then continues the script run using those new settings. When you use the Auto Update (Log Changes) option, changes made during the script run are applied in memory only.However, once a script run has completed, the Auto Update panel appears, which allows you to decide which of the changes it made during the script run you want to save. Applying these changes modifies the image search settings for future test runs, and will affect any other tests using that image. When applying changes, check your test to see where the image is used, as changes saved to the image search settings may conflict with your script.

    The Auto Update option is a good choice for unattended script runs, as it does not open the Image Update panel during the script run. Frequently, it can find a match and continues the script run when otherwise the search would throw an exception and fail. Please note, however, that Auto Update (Log Changes) does cause the Auto Update panel to open at the end of each script run where search remedies were applied. Additionally, if your parameters are too broad when using Auto Update, you could end up matching the wrong image with unintended results.

    For more on using the Auto Update panel, see Using the Auto Update Panel.

When multiple occurrences are found:

  • Use First Location: Uses the first instance of the image found by Eggplant Functional.
  • Show Panel: The Image Update panel is displayed with the message "Multiple Instances Found", so you can interactively change settings and specify how you want to handle the multiple instances found.
  • Log Warning: Logs a warning in the script results. This does not affect the result of your script but will be recorded in the script results.
  • Throw Exception: Throws an exception, which will cause the running script to fail. This will be recorded in the script results.

Other actions:

Note: These options only display messages when the main panel setting regarding its response to a failure to find an image or text is set to Guided Update (Show Panel). See When Image/Text is Not Found.

  • Validate ImageFound: When a script contains Imagefound(<image_name>) and Eggplant Functional is unable to find a match for the image or text specified, it displays the Image Update panel with the validation message "ImageFound() Will Return False". Then, you can interact with the panel to determine how to handle the text or image not being found.
  • Validate Successful Image Searches: When a script contains an ImageFound function, such as ImageFound(<image_name>), and the text or image specified is successfully found, the Image Update panel is displayed with the validation message "Validate Image Found".
    Validating successful image searches can be useful in the rare situation where you need to verify that Eggplant Functional is finding the correct image. For instance, you can use Validate Successful Image Searches to review your application if visual elements are updated to match new UI elements, to make sure the script isn't still finding matches to old images.

    Note: Selecting this menu item changes The ValidateSuccessfulSearches Global Property's default setting to On.

Using the Image Update Panel

The Image Update panel comprises three tabs, described in more detail below: The Start tab, the Diagnostics tab, and the Properties tab. Each tab allows you to make adjustments that make your Eggplant Functional scripts more successful.

All of the tabs in the Image Update panel include the Manual Update drop-down menu. The Manual Update drop-down menu includes the selections for the Image Update options explained in Enabling the Image Update Panel. As you make changes to the Image Update options, you must click Try Again to continue the script with the new preference.

The Start Tab

Image Update panel, Start tab

The Start tab in the Image Update panel

The Image Update panel opens with the Start tab displayed. The Suggested Remedies section of the panel offers options to fix or replace the image that can't be found based on the following diagnostic searches performed by Eggplant Functional:

  • Standard Search: Searches for the image again using the original specifications. A result for this search means that the image has appeared since the first failure to find the image. Most likely this result means that the timing of the script at the step needs to be adjusted to allow more time for the image to appear.

  • Dynamic Tolerance: Searches for the image while increasing the RGB color tolerance value up by a particular increment (default = 5) until a match is found. A result for this diagnostic means that the RGB values of the image have shifted.

  • Alternate Types: Searches for the image while using the different SearchType filters, which are Smoothed for Text, Adaptive to Image,Pulsing to Element, and Smooth and Pulsing. If the image is text-based, a result indicating Smoothed for Text suggests that the system under test (SUT) is using anti-aliasing in its text. A result for Pulsing to Element or Smooth and Pulsing suggests that the background portion of the image either has a pulsing effect or perhaps another effect such as a variable gradient. For more on these search types, see Search Type.

  • Scaling Search: Searches for the image at a different scale than the original image.

  • Discrepancy Search: Performs the search while allowing a certain percentage or number of the pixels in the image to differ. The percentage of pixels (with a percent sign) or number of pixels (with no percent sign) by which the image and a match in the Viewer window can differ.

    Note: Discrepancy can be slow to use and can result in false positive matches. Preferable approaches include setting the Search Type to Smoothed for Text in the Image Update panel, cropping the image using the Image Editor, or creating an Image Collection that includes several acceptable versions of a basic image. You can also capture a new image if the current image can only be found using a discrepancy search.
  • OCR Search: Scans the original captured image for text, and then conducts a search for that text. For example, if the original image is of a button that says Search on it, this search will use OCR to look for the word Search on the screen.

  • Prior Location: References the results for the current script in which the current line of code was successfully run. Highlights the location on the screen where the image was found in that successful run of this test. The results that this search uses can be specified manually using the Mark As Prior Run button in the results portion of the suite for any script.

    Note: This search is not used when the Image Update panel menu is set to use the Auto Update (Log Changes) option.
  • Original Location: References the coordinate pair of the location at which the image was first captured.

  • Crop Edges: Searches by progressively cropping in on the image, looking for smaller and smaller images. As long as the inside of the image does not match anything else on the screen, the edges are not important.
Note: The remedies that appear in the Image Update panel provide a fix that either allows the image to be found exactly once on the current SUT screen or offers the possibility of capturing a new image at a specific location on the screen. Each option shows the image that would be captured from the current screen if used.

Selecting a Suggested Remedy shows a highlight rectangle on the SUT screen in the color for that diagnostic. The Remedy drop-down list changes depending on the option you choose. Available remedies are:

  • Apply Fix: This option performs the selected fix to the current image.
  • Replace Image: This option automatically recaptures the image without changing any of its properties. The search is then run again.
  • Add Representation: This option adds a new representation of the image to the current suite. The new representation is added to the same collection as the original image, and Eggplant Functional suggests a name for the image in the New Representation field.
  • Capture to Helper: This option captures a new version of the desired image and saves it in a helper suite. If you select this option, a drop-down menu appears, listing available suites to which the image can be captured

Select a Remedy action from the drop-down list, then click Do This. For the options listed above, Eggplant Functional will fix the image and try the search again.

Note: If you select Standard Search as the remedy, none of the Remedy actions are enabled. Clicking Do This will log a warning telling you to change the timing in the script before trying again.

Enter a name in the New Representation field. If you choose to add a new representation of this image to a collection, the name in this field is used. If you alter the naming convention used in this field, it will remember the convention for further image updates.

The Diagnostics Tab

The Diagnostics tab lists the results of diagnostic searches so that you can see what settings can be adjusted or what search types can be used in order to find the image on the screen. The options listed correlate to the options in the Start tab, described above.

To use the Fix, Recapture, or Add Rep (which stands for Add Representation) buttons, select an appropriate diagnostic search that returned positive results. For instance, in the example below, the Original Location can be used to Fix the image:

Diagnostics tab, with Pulsing Search selected

The Diagnostics tab, with Scaling Search selected

In the following example, selecting Original Location enables the Recapture and Add Rep options:

Diagnostics tab in Image Update window, Original Location selected

The Diagnostics tab in the Image Update panel

Recapture will capture the image again using the location where the selected search type is finding the image. Add Rep will add a representation of the image to an image collection, or create a collection if one does not already exist.

The Properties Tab

Properties tab of Image Update window

The Properties tab, visible in the Image Update panel

The Properties tab gives you the ability to manually adjust different image settings without permanently altering the state of the captured image in your suite. You can change the Search Type, Tolerance, Discrepancy, Scale, and Rotate values in this panel, then press Enter to conduct a search of the current active connection to see what settings work best. The original settings will show to the right in red. To fix the image permanently, click Save Changes. To revert to the original settings, click Revert Changes.

Working with Images in the Viewer Window

Image diagnostics in the Viewer Window

The Viewer window during an Image Update panel Diagnostic Search

When an image is not found, or multiple instances of an image are found, the Image Update panel appears and conducts a variety of diagnostic searches. The results are highlighted in the remote screen window. Boxes of corresponding colors are shown in every location on the screen of the SUT where an image is found with one of the diagnostic searches.

Hovering over one of these colored squares turns the square white.

Highlighted image match in Viewer Window

A Highlighted image match in the Viewer window

If clicked, the capture area snaps to that box, and you can use that selection to add a representation for an image collection, or fix the current image that you are working with in the Image Update panel.

Viewer Window diagnostics mode, with menu arrow

A selected image match in the Viewer window

Clicking the arrow to the right of the capture area in this instance brings up the following drop-down menu:

Viewer window image update contextual menu

The capture area drop-down menu in the Viewer window

Options are to:

  • Add Representation and Try Again: Captures an image using the selected image-capture bounds and adds it to the current collection or creates a collection using the current image and the newly captured image. Then, it tries the search again with the new/ updated image collection.
  • Add Representation: Captures an image using the selected image-capture bounds and adds it to the current collection or creates a collection using the current image and the newly captured image.
  • Recapture and Try Again: Captures an image using the selected image-capture bounds and replaces the current image with this new image. Then, it tries the search again with the newly captured image.
  • Recapture: Captures an image using the selected image-capture bounds and replaces the current image with this new image.
  • Capture Image: Captures an image using the selected image-capture bounds and brings up an image capture panel so you can name the image and adjust any other settings as desired before saving.
  • Show Update Image Panel: This brings up the Update Image panel if it is not currently displayed.
  • Hide Highlight Rectangles: This hides the highlighted rectangles displayed after a search.
Tip: The arrow to the right of the capture area that reveals this drop-down menu is disabled when you hold down the Command or Control key to move the hot spot relative to the capture area. This behavior enables more precise placement of the hot spot, especially when using captured images to create search rectangles, as might be done with an OCR search.

Using the Auto Update Panel

The Auto Update panel displays at the end of every test run during which one or more image or text searches failed initially but were successfully found through the Auto Update functionality. When an image or text is not found, Auto Update makes remedies to that image or text search's settings during the test run in order to find a match, but does not save those changes to the image settings or script for future use. The Auto Update panel allows you to review each of the remedied search instances in turn.If you want to save the remedies used by Auto Update for image searches, you can choose to save the changes on the panel. For remedied text searches, you must modify your script with the remedy used in order to save the change. This feature will not engage unless Auto Update (Log Changes) is selected in the Eggplant Functional menu under Run > Image Update > Auto Update (Log Changes).

The Auto Update Panel, displaying an instance of a modified image search following a completed test run.

Note: The top bar of the Auto Update panel displays the name of the script and the date of the script run. This is helpful to know, as multiple Auto Update windows can be open at the same time.

Step by Step: Viewing and Saving Remedied Search Instances

The Auto Update panel presents all remedied search instances from the recently completed test run. For each of these instances, the image or text not found will be displayed in the upper left corner along with the settings Auto Update used during the script run to find that text or image. A screenshot of the SUT (System Under Test) at the time of search failure occupies the lower portion of the panel. This screenshot also shows the location of the found image or text, and the original location, if available.

The purpose of the Auto Update panel is to allow you to review and apply the remedied search settings for each image or OCR search. Changes to images can be applied through the panel; for OCR search updates, you must edit your script directly. To view and save the remedied search instances, follow these steps:

  1. Review the image or text, related screenshot, and settings adjusted by Auto Update. Determine whether or not the changed settings are ones you want to apply permanently.
    The provided SUT screenshot taken at the time of initial search failure shows where the image was found with the adjusted search settings. This location is highlighted with a solid-outline rectangle. The original image location will be highlighted with a dashed rectangle. To change the color of the background behind either the screenshot or the captured image as well as the color of the highlight rectangles for better contrast, click the image.
    For information regarding the various settings that may be modified and what they mean, see Using the Image Update Panel: The Start Tab.
  2. Click the Apply button if you want to apply the updated search settings of the current remedied search instance to your image. This action cannot be undone.
    Important: Applying these changes will modify the image search settings and save them permanently. This will affect any script in which that image is used, so take that into consideration when applying changes.

    Remedied text search instances - those using Optical Character Recognition (OCR) - also cannot be applied from this panel. To make the search setting changes applied at runtime permanent, you must edit your script to incorporate the changes suggested by the panel. The name of the script that this search was a part of is shown in the top bar of the panel, as well as the run time, which are useful when locating the OCR search to be modified.
  3. Use the arrows in the upper right corner of the panel to navigate back and forth between remedied search instances to review and apply changes. Apply as many or as few of the suggested changes as desired.
  4. Close the panel when you are done applying changes. This panel is accessible at any time through the Results pane of the Suite Editor under the Remedy column.

Checking the Frequency of Auto Update Engagement

Because Auto Update engages when your script encounters image and text search failures, it is good to track how often your scripts are engaging the Auto Update functionality. To see how often this is happening, do the following:

  1. Click the script in question in the Results list on the left side of the Suite window. This action displays a Results pane in the main body of the suite window, showing individual test records for each execution of that script.
  2. Select a test run from the script run list to analyze.
  3. Review the results in the bottom of the Results pane, looking for the warning messages logged by AutoUpdate during the test run. By searching for these AutoUpdate occurrences, you can see how many times auto update engaged during the test run and how long it took to resolve each case. For example, in the following screenshot, the AutoUpdate warnings are highlighted:

    Auto update being engaged

    You can also look in the Remedy column of the script run list to see if the Auto Update engaged and was able to find a match, resulting in an Auto Update panel appearing after the completion of that test run. Open this panel to view all remedied search instances during that test, and apply any of the changes as desired.


This topic was last updated on October 15, 2020, at 03:15:41 PM.

Eggplant icon | Documentation Home | User Forums | Support | Copyright © 2020 Eggplant