Organizing Your Testing

The following planning tips can help you use Eggplant Functional more effectively.

Have a Plan

When you start developing a test script, you should already be familiar with the application you are testing, and know exactly what you intend to test. Simply put, you should have a test plan.

If you have conducted manual tests of the software in the past, you might have developed manual test scripts—written sequences of steps that a person can perform to verify that particular parts of the system are functioning correctly. Because Eggplant Functional interacts with software just like a person does, these manual test scripts are an ideal starting point for developing automated test scripts.

If you do not have any formal tests already, you might be wondering where to start. Some people choose to model the entire application, and map out a strategy to systematically exercise every part of it. Others choose to focus on areas of a system that present the highest risk, defining the highest risk as the greatest potential cost of failure or the most likely to contain bugs.

Whatever approach you take, the most productive method is to start small, get some simple tests working, and then build larger and more ambitious tests as you progress. Remember that a small amount of automation is already better than none; and every small, manageable step brings you closer to your ultimate goal.

Think in Terms of End-User Functionality

Eggplant Functional interacts with the system under test (SUT) like a "virtual user", making it ideal for testing user-level functionality. Think in terms of the high-level tasks that a user performs with your software: What are the major functions that must be working for end users to get value from your product? How can you test these functions? Depending on how much time you have, go down a level and repeat this process.

Kent Beck presents this concept of testing very well in his book Extreme Programming Explained (Addison Wesley Longman, 2000). Many people want to relate testing coverage to a percentage of the application. Beck makes the point that you should not set goals that you cannot measure. When people say they test 80% of their application, what exactly are they counting? Would 100% testing mean that all valid and invalid data cases were tested, or that every permutation of system options and application options was tested? A more meaningful goal is to test all of the application’s essential functions.

Be Aware of the Visible Interface

Automated testing with Eggplant Functional requires that you be keenly aware of how a real user would go through the steps that you want to test. You always have to keep in mind what the user would see and how the user would interact with what is actually visible on screen.

For example, suppose you are testing these actions:

1. Double-click an icon to launch a web browser.
2. Type a URL in the address field.
3. Press Return to load the page.

If these actions are executed too quickly, the browser might not be open when the text is typed. A human user would know to wait for the browser to open before typing a URL, so your script needs to know it, too. A WaitFor "Address field" command would do the trick.

Establish a Consistent Interface

Most operating systems let you customize the interface, at least to some extent. This ability is very user-friendly, but not so friendly to image-based testing. For example, on Mac, you can select the color that is used to indicate the currently chosen menu item. If you capture images of the menu items highlighted blue and then change the highlight color to gray, Eggplant Functional continues to look for blue and probably fails.

It is best if your SUTs are computers that you control so that they are not customized by other users between test cycles. If this is not the case, try to establish standard visual settings for all of the SUTs you use. You might even create scripts that check your settings and change back any that are different from the standards.

note

For most operating systems, the default settings are the best to adopt. This can make new SUTs ready to use right out of the box; and even SUTs that have been around the block are likely to use many of the default settings.

For Mac, find the following settings in System Preferences > General:

• Appearance: Blue or graphite, (in Appearance)
• Use LCS font smoothing when available (in Appearance)
• Show scrollbars (in Appearance)
• Color (In Displays)
• Solid desktop color (in Desktop & Screen Saver)

For Microsoft Windows systems, find the following settings in the Display Properties control panel:

• "Use the following method to smooth edges of screen fonts": Off
• All settings in the Taskbar and Start Menu Properties control panel
• All settings in the Folder Options control panel.

Regardless of the operating system you are testing, standardize the color depth at which all of your SUTs of that type run. Some older computers cannot display 32-bit color ("millions of colors") and even if they are able to display 24-bit color, the colors that they send through VNC can be very different from those displayed on a 32-bit system.

Some of these color differences might fall within your search type tolerances, but others can cause a script that was generated on a 32-bit system to fail on a 24-bit system, and vice versa. If you have SUTs with different maximum color depths, you need to decide whether you are going to use the lower color depth for all of them, or capture an additional set of images if they are not matching up.

Finally, keep in mind that solid desktop backgrounds are easier to use than pictures or patterns. If you capture an image with part of a picture in the background, Eggplant Functional might not be able to recognize it at another place on the desktop.

note

Note: Screen size and resolution do not affect image matches.

Record the System State

One of the benefits of testing on multiple SUTs is the ability to observe how an application responds to different system set-ups. Here are some considerations:

• What operating system is running on the SUT?
• How much memory does it have?
• Which System preferences differ between SUTs?

By documenting these and similar data, you can address each configuration with tests to validate its impact on the application.

Organizing Your Images

This article offers practical advice for organizing images and reducing your own image search time.

Image Collections

An Image Collection is a folder of related images that are all acceptable versions of a basic image. For example, an Image Collection might contain a "normal" image of a button, a selected version, and a disabled version. It helps to have a theme, such as separate Image Collections for each of your applications windows or panels.

Any image folder in your suite can be used as an Image Collection, but keep in mind that Eggplant Functional looks for every image in the Image Collection, so a very large folder could slow down your script execution and cause Eggplant Functional to find matches you are not interested in.

Naming Conventions

Naming conventions make it a quite a bit easier to identify your images, and much easier to identify your co-workers’ images. The table below shows one QA team’s image-naming conventions; it might be a helpful starting point for your own guidelines.

Image attribute	Denoted by...
Highlighted	_hending
Disabled	_d ending
Normal	No ending
Menu	m_ prefix
Menu item	mi_ prefix
Checkbox	cb_prefix
Radio button	rb_prefix
List box	lb_ prefix
Text box	tb_ prefix
Group box	gb_ prefix
Slider	sl_ prefix
Button	bt_ prefix
Pull-down menu	pd_ prefix
Selected pull-down menu item	pds_prefix
Tab menu image	tm_ prefix
Dialog title	t_ prefix
Screen capture	sc_ image

Use a Test Management Tool

Test Management tools help your team to stay organized and provide a single point of control for your tests. This is critical to keep in mind as your testing infrastructure grows and you automate more of your tests.

Eggplant Manager

Eggplant provides a test management tool called Eggplant Manager that allows you to construct tests from a variety of existing Eggplant Functional tests, schedule the execution of those tests, and then collect and analyze results. This is the easiest and best way to execute multiple Eggplant Functional tests or instances of a test simultaneously.

To read more about Eggplant Manager, see the Eggplant Manager Documentation.

Eggplant Integrations

Eggplant also provides the ability to integrate into any Continuous Integration (CI) software or Application Lifecycle Management (ALM) tool with Eggplant Integrations. For specifics on these integrations, see the Eggplant Integrations Documentation.

Keep Your SUT Awake

There is nothing more frustrating than being deep into your testing when suddenly, due to inactivity on the SUT screen, your screen saver comes on causing your test to fail. Luckily, there are ways to keep this from happening.

One option you could use to solve this issue, which will work on a variety of SUTs, is to handle it in your test script. For example, if you have long wait or waitfor commands in your SenseTalk script that could cause inactivity on your SUT, you could add a repeat clause, like the one shown below, to handle those periods of inactivity and keep the SUT awake:

repeat until clause
   moveto [1,1]
   wait 10
   moveto remotescreensize()
   wait 10
end repeat

Mac and Windows SUTs Desktop SUTS

To solve the SUT going to sleep issue on macOS and Windows desktop systems, you may be able to modify settings on your operating system to handle this issue.

For Windows: In the Power & Sleep settings, you can set the Screen and Sleep battery and energy ("plugged in") settings to Never.

For macOS: In the Lock Screen settings, you can set the Turn display off on battery when inactive and Turn display off on power adapter when inactive settings to Never.

Mobile SUTs

Some mobile devices have a setting you can choose so that the device doesn't go to sleep when the user isn't using it. The procedure for setting this option is different depending on the device; follow the appropriate steps below.

iOS devices

1. Go to Settings on the device.
2. Tap Display & Brightness.
3. Tap Auto-Lock.
4. Tap Never.

For Android

Some Android devices allow the display to never turn off, while others do not. This is a device-specific option. If your Android SUT does not have this option, you can use the SenseTalk PressHomeButton command to wake the screen up.