A test configuration is a combination of test cases and exploratory tests that can be run at any time or be scheduled to run on a regular basis in Eggplant DAI. A test configuration is essentially a storyboard of tests that can be run one after the other against a system under test (SUT) in a specific testing sequence. You create, edit, and view your defined test configurations in the Test Config tab.
The Test Config tab displays the following columns:
Name: The name of the test configuration.
Description: A description of the test configuration. Provide a meaningful description to help distinguish it from other test configurations.
Tags: The tags defined for the selected test configuration that you're viewing.
SUT: The associated SUT, or criteria for choosing a SUT for the test configuration.
Updated: The date and time when the test configuration was last updated.
Steps: The number of test steps included in the test configuration. A test step is either a test case from the model, or an exploratory test with stop criteria, which could be defined by duration, iterations, or target coverage.
Click the Actions menu to Edit or Delete a test configuration.
To filter test configurations, type part of a test configuration name in the Search field as well as a model name in the All Models field and press Enter.
Creating Test Configurations
Use the Test Configuration tab to create tests that can be run on a regular basis. You must specify at least one SUT against which the test will be run and associate it with a model.
Note: For exploratory steps, a stop criterion is always set by duration, iterations, or target coverage. After the test runs out of steps, Eggplant DAI will examine the execution of each step and indicate whether the test configuration is a success or failure.
Follow these steps to add a test configuration:
- In the Test Config tab, click Add New. The New Test Configuration pane opens.
- Select a Model, then enter the Name and Description for the test configuration as you want them to appear in the Test Config tab.
- Add Tags by entering a name for the test configuration tag and pressing Enter. You can add as many tags as you need.
- Specify SUT selection for the test to run on by selecting a SUT by Name from the drop-down list or a SUT to use by defining one or more SUT criteria in the drop-down list. All the SUTs that are defined on the SUTs tab are available in the drop-down list. The SUT criteria field is similar to Tags, i.e., if you enter a criteria and press Enter, relevant values are populated.
- Select a test case type to add to your test configuration. The options are:
- Exploratory: Select to add an exploratory test.
- Test Case: Select to add a specific directed test case from the list. Note that test cases will only appear if you have created directed test cases in Eggplant AI UI.
- Repeat this step for adding more directed tests to the selected test configuration.
- To add an exploratory test step to your test configuration, click Exploratory+. You can specify a target for the exploratory test steps by selecting one of the following options:
- Iterations: Determines the number of times that the exploratory test should execute before moving on to next step in the test configuration. The test configuration will end if there is no other step left in the list of tests. By default, the iteration value is set to 10.
- Coverage: Determines the minimum coverage required to complete a step. In Eggplant DAI, coverage represents the percentage of states and actions visited in your model during a series of exploratory test runs. Note: A test will pass even when the coverage doesn’t meet the minimum target coverage. For e.g., the test will stop when the target coverage is not achieved within a reasonable time, or until the point at which coverage stops increasing.
- Time: Select to specify the duration of the exploratory step in hours.
Running a Test Configuration
You can run a test configuration from the Runner tab by clicking against the test you want to run. Using the Runner tab in Eggplant DAI, you can configure test schedules with specific frequencies to be run at regular intervals. Note that you can only have one schedule per test configuration.
Viewing and Editing an Existing Test Configuration
You can view and edit your defined tests from the Test Config tab by selecting a test configuration from the list and clicking Edit from the Actions menu.
Removing a Test Configuration
To remove an existing test configuration, in the Test Config tab select it and click to Delete from the Actions menu.