Working with Test Configurations
A test configuration is a collection of tests that can be run on demand or be scheduled to run on a regular basis in Eggplant DAI. A test configuration ("test config") is essentially a blueprint of tests that can be run one after the other. You can run directed test cases and model-based exploratory tests in a test config. See About Test Cases for more information about test cases.
Go to Controller > Test Config, to create, edit, or view your defined test configs.

Test Config page
The following columns are displayed to provide at-a-glance details about each test configuration:
- Name: Display name of the test configuration.
- Model/Suite: Name of the model or suite depending on the type of test configuration you choose to run.
- Description: Description of the test configuration. Provide a meaningful description to help distinguish it from other test configurations.
- Tags: Tags defined for the selected test configuration that you're viewing.
- SUT: Associated SUT, or criteria for choosing a SUT for the test configuration.
- Updated: Date and time when the test configuration was last updated.
- Steps: Number of test steps included in the test configuration. A test step is either a test case from the model or a script based test case, or an exploratory test with stop criteria, which could be defined by duration, iterations, or a target coverage percentage.
Click the Actions menu to Edit, Delete, or Duplicate a test configuration.
Filtering Test Configurations
You can filter test configurations in the following ways:
- By name: Type part of a test configuration name in the Search by name search bar. DAI filters the test configurations you see as you type.
- By model: If you want to see the test configurations associated with a model, click the arrow on the Models button and select a model from the suggested list or start typing a model name in the Search... field. DAI suggests models as you type.
- By suite: If you want to see the list of test configurations associated with a suite, click the arrow on the Suites button and and select a suite from the suggested list or start typing a suite name in the Search... field. DAI suggests suites as you type.
Creating Test Configurations
Go to Controller > Test Config to create a test configuration ("test config").
Step by Step: Creating a Test Configuration
Creating a test config consists of three steps, with a page for each step. The steps and pages are summarized below, followed by detailed instructions:
- Step 1: General: the tab where you provide the general information about your test config, including a name, description, test type, test config tags, execution settings, and SUT.
- Step 2: Steps/actions: the tab where you configure the exploratory test or test case actions you want your test config to run.
- Step 3: Advanced Settings: the tab where you can configure if you want to automatically rerun failed test steps, stop a run when a step fails, enable a maximum run time, assign a custom script, enable a suite synchronization, and set up email notifications to inform designated recipients of the completed test configuration runs.
A progress tracker at the top of the page shows where you are in the creation process as shown in the image below. The highlighted number in purple shows the step you are on in the process. Once you finish entering information on a page and advance to the next one, the progress tracker shows a green check for completed the step.
The pages include Cancel and Back buttons so you can cancel the test config creation or return to a previous page when you want.

Create new Test Config, Step 1: General Basic settings
Providing a name and test type enables additional settings.
Step 1: Configure the General information for the Test Configuration
-
Go to Controller > Test Config. The Test Configs page opens.
-
On the right side of the top menu, click + New Test Config. The Create new Test Config page opens with the first step in the process, the General page shown above, where you provide the Basic Settings for your test config.
-
Provide a Name and Description for the test configuration. This is the name and description that show in the list of test configs. Also note the Clear all button you can use to clear the Description field.
-
Select a Test type (Model based or Script based) for the test configuration. After you specify the test type, more settings become visible. Some of the options you see depend on the test type selected. For both test types, you see the Test configuration tags and Execution settings selectors.
-
If you selected the Model based test type, you see the Model selector.
Basic Settings Model Selector
-
If you selected the Script based test type, you see the Suites selector so you can select the suite that contains the scripts/snippets you want your test config to run.
Basic Settings Suites Selector
-
-
Select the model or suite from the selector as appropriate for your test type.
ノートNote that selecting your model or suite enables the Next button at the bottom of the page. You can proceed to the Steps/actions page at this point with the default values for the remaining settings on this page.
-
Add or select Test configuration tags by entering a name for the tag and pressing Enter. Tags provide another way for you to identify and filter by your test config. You can add as many test tags as you need. You can also clear the list of selected tags by pressing the Clear all button.
Basic Settings Test configuration tag Selector
-
In the Execution settings section of the page, select the type of execution environment (Local or Cloud) that you want your test config to use.
-
Select a SUT option for this test config that is appropriate for your execution environment as follows:
For Local execution environments, choose from the following SUTs:
-
None (no SUT) if the SUT is defined in your code or not required.
-
SUT by name to select a predefined SUT.
Basic Settings Select Local SUT by name
-
SUT by criteria tag to select a SUT tag from the drop-down list. The SUT tag is associated with a SUT that is available on the SUTs page (Controller > SUTs). If you want to create a SUT or SUT tag, navigate to the SUTs page.
Basic Settings Select Local SUT by tag
For Cloud execution environments, choose from the following SUTs:
-
The same options listed above for Local execution environments.
-
Eggplant hosted cloud: Select this to execute tests on Cloud SUTs where Eggplant will provision the Cloud Execution Environment (including the Agent and Eggplant Functional) for you. For more information, see Connecting to DAI Cloud SUTs. (This option is only available for cloud SUTs.)
-
-
When you finish configuring the Basic Settings, click Next. You advance to the second part of the test configuration creation process: the Step 2: Test steps/actions page.
Step 2: Configure the Steps/actions for the Test Configuration
The options you see on the Steps/actions page depend on the test type you selected. The top of the page shows the test type you selected (Model based or Script based) and either the model or suite as appropriate for the test type. Follow the appropriate steps below for your test type:
- Configuring steps/actions for Model based test types
- Configuring steps/actions for Script based test types
Configuring steps/actions for Model based test types
For Model based test types, you see the Available exploratory and test cases list.

Create new Test Config Step 2: Test steps/actions test case selector
-
Click to select any of the test cases from the Available exploratory and test cases list. If the Available exploratory and test cases list is too long, you can filter it by typing test case names. What you see added to the Steps added list on the right side of the page depends on the type of test you select:
Test steps/actions filtered test case steps
-
For exploratory tests, you see exploratory test options where you can specify the number of Iterations, Coverage, and Time (hrs) as shown below.
Test steps/actions Exploratory test step
ノートFor exploratory steps, a stop criterion is always set by duration, iterations, or target coverage. After the test runs out of steps, Eggplant DAI will examine the execution of each step and indicate whether the test configuration is a success or failure.
-
For test cases, you see the test case.
Test steps/actions test case steps
-
For test cases with a test case tag assigned, you see the tag under the test case name.
-
For test cases set to be ignored in a test run, you see an ignored tag at the end of the row.
-
There is a View settings button available to allow you to hide and show Test case tags and Parameters.
-
-
Repeat step 1 above until you have added all the tests you want your test config to run or click Add All to add all the test cases in Available exploratory and test cases list. Note that clicking Add All when filters are applied to the Available exploratory and test cases adds only the test cases in the filtered list.
ノートClicking Add all while the Available exploratory and test cases list is filtered only adds test cases in the filtered list.
- While you are editing a test case, if any step is added, removed, or re-arranged, you will see a confirmation message when you click Save to ensure you are aware that the changes will affect the latest status of the step in the test config, resetting it to Untested.
-
Click Next to advance to Step 3: Advanced Settings page and follow the instructions for that page.
Configuring steps/actions for Script based test types
For Script based test types, you see the Available test cases and features selector that contains two tabs. The Test case tab lists the available scripted test cases and the Features tab lists the available Gherkin feature files included in the suite used to create the script-based test config. Each list has a Search... bar so you can search for test cases or features by their names.

Test steps/actions script-based test case selector
-
Select a scripted test case or feature from the list or search for it by name in the search bar and then select it. You see the test case or feature added to the Steps added list on the right.
-
Repeat step 1 above until you have added all of the steps you want your test config to run.
-
Click Next to advance to the Step 3: Advanced settings page and continue to the following section for information about configuring it.
We recommend omitting the BeginTestCase and EndTestCase commands in your Gherkin test scripts because they can interfere with the Gherkin results obtained.
Global Parameters
Global parameters are parameters that are shared across all test cases within a script-based test config. Once they are defined, you can use them as your custom parameter values. Global parameters are only available in script-based test configs.
To Add global parameter of a script-based tests config:
-
Go to Steps / actions tab.
-
Click the Global parameters button to open global parameters' panel.
-
Once global parameters' panel is visible, click the Add global parameter button.
-
Enter a unique Global parameter name. You cannot specify a global parameter name that already exists in the Global parameters' list.
-
Enter a Default value to override during the test run.
-
Click the Add button to add the new global parameter to the global parameters' list.
To remove the added global parameter from the global parameters' list, click the trash button.
Once you add a global parameter, you cannot edit its name. However, you can still change the default value directly from the global parameters' list.
To set a global parameter as a custom parameter value on the Edit parameters' panel:
-
Click the Edit parameters button on a test case in the Steps added list to open the Edit parameters panel.
-
Click the Global parameters chip on the Custom parameter tab to copy the global parameter name.
-
Paste the copied value into the Custom parameter value's input box.
Clicking chips to copy parameters might not work as expected on an insecure website (for example, a website with a URL that begins with http instead of https).
If you want to manually enter the global parameter as a custom parameter value, you must enter in it in the format of [[global_parameter_name]].
Remember to click Save before you leave the Edit parameters and Global parameters panels. If you do not save your changes, the newly configured settings will not take effect when you are configuring the test config.
Step 3: Configure Advanced Settings for the Test Configuration
When you reach the Advanced settings page, you have the following options:
-
If you want to create your test config, click the Create test config button (not shown in the following sample page). See Finish Creating a Test Config for more information.
Create new Test Config Step 3: Advanced Settings
-
If you want to enable automatic reruns of failed or errored test steps for your test config, click the corresponding slider next to Automatically rerun failed test steps or Automatically rerun errored test steps. See Enabling Reruns for an explanation of how to configure DAI to rerun your test config.
-
If you want to stop your test config run when a step fails, disable Run all test config steps to completion by clicking the slider next to it. See Stop On Failure for an explanation of how to configure DAI to run all test config steps to completion or stop when a step fails.
-
If you want to enable a maximum run timeout for a test config, you can choose from one of 3 options (System default, Unlimited, or Custom) to set the timeout period. See Timeout Management for an explanation of how to configure the timeout period for your test config.
-
If you want to send email or webhook notifications to inform designated recipients of the completed test configuration runs, click + Add notification to create a notification. See Notifications (Webhook or Email) below for information about how to create the different types of notifications.
The No notifications panel
-
If you want to set a custom script to run when executing this test config, select the desired custom script from the dropdown list. If a default custom script is already set, you do not need to make any changes unless you wish to override it.
Custom script settings
A message below the dropdown indicates the currently set default custom script. This message will disappear once you make a selection.
This custom script setting is only editable for DAI Administrator users.
-
If you want to always synchronise the latest suites from your script repository (DAI Server or Git Connections), make sure the option for Suite Synchronisation is enabled.
Suite synchronisation settings
This suite synchronisation setting is only editable for DAI Administrator users. Only disable it if you are sure you don't want to sync your suites, or you are using an alternative sync method such as custom scripts.
See Finish Creating a Test Config below to finish creating your test config.
Notifications (Webhook or Email)
DAI supports two types of notifications for test configurations: email or webhook. See the Add an Email Notification or Add a Webhook Notification section below for information about how to use them.
Notifications can be added to or updated in a test configuration at any time after they are enabled globally.
Test Configuration notifications must be enabled globally before you can set up notifications for a specific test configuration within DAI. You need to be an administrator to do this. See Manage DAI Settings for details.
Add an Email Notification
If you want to enable email notifications to report the completed runs for a test config, click the test config's Advanced settings tab to display it. If there are not any existing notifications, you see the No notifications panel shown above.
-
Click Add notification to add your notification settings. The New notification panel slides out from the right, where you can select the notification type.
The New notification panel
-
Select Email as your preferred notification type. The settings for email notifications display on the panel.
Email notification settings
-
Select the users you want to receive notifications in the Recipients box.
- Rules for Recipients:
- If notifications are enabled, you must pick at least one recipient before you are allowed to save.
- If notifications are disabled, then you can leave the field blank and save.
- Rules for Recipients:
-
Select the event(s) or status(es) that trigger the notifications.
- Rules for Event:
- If notifications are enabled with Enable this notification, notfications for all selected events are sent to the recipients. If you deselect all the event checkboxes, notifications are disabled and Enable this notification is to disabled.
- If notifications are disabled with Enable this notification and all the event checkboxes are deselected, enabling Enable this notification enables notifications, and all checkboxes will be enabled by default.
- Rules for Event:
-
Enable the notification by sliding the Enable this notification slider to the right. (Disable it by sliding to the left.)
-
Click Add to include notifications in the test configuration.
A configured email notification
-
Click Save to save the test configuration along with the email notification.
Add a Webhook Notification
Before you can add a webhook notification, you must create a webhook profile. See the Webhook Notifications page for more information.
-
Click + Add notification. The New notification panel will slide out from the right where you can select the notification type.
Add a new notification
-
Select Webhook as your preferred notification type. The panel displays the webhook options. From here, you can select a webhook profile, the triggering event (status), and enable or disable the webhook notification.
Select webhook notification
-
Select a webhook profile from the drop-down list. You must select a profile before you are allowed to save a webhook-type notification. Rule for Webhook profiles:
- You need to select a Webhook before you are allowed to save.
-
Select the events (or test status) you want to trigger the notification (Pass, Fail, Error, or Cancelled).
Rules for Event:- If notifications are enabled with Enable this notification, and then you deselect all event checkboxes, notifications (and Enable this notification) will be disabled.
- If notifications are disabled with Enable this notification and all the event checkboxes are deselected, enabling Enable this notification enables notifications, and all checkboxes will be enabled by default.
-
Enable the notification by sliding the Enable this notification slider to the right. (Disable it by sliding to the left.)
-
Click Add notification to include the notification in the test configuration.
A configured webhook notification
-
Click Save to save the test configuration along with the webhook notification.
Finish Creating a Test Config
When you finish creating your test config, click the Create test config button in the lower-right corner. DAI saves your new test config and the Test Configs page opens. The new test config will be in the list but you may need to scroll or search for it to see it.
Viewing and Editing the Notification Settings of a Test Configuration
To view or edit the notification settings in a test configuration (test config):
-
Go to Controller > Test Config.
-
Find the test config you want to update or view, select it, and then, from the
Actions menu, select Edit. From here you can view or update the test config details. For more information about the settings for the different notification types (email or webhook), see Notifications above. You can also manage the notification settings of a test config from the list under Notifications on the test config's Advanced settings tab by clicking the Actions menu and selecting Edit.
Manage notifications
Configure Custom Parameters for Steps Added
The Custom Parameters feature allows you to overwrite the default value of a test case's parameter with a Custom value. DAI then uses your Custom value when a test config runs the test case. You can customize as many of your test case parameters as you want.
To configure a custom test case parameter value for any test step:
-
Click the
icon on the step. A slide panel that displays all test case parameters will open from the right.
-
Add a value in the Custom value column to overwrite the default value of the test case parameter.
-
Click Save.
This configuration allows you to diversify the same test case with different parameter values without replicating the test case with different default values.
Enabling Reruns
The rerun feature allows you to save time and effort by automatically rerunning failed or/and errored test steps up to 50 times without executing the entire test configuration. A test step can be a model based test case, a script based test case, or an exploratory test.

Advanced Settings Automated Reruns enabled
To activate the rerun functionality for failed or errored test steps:
-
Enable reruns by sliding the Automatically rerun failed test steps or/and Automatically rerun errored test steps toggle to the right. (A green check indicates the option is enabled.)
-
Configure the rerun parameters with the following controls:
- Number of Runs: Specify the maximum rerun count for failed and errored test steps. You can adjust this value using the text box or slider, with a maximum of 50 reruns.
- Wait Duration: Define the interval between each rerun, optimizing the rerun process. Choose a time interval from the dropdown list, measured in seconds. The default is set to zero, implying no wait interval.
This streamlined process allows you to effectively manage reruns, enhancing your testing workflow.
- The rerun count and wait duration are the shared settings between automatically rerun failed and errored test steps.
- The chosen rerun count and wait duration will be saved even if you deactivate the rerun feature. This way, you can easily re-enable it at a later time without reconfiguring.
- Access rerun statistics by viewing the total reruns completed under the Controller menu on the Runner page. Expand individual test results to find this information.

Rerun stats