Working with Test Configurations

A test configuration is a combination of test cases and exploratory tests that can be run at any time or be scheduled to run on a regular basis in Eggplant DAI. A test configuration is essentially a storyboard of tests that can be run one after the other against a system under test (SUT) in a specific testing sequence. You create, edit, and view your defined test configurations in the Test Config tab.

Viewing Test Configurations

The Test Config tab displays the following columns:

Name: The name of the test configuration.

Model/Suite: The name of the model or suite depending on the type of test configuration you've chosen to run.

Description: A description of the test configuration. Provide a meaningful description to help distinguish it from other test configurations.

Tags: The tags defined for the selected test configuration that you're viewing.

SUT: The associated SUT, or criteria for choosing a SUT for the test configuration.

Updated: The date and time when the test configuration was last updated.

Steps: The number of test steps included in the test configuration. A test step is either a test case from the model, or an exploratory test with stop criteria, which could be defined by duration, iterations, or target coverage.

Click the Actions menu to Edit or Delete a test configuration.

To filter test configurations, type part of a test configuration name in the Search field or if you know the name of the model, select it from the All Models drop-down list.

Creating Test Configurations

Use the Test Configuration tab to create tests that can be run on a regular basis. You must specify at least one SUT against which the test will be run and associate it with a model.

Note: For exploratory steps, a stop criterion is always set by duration, iterations, or target coverage. After the test runs out of steps, Eggplant DAI will examine the execution of each step and indicate whether the test configuration is a success or failure.

Step by Step: Creating a Test Configuration

Follow these steps to add a test configuration:

  1. In the Test Config tab, click Add New. The New Test Configuration pane opens.
  2. Select a Model, then enter the Name and Description for the test configuration as you want them to appear in the Test Config tab.
  3. Add Tags by entering a name for the test configuration tag and pressing Enter. You can add as many test configuration tags as you need.
  4. Specify the SUT selection for the test to run on by selecting the appropriate options:

      • SUT by Name to select a predefined SUT.
      • SUT to use based on criteria to define one or more SUT criteria in the drop-down list. All the SUTs that are defined on the SUTs tab are available in this drop-down list. The SUT criteria field is similar to Test configuration tags, i.e., if you enter a SUT criteria and press Enter, relevant values are populated.

      For Cloud users, the options are slightly different:

      New Feature: The Cloud based SUT option is available from Eggplant DAI 5.0.0 for users of DAI Eggplant Cloud (previously called Hosted Eggplant DAI) only.

      • Execution Environment: Select from the options below to determine whether you want to use DAI Eggplant Cloud Execution Environment including the Agent and EPF with Cloud SUTs or your local SUTs defined in the SUTs tab. Select Cloud to execute tests on Cloud SUTs. For using the Cloud SUT in Design mode, see Connecting to DAI Cloud SUTs.

      • Local: Lets you select a predefined SUT or define a SUT based on criteria. To define a SUT based on criteria, enter a SUT criteria and press Enter.
      • Cloud: Enables the Eggplant Cloud Hosted SUT option:
        • Eggplant Cloud Hosted SUT (run on Chrome or Firefox): Select this to execute tests on Cloud SUTs where Eggplant will provision the Cloud Execution Environment (including the Agent and Eggplant Functional) for you. For more information, see Connecting to DAI Cloud SUTs.

      To execute tests using Cloud Execution Environment and your own local SUTs, you must ensure that your firewall rules allow connections from the Cloud Execution Environment. All connections from the Cloud Execution Environment to the SUTs will come from one of these fixed IP addresses:

      • 54.195.73.56
      • 54.170.198.89
      • 54.170.92.66

  5. Select the Type of test configuration you want to run. The selection you make here will determine the steps to add to your test configuration.

    • Script Based: Enables the option to select a suite and then add script based steps. Note that scripts are only be available if you have at least associated one suite in your Model tab. For detailed steps, see Using Script Based Test Configuration.
    • Model Based: Enables the option to select a model in order to run model based test configuration.
  6. For Model Based test configuration, select a model, then add an exploratory step or directed test cases, or both.
    • To add an exploratory step, select Add Exploratory and specify a target for the exploratory test steps by selecting one of the following options:
      • Iterations: Determines the number of times that the exploratory test should execute before moving on to next step in the test configuration. The test configuration will end if there is no other step left in the list of tests. By default, the iteration value is set to 10.
      • Coverage: Determines the minimum coverage required to complete a step. In Eggplant DAI, coverage represents the percentage of states and actions visited in your model during a series of exploratory test runs. Note: A test will pass even when the coverage doesn’t meet the minimum target coverage. For e.g., the test will stop when the target coverage is not achieved within a reasonable time, or until the point at which coverage stops increasing.
      • Time: Select to specify the duration of the exploratory step in hours.

    • To add specific directed test cases to your test configuration, select from the list. Note that test cases will only appear if you have created at least one directed test case in the Test Cases tab in Modeler.

  7. Click Save. The new test configuration is saved to the list of test configurations in the Test Config tab.

Tips:

  • Reorder your steps by clicking the Drag icon, then drag and drop steps as required.
  • Delete your steps by clicking the Delete icon.

Running a Test Configuration

You can run a test configuration from the Runner tab by clicking against the test you want to run. Using the Runner tab in Eggplant DAI, you can configure test schedules with specific frequencies to be run at regular intervals. Note that you can only have one schedule per test configuration.

Tip: Remember to always associate suites to a model before running tests.

Viewing and Editing an Existing Test Configuration

You can view and edit your defined tests from the Test Config tab by selecting a test configuration from the list and clicking Edit from the Actions menu.

Removing a Test Configuration

To remove an existing test configuration, in the Test Config tab select it and click Delete from the Actions menu .

 

This topic was last updated on August 19, 2021, at 03:30:47 PM.

Eggplant icon Eggplantsoftware.com | Documentation Home | User Forums | Support | Copyright © 2022 Eggplant