Version: DAI 25.2

Running Automated Tests

Adding test automation to your build pipelines supports continuous integration, and allows you to implement and test new features as early as possible.

In Eggplant DAI, you can trigger a set of tests, or test configuration, from the command line. To do this, you need to download, install, and run the DAI CLI tool or any of the provided pipeline plugins for Azure, Github, or Jenkins as described below.

Download the DAI CLI tool for your platform. The following platforms are supported:
- Windows 10/11/ and Server 2019/2022
- macOS: 11 and above
- Linux: RHEL 8, Ubuntu 16 and Ubuntu 22
Plugins for Azure, Github, and Jenkins are available in their respective market stores. You can find instructions for installing and configuring them at the links below.
Run your test config with the CLI tool or plugin executable file from the command line. See Run test config by Id or Run test config by Name for more information.

note

The shell in which you run the DAI CLI tool needs to support UTF-8 encoded characters. This is true if you run the tool at the command line or from a batch file.

The executable runs until the test config is complete.

If the test config runs successfully, it will complete with an exit code of 0.
Other exit codes indicate that the test config has failed.

Run Test Configuration by Test Config Id

When creating a test config, each test config will be assigned a unique identifier (TEST_CONFIG_ID).

To retrieve this ID, go to Controller > Test Config and edit the test config that you want to run. The TEST_CONFIG_ID value is appended to the URL.

.

Other mandatory options:

Parameter	Value Description
`SERVER_URL`	URL for the Eggplant DAI server. For example: `https://myeggplantserver.com`
`TEST_CONFIG_ID`	Test config ID. For example: `adcb4d6e-3b3e-4cc3-a62b-3bba84fe5ab0`. It is the string appended to the DAI URL when you open a test config. See image above.
`--client-id <CLIENT_ID>` `--client-secret <CLIENT_SECRET>`	Retrieve the client secret pair set up during installation. For security, you might want to set the `DAI_CLIENT_SECRET` environment variable instead of passing your credentials on the command line.

For the cli --help to see all parameters and options, run this command:

eggplant-runner-Windows-xx.x.xx.exe --help

At your command line, run the following command:

eggplant-runner-Windows-xx.x.xx.exe <SERVER_URL> <TEST_CONFIG_ID> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET>

For more configuration settings, see advanced options.

Run Test Configuration by Test Config Name

Alternatively, you can also run a test config by using the test config name configured in DAI server, along with its model name or suite name. You can find this information from Controller > Test Config.

Other mandatory options:

Parameter	Value Description
`SERVER_URL`	URL for the Eggplant DAI server. For example: `https://myeggplantserver.com`
`--client-id <CLIENT_ID>` `--client-secret <CLIENT_SECRET>`	Retrieve the client secret pair set up during installation. For security, you might want to set the `DAI_CLIENT_SECRET` environment variable instead of passing your credentials on the command line.

Model based Test Configuration

To run a model-based test config, retrieve the values for following options from your test config:

Parameter	Value Description
`--test-config-name <TEST_CONFIG_NAME>`	DAI test config name that you want to run. For example: `--test-config-name=TestConfigEggplantApp`
`--model-name <MODEL_NAME>`	DAI model name for the specified test config. For example: `--model-name=ModelEggplantApp`

At your command line, run the following command:

eggplant-runner-Windows-xx.x.xx.exe modelbased <SERVER_URL> --test-config-name=<TEST_CONFIG_NAME> --model-name=<MODEL_NAME> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET>

Script based Test Configuration

To run a script-based test config, retrieve the values for following options from your test config:

Parameter	Value Description
`--test-config-name <TEST_CONFIG_NAME>`	DAI test config name that you want to run. For example: `--test-config-name=TestConfigEggplantApp`
`--suite-name <SUITE_NAME>`	DAI suite name for the specified test config. Full suite name including .suite at the end. For example: `--suite-name=EggApplication.suite`

For the cli --help to see all parameters and options for script-based, run this command:

eggplant-runner-Windows-xx.x.xx.exe scriptbased --help

At your command line, run the following command:

eggplant-runner-Windows-xx.x.xx.exe scriptbased <SERVER_URL> --test-config-name=<TEST_CONFIG_NAME> --suite-name=<SUITE_NAME> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET>

For more configuration settings, see advanced options.

Advanced Options

Advanced options are optional but do allow you to fine-tune test executions. For an up-to-date list, run the --help option. See advanced examples, for CLI examples.

Option	Description
`--request-timeout INTEGER`	Timeout in seconds for each HTTP request to the DAI server. Default: `30`
`--request-retries INTEGER`	Number of times to attempt each HTTP request to the DAI server. Default: `5`
`--backoff-factor FLOAT`	Exponential backoff factor between each HTTP request. Default: `0.5`
`--log-level TEXT`	Logging level. Default: `INFO`
`--ca-cert-path PATH`	Path to an alternative Certificate Author pem file
`--test-result-path PATH`	Path to a file where the test results will be stored in junit xml format
`--version`	Shows the version number and exits
`--dry run / --no-dry-run`	Perform a dry run only. Default: `no-dry-run`
`--param TEXT`	The test config global parameter you want to override in the format `parameter_name=parameter_value`. For example: `--param username=Lily` or `--param "textEntry=\"Text entry with spaces and special symbols!\""`. Instead of passing individual parameters, we recommend that you put the parameters in a JSON file and specify its location using one parameter: the `--param-file PATH` described below.
`--param-file PATH`	The path to a user-created JSON file that contains the global parameters you want to override. The keys in the file are the names of the global parameters you want to override, and their values are the values you want to use instead. Both parameter names and values must be of type string.
`--filter-by TEXT`	The filters to run specific steps within a test config in the form of filter_name=filter_value. For example: `--filter-by step_status_include=Failed,Error` or `--filter-by "test_case_tag_exclude=\"tag with space,another tag\""`. The full list of filters available are provided below in Advanced Example. Instead of passing individual filters, we recommend that you put the filters in a JSON file and specify its location using one parameter: the `--filter-by-json PATH` described below.
`--filter-by-json PATH`	The path to a user-created JSON file that contains all the filters to run specific steps within a test config. An example of the JSON file format is given below in Advanced Example
`--previous-task-instance-id <TASK_INSTANCE_ID>`	The UUID of a previously run test config definition to be used in the current test config run. Defaults to the last run of the given test config.

note

--poll-interval INTEGER & --test-environment-timeout INTEGER are Deprecated in DAI 7.4. Please refer to Retries the connection for SUT and Execution Environments for the variable settings you should use instead.

Advanced Examples

The following DAI CLI examples show how you can use advanced options:

--log-level TRACE

eggplant-runner-Windows-xx.x.xx.exe scriptbased <SERVER_URL> --test-config-name=<TEST_CONFIG_NAME> --suite-name=<SUITE_NAME> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET> --log-level=TRACE

--test-result-path PATH

 eggplant-runner-Windows-xx.x.xx.exe scriptbased <SERVER_URL> --test-config-name=<TEST_CONFIG_NAME> --suite-name=<SUITE_NAME> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET> --test-result-path="c:\EggplantResults\ResultEggplantAppJunit.xml"

--param TEXT

eggplant-runner-Windows-xx.x.xx.exe scriptbased <SERVER_URL> --test-config-name=<TEST_CONFIG_NAME> --suite-name=<SUITE_NAME> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET> --param MyEnvironment=Production --param username=Eggy

--param-file PATH

Example of a parameter json file

{
  "MyEnvironment": "Production",
  "username": "Eggy@eggplant.com"
}

eggplant-runner-Windows-xx.x.xx.exe scriptbased <SERVER_URL> --test-config-name=<TEST_CONFIG_NAME> --suite-name=<SUITE_NAME> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET> --param-file="c:\EggplantCLI\TestConfigParameter.json"

--filter-by TEXT

Test config's step execution filters available:

Filter name	Example
step_status_include	`--filter-by step_status_include=Passed,Failed`
step_status_exclude	`--filter-by step_status_exclude=Error,Cancelled,Untested`
test_case_tag_include	`--filter-by test_case_tag_include=admin,tc_tag1`
test_case_tag_exclude	`--filter-by test_case_tag_exclude="logic module,critical"`
test_case_name_include	`--filter-by test_case_name_include=tc_001`
test_case_name_exclude	`--filter-by test_case_name_exclude=login_user`
scenario_tag	`--filter-by scenario_tag="@scea,@input_tag ~@order_page"`

eggplant-runner-Windows-xx.x.xx.exe <SERVER_URL> <TEST_CONFIG_ID> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET> --filter-by test_case_tag_exclude="*Ignored*, #hash#" --filter-by step_status_include=Passed,Failed

--filter-by-json PATH

Example of a test config's step execution filter json file

{
  "step_status": {
    "include": "Passed,Failed",
    "exclude": "Error,Cancelled,Untested"
  },
  "test_case_tag": {
    "include": "admin, tc_tag1",
    "exclude": "logic module, critical"
  },
  "test_case_name": {
    "include": "tc_001",
    "exclude": "login_user"
  },
  "scenario_tag": "@scea,@input_tag ~@order_page"
}

eggplant-runner-Windows-xx.x.xx.exe <SERVER_URL> <TEST_CONFIG_ID> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET> --filter-by-json="c:\EggplantCLI\TestConfigFilters.json"

--previous-task-instance-id <TASK_INSTANCE_ID>

eggplant-runner-Windows-xx.x.xx.exe <SERVER_URL> <TEST_CONFIG_ID> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET> --filter-by test_case_tag_exclude="*Ignored*, #hash#" --filter-by step_status_include=Passed,Failed --previous-task-instance-id b1cd8e74-6622-4086-9751-9631db3d3f00

Run Test Configuration by Test Config Id​

Run Test Configuration by Test Config Name​

Model based Test Configuration​

Script based Test Configuration​

Advanced Options​

Advanced Examples​

Run Test Configuration by Test Config Id

Run Test Configuration by Test Config Name

Model based Test Configuration

Script based Test Configuration

Advanced Options

Advanced Examples