Test Results
In Controller > Test Results you can view the test results for all your models and scripts to make sure that your tests are running correctly. In addition, you can:
- Determine which of your tests have passed, failed, or are in progress.
- View the execution steps, logs, and screenshots for the selected test run.
- See which execution environment was used for each test run.
- See which system under test (SUT) was used for each test run.
- Delete test results that are no longer required.
- Abort tests that are in progress.
Filtering
By default, you see all your test results on the Test Results page. You can limit the results you see by applying filters. DAI provides several types of filters, which are described below.
The top of the Test Results page provides the following options:
- Search by ID: a search field where you can enter the exact ID of a test result you want to see. The results for each test execution have a unique ID generated by the Eggplant DAI database.
- Filters: a button you can click to show or hide the filter selection bar. The Filters button also shows the number of filters applied, if any.
- No filters applied or Clear all button depending on whether any filters are applied.
You might also see the filter selectors, which display in a bar below the Filters button when they are showing. Click the Filters button to show or hide the filter selection bar. For example, screen below shows the Suites filter in use with the selected suite name. You also see the Filters button showing the number 1
in purple, indicating one filter is applied.
The filter types are as follows:
By ID
Narrow the results shown to a specfic test result by entering the exact test result ID in the Search by ID field at the top of the Test Results page. The results for each test execution have a unique ID generated by the Eggplant DAI database.
You only see the following filter options when the filter selection bar is showing. Click the Filters button to see the filter selection bar if it is not showing.
By Run Status
Narrow the results shown by the run status. The run status options are as follows:
- Passed: successful tests are highlighted in green.
- Failed: unsuccessful tests are highlighted in red.
- In Progress: incomplete tests are highlighted in yellow.
- Cancelled: tests that were stopped before they could be completed.
- Error: tests that resulted in error as well as tests that were interrupted due to loss of network connectivity or the agent process ending abruptly.
The All Run Statuses filter option was removed in DAI 7.2. All of the run statuses display when no filters are applied.
By Date range
Narrow the results shown to a specific timeframe that matters most to you. Click Date Range to see the date filter:
- After: filters the test results to only those that exist after the selected date.
- Before: filters the test results to only those that exist before the selected date.
- Between: allows you to specify a range by selecting two dates.
- None: removes the date filter.
By Model
Narrow the results shown to a specific model by clicking the Models filter button and selecting the model you want from the list. For more information about models, see DAI Build a Model.
By Suite
Narrow the results shown to a specific suite by clicking the Suites filter button and selecting the suite you want from the list. Suites contain the SenseTalk snippets (or scripts) that you use in your DAI models or test cases.
Script-based test results generated before upgrading to DAI 6.4 will not appear when filtering by suite.
By Test Case
Narrow the results shown to a specific test case by clicking the Test case filter button and selecting the test case you want from the list. For more information about test cases, see Managing Test Cases.
If there are multiple test cases with the same name, the drop-down selector groups them into one selection, but it will be able to filter the results accordingly.
By Test Config
Narrow the results shown to a specific test configuration ("test config") by clicking the Test config button and selecting the test config you want from the list. For more information about test configs, see Working with Test Configurations.
By Test Config Tag
Narrow the results shown to a specific test configuration ("test config") tag. For more information about test configuration tags, see Working with Test Configurations.
By Execution Environment
Narrow the results shown to a specific execution environment. For more information about execution environments, see Environments.
By SUT
Narrow the results shown to a specific system under test (SUT). For more information about SUTs, see Managing SUTs in Eggplant DAI.
When you change the name of an existing SUT, DAI retains the ID of that SUT. If you filter by the new SUT name, you may see the original SUT name in the filtered results.
Clear All Filters
You can clear all filters at once by clicking the Clear all button, which is to the right of the Filters button at the top of the Test Results page. Once you click the Clear all button, you see No filters applied
in its place.
If no filters are applied, you do not see the Clear all button. You see no filters applied
in its place.
Results
The list of test results provides summary information and execution metrics for the results that match your criteria:
-
Status & ID: The status and ID show in the same column.
status:
The outcome of the test run. You can filter the results by Cancelled, Error, Failed, In Progress, or Passed statuses. Click the status to view the execution details for the test.ID:
The unique ID of the test result.
ノートThe Status & ID column sorts by run status, not ID. Prior to DAI 7.2, the Status and ID were separate columns.
-
Test case / Exploratory run: Type of test execution, either: Test case or Exploratory, Replay (a test case name for directed test cases), or a Script name in the case of a script-based exection. Exploratory or Replay display as a label.
-
Model / Suite: Model or suite used in the test run.
-
Test config / Live run: Test configuration to which the test belongs, or Live run. If the test config contains any tags, they display underneath.
Prior to DAI 7.2, this column's name was Run mode.
- Start date: Date and time when the test started running.
- Duration: The duration of the test runs, formatted in
hours
,minutes
, andseconds
.
If you export test results to a csv file, the Duration column is formatted in seconds in the file.
- Environment: The name of the execution environment used for the test run, or None if no execution environment was used.
- SUT: The system under test (SUT) used in the test run.
- Failed at: The last failed action for each failed test run.
If a test is in In Progress status, a stop button () appears in the row. You can stop a test run or test config runs using the button.
When you abort a test run that is part of a test configuration schedule, all runs that are part of that configuration schedule are aborted.
-
Actions menu : Provides the option to delete test runs. You can remove unwanted results, such as failed test results that remain in an errored state.
-
Delete: If the test result was for a Live Run, and not part of a test configuration run. This option deletes just the selected test run.
-
Batch Delete: If the test result was part of a test configuration run. This option will delete the results for all tests in that run of the test configuration. For more information, see Batch Delete below.
A test result is considered part of a test configuration if it has a value in its row under the Test Configuration column.
For more information, see Batch Delete below.
- View > (Details): to view the execution steps, log entries and screenshots for the selected test run.