Using the Eggplant DAI Runner

The Eggplant DAI Runner lets you run your pre-configured tests that were created in the Test Configuration tab and view the results of those tests via color coded dots. You can also schedule test runs from the Runner tab and see the tests that are in progress. To access the Runner tab, click Controller on the left menu of the Eggplant DAI interface.

Running Tests

To run a test, from the Runner tab, click the Run button against a test configuration that you want to run. Repeat this step if you need to run more tests.

note

Remember to re-upload any scripts that you may have changed manually via Eggplant Functional or in the Eggplant DAI Snippet Editor back to the Execution Environment (Manage Agents > Manage Suites) before running the test, or your changes may be overwritten.

Important

Note that when executed through a parent model, submodels will use the same suite as the parent model.

To stop a test configuration instance that's already in progress, click .

note

When you abort a test run that is part of a test configuration schedule, all runs that are part of that configuration schedule are aborted.

note

If a test execution is initiated (either at a scheduled time or by clicking the Run button) while another test execution for the same test configuration is in progress, then the newly-started test execution immediately completes with an error.

The Runner Interface displays the following information about your test configurations:

• Test Configuration: The names of your test configurations. This column can be sorted alphabetically.

• Model/Suite: A link to the Eggplant DAI model or suite used for the selected test configuration, each indicated by their respective icons to the left - where a building block represents a model and a folder represents suites.

• Schedule Type: The type of schedule for the test configuration. Displays No Schedule if none exists for a given test configuration. See Scheduling Options below for a list of schedule types.

• Scheduled Start: The date and time that a scheduled test run will start for a given configuration. This field is blank if no schedule exists for that test configuration.

• Previous Results: Shows the results of the ten most recent test executions (if applicable) represented via color-coded dots (described below). Hovering on an individual dot shows the status and date/time of the execution. Click to expand to see a snapshot of the results of that execution.

  Indicates that the test was successful.

  Indicates that the test has failed.

  Indicates that the test configuration is in progress.

  Indicates that the test resulted in error.

  Indicates that the test run was aborted.

See Viewing Execution Results for more information.

• Last Executed: The date and time of the most recent execution. Displays In Progress if a test execution is ongoing. This column can be sorted chronologically.

Viewing Execution Results

Accessing Detailed Test Execution Results
In the "Previous Results" column, you can gain insights into your test config execution by clicking on the individual run result icons. These icons represent various outcomes such as Pass, Fail, In progress, Error or Canceled. Clicking on them will expand a detailed summary of the test execution.

Test execution summary view
Within this summary, you'll find following information:

• Execution result: This indicates whether the test was successful or encountered issues.
• Run details: Details such as when the execution started, its duration, and information about the System Under Test (SUT).
• Steps completed: The number of steps successfully executed.
• Total reruns completed: This includes the total number of reruns performed across all steps.
• Total test runs completed: This encompasses the overall test runs, which includes both initial runs and any reruns that were necessary.

Donut chart
A donut chart provides a visual representation of your test configuration's results. The chart showcases the following:

• Passed first time: The number of test cases that passed on their initial attempt.
• Passed after rerun: Test cases that initially failed but ultimately passed after a rerun.
• Failed: Test cases that did not meet the success criteria.
• Errored: Test cases that encountered errors during execution.
• Cancelled: Test cases that were incomplete.
• In progress: Test cases that are currently running.

Please note that in cases involving multiple reruns, the donut chart will display only the final result of the rerun. To learn more about the re-run feature visit the following page: Test configuration

Test Configuration Logs
For a more in-depth understanding of the testing process and detailed test case execution results, you can refer to the Test Configuration Logs. These logs provide a record of the testing and relevant details.

Click Open in results to drill down into the results of the selected test execution. This opens the Results tab showing all the individual test runs for the selected test execution, but you can switch to the default All Runs view by clicking View all results link at the top of the display. Note that the Open in results button is available for all test results except for the runs that went into Error state.

Clicking the Error execution dot shows the reason for the error in the expanded view:

Deleting a Test Runs from Runner

In the Previous Results column, click the individual color-coded dots representing Success, Failure, In Progress , Error, Cancelled to view the summary of that test execution in an expanded view.

In the expanded view, click Open in Results to further drill down into the results of the selected test execution. This opens the Test Results tab showing all the individual test runs for the selected test execution.

At the top right corner of the screen is a red button labeled as Delete Run. Clicking on it will render a confirmation box. Click on Confirm to proceed with the deletion.

The instance of the selected test execution as well as all of its results will then be deleted.

Once deletion is completed, the screen will redirect back to Runner tab.

Scheduling Test Runs

Scheduling adds great flexibility to your test runs, enabling you to set your tests to run at specific intervals with the frequency you require.

note

If a test execution is initiated (either at a scheduled time or by clicking the Run button) while another test execution for the same test configuration is in progress, then the newly-started test execution immediately completes with an error.

Scheduling Options

Use the Test config schedule page shown above to set schedules for your test config runs. You can display the Test config schedule page using one of the following two methods:

• Display the Runner page (Controller > Runner), locate the test configuration you want to create a schedule for, and click its specified Schedule Type. (The default schedule type is No schedule.)
• Display the Runner page, locate the test configuration you want to create a schedule for, click its Actions menu (...), and select Edit.

DAI provides the following schedule types:

schedule type	Description
No Schedule (default)	Runs are not scheduled for this test config. You do not see the calendar date or time selectors on the Test config schedule page for this option.
Run once at selected time	Runs a test once at the selected date and time. Selecting this option displays the calendar date and time selectors shown above.
Run on selected then repeat every	Runs a test at the selected date, time, and frequency. This option displays the calendar and time selectors, and a Repeat every field with a drop-down list shown below where you can select Days (default), Minutes, Hours, or Weeks. Selecting this option runs the test for the first time at the selected date and time, and then repeatedly according to the frequency you specify.
Custom	Runs a test at a specified date and time, and provides the option to use Cron syntax in the Custom time interval field shown below. Selecting this option runs the test for the first time at the selected date and time, and then repeatedly according to the Cron syntax schedule you specify. DAI Cron syntax recognizes Monday as the first day of the week instead of Sunday as shown below in Custom Cron Schedule Examples.

Custom Cron Schedule Examples

# schedule executing at midnight from Monday to Friday \{#schedule-executing-at-midnight-from-monday-to-friday}
0 0 * * 0-4 

# schedule executing every weekday at 5:30PM \{#schedule-executing-every-weekday-at-530pm}
30 17 * * 0-4

note

For information about using Cron syntax, click the help (?) button next to the Custom time interval field. You can also see this article. If you are not familiar with using Cron syntax but would like to learn more, please contact our Customer Support.

Step-by-Step: Scheduling Tests

1. Display the Test config schedule page using one of the following two methods:

   • Display the Runner page (Controller > Runner), locate the test configuration you want to create a schedule for, and click its specified Schedule Type. (The default schedule type is No schedule.)
   • Display the Runner page, locate the test configuration you want to create a schedule for, click its Actions menu ..., and select Edit.

2. Select the radio button for the scheduling option you want:

   • Run once at selected time
   • Run on selected time and repeat every
   • Custom

   The Test config schedule page displays the scheduling option you selected and the Start date and time calendar date and time selectors.

   • If you choose Run on selected and the repeat every, you also see the Repeat every field and interval selector shown above in Scheduling Options.
   • If you choose Custom, you also see the Custom time interval field where you can enter Cron syntax as shown above in Scheduling Options.

3. Choose the date and time you want the test config to execute or execute for the first time, and specify the time interval or Cron syntax if appropriate for your selection.

4. Click Save to save your schedule or Cancel to discard it.

Viewing All Runs

You can view all runs within a test configuration, including any executions that didn't go through, runs that are in progress, and typical runs with usual pass or fail status. Viewing all runs within a test configuration is particularly useful to locate the details to determine the likely cause of a failed run.

Viewing Run History

From the Actions menu , select View All against the test configuration you want to view the history of. The results open in a separate interface displaying the following information with most recent runs:

• Execution Date: The date and time of the test configuration.
• Status: Shows the status of the test executions in color-coded capsule format. The options are:
  • Success
  • Failed
  • In Progress
  • Error
  • Canceled
• Open in Results: Click Open in Results to further drill down into the results of the selected test execution. This opens the Results tab showing all the individual test runs for the selected test execution, but you can switch to the default All Test Runs view by clicking View all results.
• Expand row button: Click to view a snapshot of the results of the selected execution.

Click the Runner navigation link to switch back to the Runner tab.

Deleting Test Runs from Run History

Click Open in Results to further drill down into the results of the selected test execution. This opens the Results tab showing all the individual test runs for the selected test execution.

At the top right corner of the screen is a red button labeled as Delete Run. Clicking on it will render a confirmation box. Click on Confirm to proceed with the deletion.

The instance of the selected test execution as well as all of its results will then be deleted.

Once deletion is completed, the screen will redirect back to Run History tab.

Set Retry Limits and Intervals for Connections to SUTs and Execution Environments

You can configure the DAI Runner to automatically retry failed test connections to a SUT or Execution Environment with the following environment variables:

• RETRY_SLEEP_INTERVAL_SECONDS

• RETRY_TIMEOUT_SECONDS

The DAI Server configuration file includes these variables by default. See Retries the connection for SUT and Execution Environments for explanations of these variables and instructions for modifying your DAI Server configuration file.