KPIs in Eggplant Performance Studio

In Eggplant Performance Studio, you can establish key performance indicators (KPIs) that are relevant to your data, and then apply those KPIs to your test runs so that you can see how your tested systems are performing relative to your objectives with an easy-to-read pass/fail interface.

KPIs are a set of measurements that help you track the strategic or operational goals for your systems under test (SUTs). They're typically going to represent the minimum (or maximum) acceptable value for any given statistic. However, what those goals (and values) actually are will vary depending on your specific needs and environment.

You might come up with additional KPIs over the course of development. You'll be able to establish and apply new KPIs to your data, as well as making additions or adjustment to existing KPIs as your testing progresses. For information about viewing KPIs while your test is running, see: Viewing KPI Statistics In Test Controller.For information about viewing KPIs on your analysis views and charts, see Viewing KPI Statistics Against your Test Runs.

Step by Step: Creating a KPI Collection

You set up KPIs from the Key Performance Indicators (KPIs) node in the Project tree. KPIs are created within KPI collections, which can then be applied to your tests. You define individual KPIs when you create the collection, but you'll be able to add or delete KPIs as well as edit existing KPIs after you create the collection.

To create a KPI collection in Eggplant Performance Studio, follow these steps:

1. Right-click KPIs in the Project tree, then select New KPI collection. When the KPIs node is selected in the Project tree, you can also click the New KPI collection button on the Eggplant Performance Studio toolbar.

2. In the Create KPI Collection wizard, enter a name for the KPI collection.

   Now you're ready to define specific KPIs to include in the collection.

   

3. Click the green plus (+) button on the right side of the window to open the Create KPI dialog box.

4. Enter a name for the KPI.

5. Define the KPI by selecting a category (left column), statistic (center column), and series to track (right column). Note that the statistics in the center column typically have additional values, which you reveal by clicking the triangle or double-clicking the item itself. You need to select a statistic from the sub-list; for example, for Concurrency, your options are Maximum, Mean, or Minimum. Click the green plus (+) button to add it to the list.

   

   Note that most KPIs can be monitored in real time while the test is running in Test Controller. For KPIs that cannot be monitored in real-time, Test Controller shows you a notification:

   .

6. In the Series Name pop-up window, enter the name of one of your series items that you want to base the KPI on. For instance, if you choose Injector as the series (third column), you would enter the name of a specific injector in this window. You can instead leave the default <COMBINED> to include all of the items in the series. Click OK.

7. (Optional) If you've chosen to list a series by individual names, click the plus button if you want to add additional items.

   note

   Each new item you add will be based on the same criteria; if you change or adjust the settings for category, statistic, or series, you're changing that information for each of the individual series you add.

8. Set the Bucket count. This setting limits the number of transactions that are plotted across the x-axis of a chart to the number set by the "bucket." Within each bucket, or division, all the data is averaged to provide the data that gets plotted on the chart. For KPIs, the bucket count can help remove anomalous readings that might give unreliable results.

9. Set the Target for the KPI. You can choose either less than or equal to (<=) or greater than or equal to (>=) from the drop-down list, then use the arrow keys to select a value or enter your desired value in the field. Remember, the target is what you consider the acceptable value that your test should meet or exceed for the statistic you're setting.

10. When you've set the details for the KPI, click OK.

11. (Optional) Repeat steps 3 through 10 to add additional KPIs to the collection. Note that you can edit or delete KPIs as you build the collection by using the buttons along the right side of the window.

12. When you've added all the KPIs you want, click OK to create the KPI collection.

Note that each KPI collection can include multiple KPIs. You can define multiple KPI collections to track different sets of statistics.

Editing KPI Collections

During the course of your testing, you might find that you need to add additional KPIs to your collections or adjust the values of KPIs that you're already using. You can make changes to your KPIs and KPI collections by selecting the KPI collection in the Project tree.

With the KPI collection selected, the Viewer pane displays a table view of the collection, listing each of the KPIs that make up the collection. By using the buttons at the right side of the table, you can add new KPIs, edit existing KPIs, or delete KPIs from the collection:

• To add a new KPI to the collection, click the green plus (+) button to open the Create KPI dialog box. Follow steps 3 through 10 above to create the new KPI and add it to the collection.
• To edit an existing KPI, select the KPI in the table in the Viewer pane, then click the Edit button. Make your changes in the Create KPI dialog box, then click OK.
• To remove a KPI from the collection, select it in the table, then click the red X (delete) button.