Eggplant Performance 4 Release Notes

Features and Improvements (4.9.1)

• New 64-bit Injector allows mixing of 32-bit and 64-bit Virtual User Types in the same workspace/test (for rare cases such as a mix of Base Java VUs which load 64-bit native libraries with Web Java VUs in the same test).
• Adds a new Virtual User Group Runtime Ramp-Up option to specify the number of seconds/minutes for the total ramp-up time for a group; the intervals between individual Virtual User start times are calculated automatically.
• Updates to Forecast Analyzer due to the new ramp-up option introduced above. It is recommended to also update to the latest version of Forecast Analyzer.
• Updates to the Selenium Client & WebDriver Language Bindings to match the latest version (v2.32.0) as of 13/05/2013 (supports Firefox 20).
• Internal core updates to support the certificates required for mobile devices.
• Several bug fixes and improvements.

Features and Improvements (4.9.1)

• Beta Release of the Web C# Virtual User.

Features and Improvements (4.8.0)

• Selenium Java Virtual User
  • Introducing the new Selenium Java Virtual User, which uses Selenium WebDriver to automate web browsers. Selenium WebDriver provides an object-oriented API that provides a common interface for driving all of the major browsers (e.g. Internet Explorer, Firefox, Chrome, iPhone, etc.). It can also drive the HtmlUnit GUI-less browser, which is the fastest and most lightweight implementation of WebDriver, and hence is the most appropriate choice for a load testing application.
  • This release packages up the latest Selenium Client & WebDriver Language Bindings (v2.31.0, release date 2013-02-27) which is compatible with Mozilla Firefox (release 19). (Note: version 4.9.1 has an updated version of the bindings that supports Firefox 20).
  • If you need to install a later/earlier version of Firefox, you must install a different version of the Language Bindings in Studio.
• Virtual User types have now been categorized
  • As the list of supported protocols and Virtual User types continues to grow in Forecast Studio, we have now categorised the Virtual User types and provided some high level details for each type. The categories are:
    • Base
    • GUI
    • Network
    • Web
• Click on each category type in the Workspace view to see further details.

Features and Improvements (4.7.0)

This version contains many new features and a beta pre-release of the new Web Java Virtual User.

• Runtimes
  • Support ended for Visual Studio 2005
    • We have now ended support for Visual Studio 2005 (vc8 and cs2 profiles). This runtime and any options or references related to it have been removed. The update process (after archiving your workspace) will remove all references in your workspace.
  • Support added for Visual Studio 2012
    • We have now added support for Visual Studio 2012 (vc11 and cs4_5 profiles). The update process (after archiving your workspace) will add new references to your workspace.
    • Note that, for Windows XP and Server 2003, when acting as a controller machine - Visual Studio 2012 cannot be installed on these operating systems, so support is not available for compilation of these runtimes. However, Visual Studio 2012 redistributable is supported, so these operating systems can be used as Injectors for all runtimes.
• Default JRE packaged with Studio
  • We have added a default fully compatible 32-bit JDK (Java version 1.7) to the product for runtime of all java tests.
• Studio
  • New Enhanced Web Java Virtual User Type
    • Soon to be available in both Java and .NET, both based around a common core. For this beta release, only the Java version is available.
    • Writing scripts in Java allows for a better experience due to improved memory management, superior error reporting (runtime and compilation time), more code libraries are available, easier to program.
    • Highly scalable. Supports from one to tens of thousands of Virtual Users.
  • Unicode support. Forecast can now handle and display foreign language character sets without using encoded characters.
  • Designed to simplify the handling of asynchronous AJAX requests and complex applications.
  • An elegant and logical Object Oriented API.
  • Fully extensible and integrated with a standard Integrated Development Environment - Eclipse.
• Help
  • Adds a new format of the product Help documentation, which has changed to HTML.
  • New basic documentation (also available in Eclipse) for the New Enhanced Web Java Virtual User Type.

Features and Improvements (4.5.0)

This version contains several new features and is recommended for all users.

• Studio
  • Check for Updates / Get Support
    • A new menu option added to the Help menu, to check for the latest updates of all Facilita products has been added to Studio, together with a menu option that will open the Facilita Support Portal in your default browser. You can request a Facilita Support account (email us at eggplant.support@keysight.com) and then (subject to your support and maintenance contract) you can download the latest releases and browse through (and post to) new discussion groups, wiki pages and other documentation. The Facilita Support Portal will be constantly updated with new articles, demos and other features.
  • Touch Source
    • A new menu option added to the Build menu that "touches" all script source files with the current system time. Useful when re-building scripts in an environment that experiences frequent clock rollbacks.
  • Script Exports
    • Scripts can be individually exported into a self-contained archive. Script archives can then be imported into a different project in the same or a different workspace. See the Help for more details. Script archives can be sent to the Facilita Support Team as necessary.
  • Custom Virtual User Exports
    • Custom Virtual Users can be individually exported into a self-contained archive. Virtual User archives can then be imported into a different workspace. See the Help for more details. Custom Virtual User archives can be sent to the Facilita Support Team as necessary.
  • Test Run Results Exports
    • Test Run Results can be individually exported into a self-contained archive. Test Run Results archives can be sent to the Facilita Support Team as necessary.
• Web Recorder
  • The Web Proxy Recorder now excludes hosts by pattern matching. Previous exclusions were actioned through a mix of check boxes and pattern matches. Any existing host exclusions will be upgraded automatically to the new format.
• Web Viewer and Runtime Web Log Viewer
  • The two viewers have been merged into a single new viewer. New functionality allows real time web logs from remote Injectors to be viewed on the controller machine. New menu options in the viewer make it easier to compare the runtime and recording logs side by side. The number of iterations displayed in the runtime logs is now a configurable property of the test group.
• Documentation
  • New Citrix Virtual User guide added to the Studio Help.
  • New IP Virtual User guide added to the Studio Help.
• Test Controller
  • Clear Obsolete Transactions.
  • Added a button/menu item to the Transactions/Timings screen in Test Controller to clear the transactions/timings table.
• Analyzer
  • Regular Expression matching for Transactions.
  • The user can now group otherwise identical HTTP requests that vary only by the presence of a dynamic string within the path.
• Improved memory usage
  • Memory usage has been improved making the product more robust and able to cope with extremely large datasets.

Features and Improvements (4.4.0)

This version is a major release (recommended for all users).

• Studio
  • Web recording
    • When recording a web script there is a new list of applications that can be launched automatically by the Recorder. This list contains:
      • Google Chrome
      • Microsoft Internet Explorer
      • Mozilla Firefox
    • If any of these applications are not installed on the controller machine then they cannot be selected from the list.
    • When taking a web recording, a snapshot of the browser window is taken every time a transaction name is entered into the Recorder dialog. A thumbnail strip of snapshots has been added to the Recorder dialog. When the recording is viewed in the Web Viewer application, a new tab Snapshot displays the snapshot that was taken at the time of recording and matches the currently viewed transaction.
    • Another new feature when recording a web recording in the Hosts tab, which allows the de-selection of hostnames that have been encountered during the recording, and when the web script is generated these hosts will be ignored at runtime. The hostnames are persisted at the project level, so when another recording is taken the hosts that were excluded from the previous recording can also be excluded for the current recording (this is completely configurable from the Hosts tab in the Recorder). Hostnames can be deselected individually and by defining regular expressions.
    • Another new tab added to the Recorder is the 'Messages' tab, which displays a log of commands and any messages.
    • The 'Stop' (recording) button has been added to the Web Script Recorder dialog but removed from the underlying dialog, to simplify the user interface.
    • Transactions can be renamed while a script is being recorded.
  • Web Script generation
    • The protocol(s) for a generated web script have now been parameterised, making it easier to change the protocol(s) in each script.
    • In generated web scripts we now write setHeaderValue() calls for all non-standard headers encountered in the recording, ensuring they will be automatically set at runtime.
• NetBeans integration
  • If NetBeans is installed, it is possible to launch this IDE from within Studio and build the (java) Forecast project within NetBeans.
• dynaTrace integration (6.9.1, 7.x)
  • If dynaTrace is installed, there will be a new tab on the left-hand side of Studio named 'dynaTrace'. Various dynaTrace configuration options have been added to this new tab. The dynaTrace header can be set on a per Virtual User group basis by setting the new option on the 'Web: Logging' tab of the 'Runtime Settings' tab for the Virtual User group.
• Test export
  • This new feature within Forecast Studio allows tests to be exported as single .zip files that contain everything required to run the test. The .zip file can be extracted onto any target machine and run without having to install any other Facilita software.
  • The limitation is that only tests that are defined to run with a single Virtual User can be exported.
• New product icons, splash screen and logo.
• Communications
  • The communications between Test Controller/Injectors/Engines have been recoded to provide a more robust and reliable service.
  • Error message texts have been improved.
  • All files sent between Test Controller/Injectors/Engines are now zipped for transfer thereby decreasing transfer times.
  • A new status bar added showing the progress for the retrieval of the log files from the remote Injectors.
  • The default Injector and Engine port numbers have changed to ensure that we do not use ports in the dynamically allocated range (of ephemeral ports). This will resolve port conflicts with other running applications and provide smoother test executions. To see what this change complies with (and for further information) read this article.
  • New registry settings added to optimize the usage of TCP ports for all versions of windows. For a controller machine, run Optimize System Settings ... from the Tools menu in Test Controller. Reboot the Injector machine as appropriate to ensure the registry settings are applied.
  • For Injector machines, click forecastOptimalSettings_x86.reg or forecastOptimalSettings_x64.reg from within the installation folder. Reboot the Injector machine as appropriate to ensure the registry settings are applied.
  • When running a large number of Engines on an Injector, if there are any problems starting all of the Engines, increase the Injector 'Connect-back timeout for Engines' value to ensure that all Engines start correctly.
• Test Controller
  • We have added new 'Aborted' and 'Unknown' states to Test Controller which more clearly define the known statuses of the running Virtual Users.
  • The 'Last Alert' is now displayed at the top of the main window and is updated in real time.
• Analyzer
  • Analyzer will now warn (during database creation) if not all of the expected Virtual User log files are available for a test run.
• Miscellaneous
  • Many other issues resolved and many small additional features added.

Features and Improvements (4.3.0)

This major release has been superseded by 4.4.0, which is recommended for all users.

• Studio
  • The Studio graphical user interface has undergone a technology refresh and looks cleaner and crisper than before.
  • When loading a workspace that was created with a previous verson of Studio, an upgrader will automatically run and provides an automated backup function.
  • The 'Handle Dynamic URLs' web-script generation option has been removed.
  • The workspace and project are now built together. F8 performs a build and ALT+F8 performs a full rebuild. The options to build the project and workspace separately have been removed, whereas the ability to open each separately in an IDE has been maintained.
  • Much improved dynamic build output messaging windows, which can be maximised or hidden from view until needed (press SHIFT+ESCAPE or use context menu).
  • Whenever there is a compile-time error, clicking on an entry in these windows will open the script at the failing line in the Scite text editor.
  • New (clickable) build indicators added.
  • New build progress bar added.
• Web
  • The Internet Explorer Recorder has been removed, as has the old Proxy Recorder. These have been replaced by the new Fiddler Core Proxy Recorder, which provides an option to delete the Internet Explorer cache and files before recording starts.
  • New HTTP 'PUT' and 'DELETE' support.
  • Added support for the WinHTTP socket handler. This handles NTLM authenication better than WinInet. WinHTTP should be used in preference to WinInet for HTTP and HTTPS.
  • The setOptions() method call in web scripts has been replaced with a new pre() method.
• Other Protocols
  • Custom Virtual User Scripts now have pre() methods (all protocols).
  • New TCP recorder and generator. New scriptable UDP support (generator to be released soon).
• Test Controller
  • Now reloads changes to the workspace or tests automatically.
• Miscellaneous
  • Many further minor improvements and enhancements (>200).

Features and Improvements (4.2.3)

This version is a minor release, recommended for those users who wish to take advantage of the new Visual Studio 2010 support. There is no additional functionality.

• Support for Microsoft Visual Studio 2010 (full and express editions - express editions must be registered): all operating systems above (and including) Windows XP/2003 now fully support the building of workspaces and projects within Visual Studio 2010 (C++ and C#).
• Optional Injector update will send alerts to the Test Controller if the required .NET 4 frameworks are not installed on the Injector machine.
• Numerous bug clearances.

Features and Improvements (4.2.2)

This version is a maintenance release, providing numerous minor improvements and enhancements.

• setAllowOverlappingTransactions() and getAllowOverlappingTransactions() have been removed from the VirtualUser and VirtualUserScript classes. These methods are no longer valid as the transaction handling logic has changed and we no longer need to make this distinction.
• WinDriver support added for Windows Presentation Foundation (WPF).
• Enhancements to Cookie handling in web script generator. Will now accurately track client originated cookies set by client-side scripting as well as those in the cache when recording started.
• New VirtualUserScript methods to retrieve the elapsed time since a timing or transaction was started.
• Web ‘Content check’ rule type has been extended such that script code can be inserted before and after the navigation call when text is matched in the page source.
• Numerous bug clearances.

Features and Improvements (4.2.1)

• Injectors are now self-maintaining. Once an Injector has been installed on a machine, it should not require updating. When a test connects to an Injector, all appropriate libraries, data files and Engines will be transferred, meaning that when changes are made to the Engine code in the main product it is just the controller machine which will require updating, and the new Engines will automatically be transferred to the Injector machines.
• Numerous minor improvements and enhancements.

Features and Improvements (4.2.0)

• Transactions tab has been removed from web script generation options. The generator now auto-detects whether transactions exist.
• User defined metrics are collected at runtime, and have corresponding charts in Analyzer and Test Controller ('Average', 'Sum' and 'Rate').
• Web metrics are collected at runtime, and are written to the VU logs - from this additional data new charts have been created, and added to both Analyzer and Test Controller.
• Using higher resolution multimedia timer, instead of standard win32 timers, to improve the accuracy of timings.
• New 'filter' feature implemented for the event logs in Analyzer and Test Controller.
• Product now built against .NET 3.5.
• Web runtime: more efficient data compression support & direct access to this functionality from scripts.
• TCP/IP network traffic Virtual User support.
• Runtime supports "Allow overlapping transactions" functionality.
• Numerous minor improvements and enhancements.

Features and Improvements (4.1.1)

• Licensing has changed and is different to v4.1.0 and any previous release.
• Web Service scripts can now be generated from XML source files in Studio.
• A new .zip file allows for easier and faster installation of Injectors (C++ and Web) without the need to install the full Forecast product on the Injector machine.
• A new .msi file allows for easier and faster installation of Injectors (all protocols) without the need to install the full Forecast product on the Injector machine.
• Proxy Recorder SSL Certificate Handling - A change has been made in Release 4.1.1 so that the proxy recorder will now create a valid certificate for each new HTTPS site recorded. The certificate is issued by the Facilita Certification Authority. To prevent warnings from your web browser about this certificate you should import the Facilita Root Certificate as a Trusted Root Certificate Authority. Instructions for installing this certificate are shown on the Proxy Recorder Settings dialog, available from the Proxy Recorder wizard "Application Launcher" page. After the Facilita Root Certificate is installed as a trusted root certification authority, then no further browser warnings about certificates will be displayed. Also available on the Proxy Recorder Settings dialog is the option to set a path to a client certificate file. This is required if your web server requires a valid certificate to be sent from the client - which is the proxy recorder when recording HTTPS. Note that this client certificate must be in PEM format. The open source tool openssl.exe can be used to convert certificates to PEM format. This tool is delivered in the Forecast installation bin folder.
• Many fixes and minor improvements. See the Releases section of the website for details of these.

Features and Improvements (4.1.0)

• Licensing has changed. There is no longer a need for a 'runtime.lic' file. See Forecast Controller Installation and Licensing or the User Guide for more details.
• Web generation rules are now defined using the GUI. Rules can be applied at both the Virtual User and Script level. See the User Guide for more details. Old style (.ini) generation rules can still be processed but we recommend that you convert these to the new style.
• Application logs are now automatically flushed. It is no longer necessary to close Studio or Test Controller in order to flush the logs.
• An Analyzer SDK (.NET) that enables programmatic creation of Analyzer databases and views. Note that charting functionality is not yet available from the SDK.
• Framework mode has been removed. Custom VUs are now accessed through the Framework tab and built via the Build menu option.
• New Python APIs allow tests to be run from the console or from user created Python scripts.
• Import and export data with the Shared Data Server in .csv format.
• Workspaces can now only be opened by one user at a time.
• Many improvements to the handling of multiple Injectors/Engines.
• Studio will no longer start/stop the local Injector processes.
  • Test Controller will start any required local Injector processes and terminate them when no longer required.
  • The local Injector is no longer installed as a service by the installer. This does however mean that manual installation of the service on remote Injector machines is required. Do this by opening a command prompt and navigating to the %FORECAST4HOME%\bin\ folder. Here type the following to install the Injector service:
    • fcinjector -i
  • If for any reason you want to remove the Injector service use the -u switch. The Forecast installer is used to uninstall the release when Forecast has been removed using Add/Remove programs. This will not remove the Injector service. Therefore when upgrading a remote Injector from v3.0 or v4.0 it is unlikely that a manual install of the service will be required and likewise when upgrading a remote v4.1 Injector which has previously recieved a service installation from the command line.
• Numerous minor improvements and enhancements.

Migrating from Version 3 to Version 4

• A project upgrader is distributed with v4.0 to enable upgrading of projects from v3.0 to v4.0.
• Manual edits will be required to some existing scripts as a result of migration.
• Data Streams no longer exist within v4.0. Remove any #include "DataDictionaryStream.h" statements to ensure successful compilation.
• Data Streams and Data Tables have merged into one data source; namely Data Table. To achieve Data Stream functionality assign a Data Table at the VU Group level.
• Data Dictionary data source files can only be assigned at the Test level.

Migrating from Version 4.1.0 to Version 4.1.1 (if upgrading from an earlier release, read the appropriate section(s) above)

Licensing has changed, it will be necessary to perform the following steps:

1. De-register v4.1.0 (run <facilita_installation_dir>\bin\fcstudio unregister and follow the instructions provided by the dialogs.
2. Uninstall v4.1.0 (Control Panel > Add/remove programs).
3. Install v4.1.1 (a new hardware fingerprint will be generated) and forward the new hardware fingerprint to eggplant.support@keysight.com.
4. Install the new licenses issued by Facilita; instructions for this will be provided along with the new licenses.

Migrating from Version 4.1.1 to Version 4.2.1 (if upgrading from an earlier release, read the appropriate section(s) above)

All Injector machines should be upgraded to the new self-maintaining Injectors (see above for instructions on upgrading Injector machines).

Upgrading from Version 4.X to Version 4.2.3 (if upgrading from an earlier release, read the appropriate section(s) above)

• Workspaces must be updated to include the necessary supporting files for Visual Studio 2010. You should always archive your existing workspaces before upgrading to a new version of Forecast. In v4.2.3, whenever a new workspace is opened in Studio, an upgrade check will run automatically and issue a prompt, requesting whether to upgrade the workspace.
• The prompt will also remind you that this operation is one-way, so once a workspace has been upgraded to v4.2.3, it cannot be reverted back to a previous version. Upon encountering this prompt, you can either proceed or you can cancel the upgrade process, and your workspace will not be upgraded, allowing you to first archive your workspace and then begin the upgrade operation again.
• If you wish to run scripts compiled with Visual Studio 2010, you must either update each Injector machine with the latest v4.2.3 Injector release, or install the Visual Studio 2010 runtime.

Upgrading from Version 4.2.3 to Version 4.3.0 (if upgrading from an earlier release, read the appropriate section(s) above)

• When a workspace that was created with a version previous to v4.3.0 is opened, an automatic upgrader will run, and an automated backup process will run. An archive of your workspace is necessary as changes will be made that are not backward-compatible. Once the upgrader has completed the workspace will be opened. Whenever an upgrade to Studio is applied, please perform a full rebuild of your workspace and projects.

Upgrading from Version 4.3.0 to Version 4.4.0 (if upgrading from an earlier release, read the appropriate section(s) above)

• To upgrade to the latest version of Forecast Studio, all Injector machines must also be upgraded.
• The TCP/IP Virtual User has been superseded by the new IP Virtual User - if you have any custom Virtual Users or any scripts based on the existing TCP/IP Virtual User, please contact the Facilita support team before upgrading.

Upgrading from Version 4.4.0 to Version 4.5.0 (if upgrading from an earlier release, read the appropriate section(s) above)

• For controller machines, if upgrades are necessary the automated upgrader will run and prompt you to archive your workspace before it makes any changes.

Upgrading from Version 4.5.0 to Version 4.7.0 (if upgrading from an earlier release, read the appropriate section(s) above)

• The automated upgrader must run as there are many changes:
  • Removed support for Visual Studio 2005.
  • Added support for Visual Studio 2012.
  • The installer will add the new Web Java Virtual User type.

Upgrading from Version 4.7.0 to Version 4.8.0 (if upgrading from an earlier release, read the appropriate section(s) above)

• No changes required. The installer will add the new Selenium Virtual User type.

Upgrading from Version 4.8.0 to Version 4.9.1 (if upgrading from an earlier release, read the appropriate section(s) above)

• No changes required. The installer will add the new Web C# Virtual User type.

Upgrading from Version 4.9.1 to Version 4.9.1 (if upgrading from an earlier release, read the appropriate section(s) above)

• All users should upgrade to the latest version of Forecast Analyzer.

Upgrading from Version 4.9.x to Version 5.x (if upgrading from an earlier release, read the appropriate section(s) above)

• No changes required.

Known Issues and Limitations

• On Windows machines, after installation, you may need to change the permissions level of the 'forecast4 Injector Listener' service so that it will run under a local administrator account. By default the service will be setup to use the Local system account. If you receive any of the following errors when trying to execute a test you will need to change permissions level for the service.
  • C++ Engine
    • External data: Cannot load library C:\fcCache\%FORECAST4HOME%\bin\fc_vc10.dll
  • Java Engine
    • Pipe has ended: . NoClassDefFoundError
  • Steps for changing Injector service permissions
    1. From the Start menu, click Run.
    2. Enter services.msc.
    3. From the list of services, locate forecast4 Injector Listener, then right-click and select Properties.
    4. Select the Log on tab.
    5. Select 'this account', then enter a local administrator username and password.
    6. Click OK to save the changes.
  • Alternatively the Injector process can be started from a command line:
    • %FORECAST4HOME%\bin\fcinjector -e -p <port_number>
• Version 4.0 now executes project binary files from a File Transfer Repository. By default this is <ForecastInstallationDrive>\fcCache. For this reason, the user account which Forecast is installed under will need to have write access to the root of <ForecastInstallationDrive> (default is C:).
• The lightweight web page viewer used for the web log in RunMonitor and the web recording in Studio does not adequately display web pages that utilize style sheets.
• The OpenSSL default timeout is currently set to four minutes, and cannot be altered in this version.