Technical Overview

The following steps provide an overview at a more technical level of how the Testimony product operates.

Installation Process

Testimony is installed as a third-party add-on solution to a central ABAP system (typically your SAP Solution Manager system). Transports are provided, which include code and enhancements that must be installed on the recording (Source) systems and the playback (Target) systems. Please see the link here to the installation guide.

Test Plan Configuration

All Testimony processes are executed within the organization of a “Test Plan”. These represent the regression test cycle(s) that are currently being performed for the necessary scenario (e.g. HANA DB migration, AWS re-platforming, technical upgrade etc). Within the central Testimony system, you must configure a test plan and provide some basic information relating to it (e.g. which the system(s) involved for recording and playback, which users are involved from a testing perspective). RFC destinations must be setup to allow Testimony to trigger both the recording and playback processes.

Recording Setup

A setup activity is then run to prepare the system to be recorded. After this point, the system is ready to enable you to activate the “recording” function in Testimony. Please note that until you perform this activation, there is zero impact on production service.

Recording Activation

When you are ready to perform a regression test cycle, the Testimony Administrator is able to activate the recording process. This will record all technical operations upon the SAP source system including activities such as SAP GUI dialog transactions and RFC / BAPI calls. Immediately after activating the recording, a backup of the source (SAP production) system should be taken. This will be used as the basis for the creation of the playback system. It is typically envisaged that a period of approximately 24 hours of operational activity would be recorded. At the end of the 24 hour recording period, the Testimony Administrator deactivates the recording process. Due to the unique manner in which Testimony operates, the impact on the production system is near zero (in terms of response times and system performance).

Coverage Analysis

The recording process has in effect automatically generated an entire test script library covering percentage of the customers actual SAP system usage. After the recording process is finished, the testing team can understand the level of coverage they have with the recording based upon the usage information of what the customer actually uses over the past M months (where M is typically 3 months). If key transactions are missing (e.g. month end processes) then the coverage analysis within Testimony can provide this information. In this case, you can either test these processes outside of Testimony or look to include the periods when these processes run in your next Testimony recording.

Playback Process

Once the recording and coverage process are run, then it is possible to execute the playback process. The playback (target) system that was built from the production database backup must now be started. The Testimony playback agent must be setup on the target system and “bots” run upon supporting infrastructure including Windows based machines. The bots simulate the actions of users as well as other external systems during the playback. Changes that are being made must be deployed to the target system (e.g. upgrade process, project release transports deployed, database migrated). Once the landscape is ready, the playback process can begin. This is simply a process of starting the “execution queue” and allowing it to execute (technically). If issues arise, notifications can be sent to appropriate test team members. The queue can be restarted if problems are detected.

Results Review

After the playback process has completed, the test team can review the results. They are specifically looking for issues that have arisen during the playback where the “expected output” differs from the “actual output”. Any discrepancies identified can be automatically generated as defects. The testing and development team can review these defects (once created) and provide a resolution. Once the bulk of defects are resolved, a new test cycle can be created and the system copy image restored. The playback process can be repeated to ensure that defects identified have been correctly resolved.